ci: fix markdownlint/prettier checks and container pre-commit env
All checks were successful
CICD Start / Sanity and Base Decision (pull_request) Successful in 10s
All checks were successful
CICD Start / Sanity and Base Decision (pull_request) Successful in 10s
This commit is contained in:
@@ -6,13 +6,13 @@
|
||||
|
||||
## 📊 **Performance Metrics - Validated Results**
|
||||
|
||||
| Metric | Before Optimization | After Optimization | Improvement |
|
||||
|--------|-------------------|------------------|------------|
|
||||
| **Total Pipeline Time** | 15-25 minutes | 3-5 minutes | **85% faster** |
|
||||
| **Build Success Rate** | ~70% (various failures) | **100%** | **30% improvement** |
|
||||
| **E2E Test Reliability** | ~60% (browser issues) | **100%** | **40% improvement** |
|
||||
| **Resource Efficiency** | High CPU/memory load | Optimized usage | **Significant** |
|
||||
| **Developer Experience** | Frequent CI failures | Reliable pipeline | **Excellent** |
|
||||
| Metric | Before Optimization | After Optimization | Improvement |
|
||||
| ------------------------ | ----------------------- | ------------------ | ------------------- |
|
||||
| **Total Pipeline Time** | 15-25 minutes | 3-5 minutes | **85% faster** |
|
||||
| **Build Success Rate** | ~70% (various failures) | **100%** | **30% improvement** |
|
||||
| **E2E Test Reliability** | ~60% (browser issues) | **100%** | **40% improvement** |
|
||||
| **Resource Efficiency** | High CPU/memory load | Optimized usage | **Significant** |
|
||||
| **Developer Experience** | Frequent CI failures | Reliable pipeline | **Excellent** |
|
||||
|
||||
## 🔧 **Key Technical Achievements**
|
||||
|
||||
@@ -37,6 +37,7 @@
|
||||
## 🛠️ **Critical Issues Resolved**
|
||||
|
||||
### **Build Phase Issues**
|
||||
|
||||
1. **✅ README.md Dependency Error**
|
||||
- **Problem**: Local package build failed during dependency-only phase
|
||||
- **Solution**: Dummy file creation for minimal package structure
|
||||
@@ -53,6 +54,7 @@
|
||||
- **Impact**: 100% reliable frontend dependency management
|
||||
|
||||
### **Test Phase Issues**
|
||||
|
||||
1. **✅ E2E Docker Pull Complexity**
|
||||
- **Problem**: Over-engineered retry logic for E2E tests only
|
||||
- **Solution**: Use same simple approach as all other successful tests
|
||||
|
||||
@@ -50,11 +50,11 @@ RUN git clone full_repo && merge_preserving_deps # ✅ Source changes don't bus
|
||||
|
||||
3. **No rsync Available**: Base image doesn't include rsync for selective copying
|
||||
|
||||
```dockerfile
|
||||
# Fix: Use standard cp with backup strategy instead of rsync
|
||||
# rsync -av --exclude='node_modules' /tmp/fullrepo/ /workspace/ # ❌ Not available
|
||||
# Standard cp with manual exclusions # ✅ Works everywhere
|
||||
```
|
||||
```dockerfile
|
||||
# Fix: Use standard cp with backup strategy instead of rsync
|
||||
# rsync -av --exclude='node_modules' /tmp/fullrepo/ /workspace/ # ❌ Not available
|
||||
# Standard cp with manual exclusions # ✅ Works everywhere
|
||||
```
|
||||
|
||||
**Metrics**:
|
||||
|
||||
|
||||
@@ -36,7 +36,7 @@ Scope boundary:
|
||||
|
||||
- This repository separates contract definition from enforcement mechanics.
|
||||
- CI workflow rewiring and test execution redesign are out of scope for PP-58
|
||||
and belong to follow-up work under epic #66.
|
||||
and belong to follow-up work under epic #66.
|
||||
|
||||
## Quick Start
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ This document contains solutions to common issues with Gitea Actions CI/CD pipel
|
||||
## Critical Issue: Jobs Stuck in "Waiting" State Forever
|
||||
|
||||
### Symptoms
|
||||
|
||||
- Workflows are created but jobs show "Waiting" indefinitely
|
||||
- Runners are online and healthy
|
||||
- No tasks appear in `action_task` database table
|
||||
@@ -12,9 +13,11 @@ This document contains solutions to common issues with Gitea Actions CI/CD pipel
|
||||
- UI shows "Waiting" but database shows status 5 (cancelled)
|
||||
|
||||
### Root Cause
|
||||
|
||||
**Docker syntax in `runs-on` labels** causes Gitea Actions to immediately cancel jobs.
|
||||
|
||||
### Problem Syntax (BROKEN)
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
setup:
|
||||
@@ -26,6 +29,7 @@ jobs:
|
||||
```
|
||||
|
||||
### Solution Syntax (WORKING)
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
setup:
|
||||
@@ -37,7 +41,9 @@ jobs:
|
||||
```
|
||||
|
||||
### Why This Works
|
||||
|
||||
The runners are configured with Docker images in their labels:
|
||||
|
||||
```bash
|
||||
GITEA_RUNNER_LABELS=ubuntu-latest:docker://ubuntu:22.04,node-latest:docker://node:20-bookworm-slim,python-latest:docker://python:3.14-slim
|
||||
```
|
||||
@@ -47,11 +53,13 @@ So jobs still run in the correct Docker containers, but Gitea can properly parse
|
||||
### Diagnosis Steps
|
||||
|
||||
1. **Check if new runs are created:**
|
||||
|
||||
```sql
|
||||
SELECT id, status, title FROM action_run ORDER BY id DESC LIMIT 3;
|
||||
```
|
||||
|
||||
2. **Check job status and duration:**
|
||||
|
||||
```sql
|
||||
SELECT arj.id, arj.job_id, arj.status, ar.created, ar.updated, (ar.updated - ar.created) as duration_seconds
|
||||
FROM action_run_job arj
|
||||
@@ -60,21 +68,25 @@ WHERE ar.id = (SELECT MAX(id) FROM action_run);
|
||||
```
|
||||
|
||||
3. **Check if tasks are created:**
|
||||
|
||||
```sql
|
||||
SELECT * FROM action_task ORDER BY id DESC LIMIT 5;
|
||||
```
|
||||
|
||||
4. **Verify runners are online:**
|
||||
|
||||
```sql
|
||||
SELECT id, name, last_online, agent_labels FROM action_runner WHERE last_online > (EXTRACT(epoch FROM NOW()) - 300)::bigint;
|
||||
```
|
||||
|
||||
### Key Indicators
|
||||
|
||||
- **Duration = 0 seconds** → Immediate cancellation due to syntax issue
|
||||
- **Empty action_task table** → Jobs never converted to executable tasks
|
||||
- **Status 5 jobs with Status 7 dependents** → Setup job cancelled, others skipped
|
||||
|
||||
### Test Procedure
|
||||
|
||||
Create a minimal test workflow to isolate issues:
|
||||
|
||||
```yaml
|
||||
@@ -95,13 +107,17 @@ If this works but your main workflow doesn't, the issue is likely syntax-related
|
||||
## Other Common Issues
|
||||
|
||||
### Cache/UI Synchronization Problems
|
||||
|
||||
If UI shows different status than database:
|
||||
|
||||
1. Restart Gitea: `docker compose restart server`
|
||||
2. Clear browser cache
|
||||
3. Check database vs UI status discrepancies
|
||||
|
||||
### Stuck Runs from Previous Sessions
|
||||
|
||||
Clean up stuck runs:
|
||||
|
||||
```sql
|
||||
-- Clear stuck pending jobs
|
||||
UPDATE action_run_job SET status = 5 WHERE status IN (1, 2);
|
||||
@@ -109,7 +125,9 @@ UPDATE action_run SET status = 5 WHERE status IN (1, 2);
|
||||
```
|
||||
|
||||
### Runner Registration Issues
|
||||
|
||||
If runners show "unregistered runner" errors:
|
||||
|
||||
1. Delete runner registrations: `DELETE FROM action_runner;`
|
||||
2. Restart all runner containers
|
||||
3. Let them auto-register with fresh state
|
||||
@@ -117,6 +135,7 @@ If runners show "unregistered runner" errors:
|
||||
## Infrastructure Overview
|
||||
|
||||
### Current Setup
|
||||
|
||||
- **Gitea Server**: Docker container with PostgreSQL backend
|
||||
- **Runners**: 8 Raspberry Pi runners across 4 servers
|
||||
- pi-desktop: Pi 400 4GB (2 runners)
|
||||
@@ -125,16 +144,20 @@ If runners show "unregistered runner" errors:
|
||||
- zhokq: Pi 4B 8GB (2 runners)
|
||||
|
||||
### Runner Configuration
|
||||
|
||||
Each runner supports multiple Docker environments:
|
||||
|
||||
- `ubuntu-latest` → `ubuntu:22.04`
|
||||
- `python-latest` → `python:3.14-slim`
|
||||
- `node-latest` → `node:20-bookworm-slim`
|
||||
- `ubuntu-act` → `catthehacker/ubuntu:act-latest`
|
||||
|
||||
### Mirroring the `ubuntu-act` Runner Image
|
||||
|
||||
If GHCR pulls are flaky, mirror the runner image into your local registry and point the label at that mirror instead of the upstream tag.
|
||||
|
||||
Example mirror flow:
|
||||
|
||||
```bash
|
||||
docker pull ghcr.io/catthehacker/ubuntu:act-latest
|
||||
docker tag ghcr.io/catthehacker/ubuntu:act-latest kankali.darkhelm.lan:3001/darkhelm.org/act-ubuntu:act-latest
|
||||
@@ -142,6 +165,7 @@ docker push kankali.darkhelm.lan:3001/darkhelm.org/act-ubuntu:act-latest
|
||||
```
|
||||
|
||||
Recommended runner label once mirrored:
|
||||
|
||||
```bash
|
||||
GITEA_RUNNER_LABELS=ubuntu-latest:docker://ubuntu:22.04,node-latest:docker://node:20-bookworm-slim,python-latest:docker://python:3.14-slim,ubuntu-act:docker://kankali.darkhelm.lan:3001/darkhelm.org/act-ubuntu:act-latest
|
||||
```
|
||||
@@ -164,7 +188,9 @@ source scripts/gitea-actions/check_runner_images.xsh
|
||||
```
|
||||
|
||||
### Workflow Design
|
||||
|
||||
Multi-stage pipeline with artifact passing:
|
||||
|
||||
1. **Setup**: Checkout code, create artifacts
|
||||
2. **Parallel Setup**: Backend (Python/uv) + Frontend (Node.js/Yarn)
|
||||
3. **Parallel Tests**: Backend tests + Frontend tests
|
||||
@@ -179,5 +205,5 @@ Multi-stage pipeline with artifact passing:
|
||||
|
||||
---
|
||||
|
||||
*Last updated: June 2, 2026*
|
||||
*Issue resolved after extensive database-level debugging and syntax isolation*
|
||||
_Last updated: June 2, 2026_
|
||||
_Issue resolved after extensive database-level debugging and syntax isolation_
|
||||
|
||||
@@ -18,6 +18,7 @@ poe
|
||||
## 📋 Essential Tasks
|
||||
|
||||
### Development Environment
|
||||
|
||||
```bash
|
||||
poe dev # Start development environment (Docker Compose)
|
||||
poe dev-down # Stop development environment
|
||||
@@ -26,6 +27,7 @@ poe dev-restart # Restart development environment
|
||||
```
|
||||
|
||||
### Code Quality (Unified Backend + Frontend)
|
||||
|
||||
```bash
|
||||
poe format # Format all code (Python + TypeScript)
|
||||
poe lint # Lint all code (Python + TypeScript)
|
||||
@@ -33,6 +35,7 @@ poe type-check # Type check all code (Python + TypeScript)
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
```bash
|
||||
poe test-unit # Run all unit tests (backend + frontend)
|
||||
poe test-all # Run all tests including integration
|
||||
@@ -41,6 +44,7 @@ poe test-e2e # Run end-to-end tests only
|
||||
```
|
||||
|
||||
### CI/CD Pipeline
|
||||
|
||||
```bash
|
||||
poe ci-quick # Fast quality checks (format, lint, type-check)
|
||||
poe ci-full # Complete CI pipeline simulation
|
||||
@@ -48,6 +52,7 @@ poe quality-gate # All quality checks (like CI)
|
||||
```
|
||||
|
||||
### Docker Images
|
||||
|
||||
```bash
|
||||
poe build-cicd # Build both base and complete CI/CD images
|
||||
poe build-cicd-base # Build only base image (cached dependencies)
|
||||
@@ -81,6 +86,7 @@ poe reset # Complete reset (clean + reinstall)
|
||||
## 🔧 Individual Component Tasks
|
||||
|
||||
### Backend Only
|
||||
|
||||
```bash
|
||||
poe format-backend # Format Python code only
|
||||
poe lint-backend # Lint Python code only
|
||||
@@ -90,6 +96,7 @@ poe test-backend-cov # Backend tests with coverage
|
||||
```
|
||||
|
||||
### Frontend Only
|
||||
|
||||
```bash
|
||||
poe format-frontend # Format TypeScript code only
|
||||
poe lint-frontend # Lint TypeScript code only
|
||||
@@ -116,6 +123,7 @@ poe pre-commit-update # Update hook versions
|
||||
## 💡 Tips & Tricks
|
||||
|
||||
### Task Discovery
|
||||
|
||||
```bash
|
||||
poe --help # List all tasks with descriptions
|
||||
poe <task> --help # Get help for specific task
|
||||
@@ -123,6 +131,7 @@ poe # Interactive task picker
|
||||
```
|
||||
|
||||
### Chaining Tasks
|
||||
|
||||
```bash
|
||||
# Run multiple tasks in sequence
|
||||
poe format lint type-check test-unit
|
||||
@@ -132,12 +141,14 @@ poe clean deps-install ci-quick
|
||||
```
|
||||
|
||||
### Environment Context
|
||||
|
||||
- All tasks run from project root
|
||||
- Backend tasks automatically use `uv run` in correct environment
|
||||
- Frontend tasks automatically use `yarn` in correct directory
|
||||
- Docker tasks use the optimized multi-stage CI/CD setup
|
||||
|
||||
### Performance Tips
|
||||
|
||||
- Use parallel tasks (`*-parallel`) for faster feedback
|
||||
- Use conditional tasks (`*-if-changed`) to save time
|
||||
- `poe setup` configures everything for new developers
|
||||
@@ -146,6 +157,7 @@ poe clean deps-install ci-quick
|
||||
## 🔄 Migration from Manual Commands
|
||||
|
||||
### Before (Manual)
|
||||
|
||||
```bash
|
||||
cd backend && uv run ruff format .
|
||||
cd frontend && yarn prettier --write src/
|
||||
@@ -155,6 +167,7 @@ cd frontend && yarn test
|
||||
```
|
||||
|
||||
### After (Poe)
|
||||
|
||||
```bash
|
||||
poe format
|
||||
poe test-unit
|
||||
|
||||
@@ -84,7 +84,7 @@ Add Renovate to your existing Gitea Actions workflow:
|
||||
name: Renovate
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 8 * * 1' # Monday 8 AM
|
||||
- cron: "0 8 * * 1" # Monday 8 AM
|
||||
workflow_dispatch: # Manual trigger
|
||||
|
||||
jobs:
|
||||
@@ -163,6 +163,7 @@ Once active, Renovate will:
|
||||
### 3. Integration with CI/CD
|
||||
|
||||
Renovate PRs will trigger your existing CI/CD pipeline:
|
||||
|
||||
- Build and test in Docker containers
|
||||
- Run full quality gates (linting, type checking, tests)
|
||||
- Only merge if all checks pass
|
||||
@@ -172,6 +173,7 @@ Renovate PRs will trigger your existing CI/CD pipeline:
|
||||
### Dashboard
|
||||
|
||||
Renovate creates a "Dependency Dashboard" issue showing:
|
||||
|
||||
- Pending updates
|
||||
- Failed PRs
|
||||
- Ignored dependencies
|
||||
@@ -180,6 +182,7 @@ Renovate creates a "Dependency Dashboard" issue showing:
|
||||
### Logs and Debugging
|
||||
|
||||
For self-hosted setup:
|
||||
|
||||
```bash
|
||||
# Run with debug logging
|
||||
docker run --rm \
|
||||
@@ -204,6 +207,7 @@ docker run --rm \
|
||||
### Quick Validation
|
||||
|
||||
For basic JSON validation without installing Renovate:
|
||||
|
||||
```bash
|
||||
# Quick syntax check (no Renovate installation needed)
|
||||
./scripts/quick-renovate-check.sh
|
||||
@@ -265,6 +269,7 @@ For basic JSON validation without installing Renovate:
|
||||
---
|
||||
|
||||
**Related Documentation**:
|
||||
|
||||
- [Renovate Official Docs](https://docs.renovatebot.com/)
|
||||
- [Configuration Options](https://docs.renovatebot.com/configuration-options/)
|
||||
- [Package Rules](https://docs.renovatebot.com/configuration-options/#packagerules)
|
||||
|
||||
@@ -10,6 +10,7 @@ Historically, floating dependency constraints and non-enforced runtime assumptio
|
||||
introduce drift and hard-to-diagnose failures.
|
||||
|
||||
This branch introduced:
|
||||
|
||||
- Python 3.14 as the required runtime baseline
|
||||
- Exact dependency pinning for backend runtime and development tooling
|
||||
- Startup compatibility checks that fail fast when runtime policy is violated
|
||||
@@ -28,11 +29,13 @@ Adopt a deterministic backend runtime policy:
|
||||
## Consequences
|
||||
|
||||
Positive:
|
||||
|
||||
- Reduced environment drift across dev/CI/prod
|
||||
- Earlier and clearer failure mode for runtime mismatches
|
||||
- Improved reproducibility and troubleshooting
|
||||
|
||||
Negative:
|
||||
|
||||
- More frequent explicit dependency maintenance updates
|
||||
- Stricter upgrade process for Python/runtime packages
|
||||
|
||||
|
||||
@@ -27,13 +27,14 @@ Adopt a hardened split workflow for the CICD base image:
|
||||
3. Treat the hash-specific tag as the source of truth for all CI consumers.
|
||||
4. Keep `latest` only as a convenience tag for humans and manual debugging.
|
||||
5. Make the main CI workflow consume only the immutable hash tag and fail clearly
|
||||
if that base image has not been published yet.
|
||||
if that base image has not been published yet.
|
||||
6. Add bounded polling in main CI to tolerate short publish/consume races between
|
||||
the dedicated base workflow and the main workflow.
|
||||
the dedicated base workflow and the main workflow.
|
||||
|
||||
## Consequences
|
||||
|
||||
Positive:
|
||||
|
||||
- Faster CI via stable base-layer reuse
|
||||
- Better traceability from base image to Dockerfile content
|
||||
- Single publish, many pulls across the runner fleet
|
||||
@@ -41,6 +42,7 @@ Positive:
|
||||
- Clearer separation of concerns between artifact publication and application CI
|
||||
|
||||
Negative:
|
||||
|
||||
- Slightly more workflow complexity
|
||||
- Registry stores additional hash-tagged images
|
||||
- Main CI now fails fast when the expected base image is missing instead of
|
||||
|
||||
Reference in New Issue
Block a user