ci: fix markdownlint/prettier checks and container pre-commit env
All checks were successful
CICD Start / Sanity and Base Decision (pull_request) Successful in 10s

This commit is contained in:
copilotcoder
2026-06-19 13:04:52 -04:00
parent 6f8dfea4cb
commit 654ba6ec25
20 changed files with 251 additions and 171 deletions

View File

@@ -6,13 +6,13 @@
## 📊 **Performance Metrics - Validated Results**
| Metric | Before Optimization | After Optimization | Improvement |
|--------|-------------------|------------------|------------|
| **Total Pipeline Time** | 15-25 minutes | 3-5 minutes | **85% faster** |
| **Build Success Rate** | ~70% (various failures) | **100%** | **30% improvement** |
| **E2E Test Reliability** | ~60% (browser issues) | **100%** | **40% improvement** |
| **Resource Efficiency** | High CPU/memory load | Optimized usage | **Significant** |
| **Developer Experience** | Frequent CI failures | Reliable pipeline | **Excellent** |
| Metric | Before Optimization | After Optimization | Improvement |
| ------------------------ | ----------------------- | ------------------ | ------------------- |
| **Total Pipeline Time** | 15-25 minutes | 3-5 minutes | **85% faster** |
| **Build Success Rate** | ~70% (various failures) | **100%** | **30% improvement** |
| **E2E Test Reliability** | ~60% (browser issues) | **100%** | **40% improvement** |
| **Resource Efficiency** | High CPU/memory load | Optimized usage | **Significant** |
| **Developer Experience** | Frequent CI failures | Reliable pipeline | **Excellent** |
## 🔧 **Key Technical Achievements**
@@ -37,6 +37,7 @@
## 🛠️ **Critical Issues Resolved**
### **Build Phase Issues**
1. **✅ README.md Dependency Error**
- **Problem**: Local package build failed during dependency-only phase
- **Solution**: Dummy file creation for minimal package structure
@@ -53,6 +54,7 @@
- **Impact**: 100% reliable frontend dependency management
### **Test Phase Issues**
1. **✅ E2E Docker Pull Complexity**
- **Problem**: Over-engineered retry logic for E2E tests only
- **Solution**: Use same simple approach as all other successful tests

View File

@@ -50,11 +50,11 @@ RUN git clone full_repo && merge_preserving_deps # ✅ Source changes don't bus
3. **No rsync Available**: Base image doesn't include rsync for selective copying
```dockerfile
# Fix: Use standard cp with backup strategy instead of rsync
# rsync -av --exclude='node_modules' /tmp/fullrepo/ /workspace/ # ❌ Not available
# Standard cp with manual exclusions # ✅ Works everywhere
```
```dockerfile
# Fix: Use standard cp with backup strategy instead of rsync
# rsync -av --exclude='node_modules' /tmp/fullrepo/ /workspace/ # ❌ Not available
# Standard cp with manual exclusions # ✅ Works everywhere
```
**Metrics**:

View File

@@ -36,7 +36,7 @@ Scope boundary:
- This repository separates contract definition from enforcement mechanics.
- CI workflow rewiring and test execution redesign are out of scope for PP-58
and belong to follow-up work under epic #66.
and belong to follow-up work under epic #66.
## Quick Start

View File

@@ -5,6 +5,7 @@ This document contains solutions to common issues with Gitea Actions CI/CD pipel
## Critical Issue: Jobs Stuck in "Waiting" State Forever
### Symptoms
- Workflows are created but jobs show "Waiting" indefinitely
- Runners are online and healthy
- No tasks appear in `action_task` database table
@@ -12,9 +13,11 @@ This document contains solutions to common issues with Gitea Actions CI/CD pipel
- UI shows "Waiting" but database shows status 5 (cancelled)
### Root Cause
**Docker syntax in `runs-on` labels** causes Gitea Actions to immediately cancel jobs.
### Problem Syntax (BROKEN)
```yaml
jobs:
setup:
@@ -26,6 +29,7 @@ jobs:
```
### Solution Syntax (WORKING)
```yaml
jobs:
setup:
@@ -37,7 +41,9 @@ jobs:
```
### Why This Works
The runners are configured with Docker images in their labels:
```bash
GITEA_RUNNER_LABELS=ubuntu-latest:docker://ubuntu:22.04,node-latest:docker://node:20-bookworm-slim,python-latest:docker://python:3.14-slim
```
@@ -47,11 +53,13 @@ So jobs still run in the correct Docker containers, but Gitea can properly parse
### Diagnosis Steps
1. **Check if new runs are created:**
```sql
SELECT id, status, title FROM action_run ORDER BY id DESC LIMIT 3;
```
2. **Check job status and duration:**
```sql
SELECT arj.id, arj.job_id, arj.status, ar.created, ar.updated, (ar.updated - ar.created) as duration_seconds
FROM action_run_job arj
@@ -60,21 +68,25 @@ WHERE ar.id = (SELECT MAX(id) FROM action_run);
```
3. **Check if tasks are created:**
```sql
SELECT * FROM action_task ORDER BY id DESC LIMIT 5;
```
4. **Verify runners are online:**
```sql
SELECT id, name, last_online, agent_labels FROM action_runner WHERE last_online > (EXTRACT(epoch FROM NOW()) - 300)::bigint;
```
### Key Indicators
- **Duration = 0 seconds** → Immediate cancellation due to syntax issue
- **Empty action_task table** → Jobs never converted to executable tasks
- **Status 5 jobs with Status 7 dependents** → Setup job cancelled, others skipped
### Test Procedure
Create a minimal test workflow to isolate issues:
```yaml
@@ -95,13 +107,17 @@ If this works but your main workflow doesn't, the issue is likely syntax-related
## Other Common Issues
### Cache/UI Synchronization Problems
If UI shows different status than database:
1. Restart Gitea: `docker compose restart server`
2. Clear browser cache
3. Check database vs UI status discrepancies
### Stuck Runs from Previous Sessions
Clean up stuck runs:
```sql
-- Clear stuck pending jobs
UPDATE action_run_job SET status = 5 WHERE status IN (1, 2);
@@ -109,7 +125,9 @@ UPDATE action_run SET status = 5 WHERE status IN (1, 2);
```
### Runner Registration Issues
If runners show "unregistered runner" errors:
1. Delete runner registrations: `DELETE FROM action_runner;`
2. Restart all runner containers
3. Let them auto-register with fresh state
@@ -117,6 +135,7 @@ If runners show "unregistered runner" errors:
## Infrastructure Overview
### Current Setup
- **Gitea Server**: Docker container with PostgreSQL backend
- **Runners**: 8 Raspberry Pi runners across 4 servers
- pi-desktop: Pi 400 4GB (2 runners)
@@ -125,16 +144,20 @@ If runners show "unregistered runner" errors:
- zhokq: Pi 4B 8GB (2 runners)
### Runner Configuration
Each runner supports multiple Docker environments:
- `ubuntu-latest``ubuntu:22.04`
- `python-latest``python:3.14-slim`
- `node-latest``node:20-bookworm-slim`
- `ubuntu-act``catthehacker/ubuntu:act-latest`
### Mirroring the `ubuntu-act` Runner Image
If GHCR pulls are flaky, mirror the runner image into your local registry and point the label at that mirror instead of the upstream tag.
Example mirror flow:
```bash
docker pull ghcr.io/catthehacker/ubuntu:act-latest
docker tag ghcr.io/catthehacker/ubuntu:act-latest kankali.darkhelm.lan:3001/darkhelm.org/act-ubuntu:act-latest
@@ -142,6 +165,7 @@ docker push kankali.darkhelm.lan:3001/darkhelm.org/act-ubuntu:act-latest
```
Recommended runner label once mirrored:
```bash
GITEA_RUNNER_LABELS=ubuntu-latest:docker://ubuntu:22.04,node-latest:docker://node:20-bookworm-slim,python-latest:docker://python:3.14-slim,ubuntu-act:docker://kankali.darkhelm.lan:3001/darkhelm.org/act-ubuntu:act-latest
```
@@ -164,7 +188,9 @@ source scripts/gitea-actions/check_runner_images.xsh
```
### Workflow Design
Multi-stage pipeline with artifact passing:
1. **Setup**: Checkout code, create artifacts
2. **Parallel Setup**: Backend (Python/uv) + Frontend (Node.js/Yarn)
3. **Parallel Tests**: Backend tests + Frontend tests
@@ -179,5 +205,5 @@ Multi-stage pipeline with artifact passing:
---
*Last updated: June 2, 2026*
*Issue resolved after extensive database-level debugging and syntax isolation*
_Last updated: June 2, 2026_
_Issue resolved after extensive database-level debugging and syntax isolation_

View File

@@ -18,6 +18,7 @@ poe
## 📋 Essential Tasks
### Development Environment
```bash
poe dev # Start development environment (Docker Compose)
poe dev-down # Stop development environment
@@ -26,6 +27,7 @@ poe dev-restart # Restart development environment
```
### Code Quality (Unified Backend + Frontend)
```bash
poe format # Format all code (Python + TypeScript)
poe lint # Lint all code (Python + TypeScript)
@@ -33,6 +35,7 @@ poe type-check # Type check all code (Python + TypeScript)
```
### Testing
```bash
poe test-unit # Run all unit tests (backend + frontend)
poe test-all # Run all tests including integration
@@ -41,6 +44,7 @@ poe test-e2e # Run end-to-end tests only
```
### CI/CD Pipeline
```bash
poe ci-quick # Fast quality checks (format, lint, type-check)
poe ci-full # Complete CI pipeline simulation
@@ -48,6 +52,7 @@ poe quality-gate # All quality checks (like CI)
```
### Docker Images
```bash
poe build-cicd # Build both base and complete CI/CD images
poe build-cicd-base # Build only base image (cached dependencies)
@@ -81,6 +86,7 @@ poe reset # Complete reset (clean + reinstall)
## 🔧 Individual Component Tasks
### Backend Only
```bash
poe format-backend # Format Python code only
poe lint-backend # Lint Python code only
@@ -90,6 +96,7 @@ poe test-backend-cov # Backend tests with coverage
```
### Frontend Only
```bash
poe format-frontend # Format TypeScript code only
poe lint-frontend # Lint TypeScript code only
@@ -116,6 +123,7 @@ poe pre-commit-update # Update hook versions
## 💡 Tips & Tricks
### Task Discovery
```bash
poe --help # List all tasks with descriptions
poe <task> --help # Get help for specific task
@@ -123,6 +131,7 @@ poe # Interactive task picker
```
### Chaining Tasks
```bash
# Run multiple tasks in sequence
poe format lint type-check test-unit
@@ -132,12 +141,14 @@ poe clean deps-install ci-quick
```
### Environment Context
- All tasks run from project root
- Backend tasks automatically use `uv run` in correct environment
- Frontend tasks automatically use `yarn` in correct directory
- Docker tasks use the optimized multi-stage CI/CD setup
### Performance Tips
- Use parallel tasks (`*-parallel`) for faster feedback
- Use conditional tasks (`*-if-changed`) to save time
- `poe setup` configures everything for new developers
@@ -146,6 +157,7 @@ poe clean deps-install ci-quick
## 🔄 Migration from Manual Commands
### Before (Manual)
```bash
cd backend && uv run ruff format .
cd frontend && yarn prettier --write src/
@@ -155,6 +167,7 @@ cd frontend && yarn test
```
### After (Poe)
```bash
poe format
poe test-unit

View File

@@ -84,7 +84,7 @@ Add Renovate to your existing Gitea Actions workflow:
name: Renovate
on:
schedule:
- cron: '0 8 * * 1' # Monday 8 AM
- cron: "0 8 * * 1" # Monday 8 AM
workflow_dispatch: # Manual trigger
jobs:
@@ -163,6 +163,7 @@ Once active, Renovate will:
### 3. Integration with CI/CD
Renovate PRs will trigger your existing CI/CD pipeline:
- Build and test in Docker containers
- Run full quality gates (linting, type checking, tests)
- Only merge if all checks pass
@@ -172,6 +173,7 @@ Renovate PRs will trigger your existing CI/CD pipeline:
### Dashboard
Renovate creates a "Dependency Dashboard" issue showing:
- Pending updates
- Failed PRs
- Ignored dependencies
@@ -180,6 +182,7 @@ Renovate creates a "Dependency Dashboard" issue showing:
### Logs and Debugging
For self-hosted setup:
```bash
# Run with debug logging
docker run --rm \
@@ -204,6 +207,7 @@ docker run --rm \
### Quick Validation
For basic JSON validation without installing Renovate:
```bash
# Quick syntax check (no Renovate installation needed)
./scripts/quick-renovate-check.sh
@@ -265,6 +269,7 @@ For basic JSON validation without installing Renovate:
---
**Related Documentation**:
- [Renovate Official Docs](https://docs.renovatebot.com/)
- [Configuration Options](https://docs.renovatebot.com/configuration-options/)
- [Package Rules](https://docs.renovatebot.com/configuration-options/#packagerules)

View File

@@ -10,6 +10,7 @@ Historically, floating dependency constraints and non-enforced runtime assumptio
introduce drift and hard-to-diagnose failures.
This branch introduced:
- Python 3.14 as the required runtime baseline
- Exact dependency pinning for backend runtime and development tooling
- Startup compatibility checks that fail fast when runtime policy is violated
@@ -28,11 +29,13 @@ Adopt a deterministic backend runtime policy:
## Consequences
Positive:
- Reduced environment drift across dev/CI/prod
- Earlier and clearer failure mode for runtime mismatches
- Improved reproducibility and troubleshooting
Negative:
- More frequent explicit dependency maintenance updates
- Stricter upgrade process for Python/runtime packages

View File

@@ -27,13 +27,14 @@ Adopt a hardened split workflow for the CICD base image:
3. Treat the hash-specific tag as the source of truth for all CI consumers.
4. Keep `latest` only as a convenience tag for humans and manual debugging.
5. Make the main CI workflow consume only the immutable hash tag and fail clearly
if that base image has not been published yet.
if that base image has not been published yet.
6. Add bounded polling in main CI to tolerate short publish/consume races between
the dedicated base workflow and the main workflow.
the dedicated base workflow and the main workflow.
## Consequences
Positive:
- Faster CI via stable base-layer reuse
- Better traceability from base image to Dockerfile content
- Single publish, many pulls across the runner fleet
@@ -41,6 +42,7 @@ Positive:
- Clearer separation of concerns between artifact publication and application CI
Negative:
- Slightly more workflow complexity
- Registry stores additional hash-tagged images
- Main CI now fails fast when the expected base image is missing instead of