Stabilize self-hosted CI workflows and resolve issue #62 (#73)
All checks were successful
CICD / Build and Publish CICD Base Image (push) Successful in 6m8s
CICD / Build and Push CICD Image (push) Successful in 23m14s
CICD / Build CICD Image Failure Postmortem (push) Has been skipped
CICD / Backend Tests (push) Successful in 7m10s
CICD / Frontend Tests (push) Successful in 45s
CICD / Backend Doctests (push) Successful in 18s
CICD / Pre-commit Checks (push) Successful in 14m57s
CICD / Source Lanes Failure Postmortem (push) Has been skipped
CICD / CICD Tests Complete (push) Successful in 3s
CICD / Build Backend Base Image (push) Successful in 18s
CICD / Build Integration Tester Image (push) Successful in 1m5s
CICD / Build Backend Main Image (push) Successful in 1m52s
CICD / Build Frontend Base Image (push) Successful in 10m42s
CICD / Build Frontend Main Image (push) Successful in 33s
CICD / Build E2E Tester Image (push) Successful in 32m17s
CICD / Production Images Complete (push) Successful in 5s
CICD / Production Image Failures Postmortem (push) Has been skipped
CICD / Runtime Black-Box Integration Tests (push) Successful in 1m13s
CICD / Integration Tests Failure Postmortem (push) Has been skipped
CICD / End-to-End Tests (push) Successful in 11m23s
CICD / E2E Tests Failure Postmortem (push) Has been skipped
All checks were successful
CICD / Build and Publish CICD Base Image (push) Successful in 6m8s
CICD / Build and Push CICD Image (push) Successful in 23m14s
CICD / Build CICD Image Failure Postmortem (push) Has been skipped
CICD / Backend Tests (push) Successful in 7m10s
CICD / Frontend Tests (push) Successful in 45s
CICD / Backend Doctests (push) Successful in 18s
CICD / Pre-commit Checks (push) Successful in 14m57s
CICD / Source Lanes Failure Postmortem (push) Has been skipped
CICD / CICD Tests Complete (push) Successful in 3s
CICD / Build Backend Base Image (push) Successful in 18s
CICD / Build Integration Tester Image (push) Successful in 1m5s
CICD / Build Backend Main Image (push) Successful in 1m52s
CICD / Build Frontend Base Image (push) Successful in 10m42s
CICD / Build Frontend Main Image (push) Successful in 33s
CICD / Build E2E Tester Image (push) Successful in 32m17s
CICD / Production Images Complete (push) Successful in 5s
CICD / Production Image Failures Postmortem (push) Has been skipped
CICD / Runtime Black-Box Integration Tests (push) Successful in 1m13s
CICD / Integration Tests Failure Postmortem (push) Has been skipped
CICD / End-to-End Tests (push) Successful in 11m23s
CICD / E2E Tests Failure Postmortem (push) Has been skipped
## Summary Hardens CI workflows for self-hosted Gitea runners by stabilizing E2E execution and Renovate behavior across internal/external network paths. Closes #62 ## What Changed ### E2E workflow reliability - Fixed E2E workspace handoff to ensure expected repository contents are present during test execution. - Added stricter preflight checks for required frontend files before running E2E. - Reduced mount/path fragility while preserving runtime image pull and compose flow. ### Renovate workflow hardening - Added internal-first endpoint reachability selection with fallback handling. - Added token preflight checks for repository access. - Added explicit host-rule auth handling for API/git paths. - Added container-level connectivity preflight diagnostics. - Added git URL override aligned with selected endpoint context. - Removed incorrect forced Dogar host-IP pinning that broke HTTPS clone routing. ## Why CI behavior was sensitive to runner networking and Renovate clone/auth interactions. These changes make the workflow deterministic in our runner topology and address recurring CI failures. ## Scope - Workflow logic only (`cicd.yaml`, `renovate.yml`) - No app feature or API behavior changes ## Validation - Workflow YAML validation passed during updates. - Changes were applied and verified iteratively from real failing run diagnostics. Co-authored-by: copilotcoder <copilotcoder@darkhelm.org> Reviewed-on: #73
This commit was merged in pull request #73.
This commit is contained in:
@@ -175,10 +175,20 @@ If you want a failover strategy, keep the cached image tagged in the local regis
|
||||
Automation scripts for this workflow live in `scripts/gitea-actions/`:
|
||||
|
||||
- `scripts/gitea-actions/repair_runner_mirror.xsh`
|
||||
Repairs the mirror by pulling from GHCR, tagging/pushing to local registry, and verifying tag presence on each runner host.
|
||||
Repairs mirrors by pulling upstream images, tagging/pushing to local registry, and verifying pullability on each runner host.
|
||||
Current mirror targets:
|
||||
- `kankali.darkhelm.lan:3001/darkhelm.org/act-ubuntu:act-latest`
|
||||
- `kankali.darkhelm.lan:3001/darkhelm.org/renovate:41`
|
||||
- `kankali.darkhelm.lan:3001/darkhelm.org/playwright-browsers:v1.56.1-jammy`
|
||||
|
||||
- `scripts/gitea-actions/check_runner_images.xsh`
|
||||
Verifies host-by-host image presence for both upstream (`GHCR`) and mirrored (`MIRROR`) tags.
|
||||
Verifies host-by-host image presence and pullability for `ubuntu:22.04`, upstream act image (`GHCR`), mirrored act image (`MIRROR`), and Renovate image.
|
||||
|
||||
- `scripts/gitea-actions/runner-prechange-capture.xsh`
|
||||
Captures restart counts, OOM flags, memory state, and runner logs across all hosts.
|
||||
|
||||
- `scripts/gitea-actions/diagnose_runner_startup.xsh`
|
||||
Focused startup diagnostics for runner containers and mirror pull behavior.
|
||||
|
||||
Run from repository root (xonsh):
|
||||
|
||||
|
||||
Reference in New Issue
Block a user