DarkHelm.org/plex-playlist
Public Access
4
1
Fork 0
Code Issues 55 Pull Requests 3 Actions 9 Packages Projects 1 Releases Wiki Activity

Backend runtime upgraded to Python 3.14 with exact dependency pinning (#57)
Some checks failed
CICD Start / Sanity and Base Decision (push) Successful in 18s
Details
Runner Canary / Canary Heavy (ubuntu-act-8gb) (push) Has been skipped
Details
Runner Canary / Canary Heavy (ubuntu-act-4gb) (push) Has been skipped
Details
Runner Canary / Canary Burst (ubuntu-act (push) Failing after 11m10s
Details
Runner Canary / Canary (ubuntu-latest) (push) Failing after 12m39s
Details
Runner Canary / Canary (ubuntu-act) (push) Failing after 12m42s
Details

Browse Source 
Signed-off-by: Cliff Hill <xlorep@darkhelm.org>

## Summary

Upgrades backend runtime baseline and dependency management for issue #10.

### Changes

1. **Python Baseline**: Updated from 3.13 to 3.14
   - Updated `backend/pyproject.toml` requires-python constraint
   - Updated `backend/pyrightconfig.json` pythonVersion
   - Updated all Dockerfile and CI references

2. **Dependency Pinning**: Switched to exact version pins in `backend/pyproject.toml`
   - All dev and runtime dependencies now use `==` instead of `>=`
   - `fastapi==0.120.2`, `uvicorn==0.38.0`
   - ruff, pyright, pytest suite pinned to current resolved versions
   - Regenerated `backend/uv.lock` under Python 3.14

3. **Startup Compatibility Guard** (TDD via RED→GREEN)
   - New `compatibility_status()` function evaluates runtime and pinned deps
   - Startup raises `RuntimeError` if policy fails
   - Implemented via FastAPI lifespan (non-deprecated) handler

4. **Compatibility Status Endpoint**
   - New `GET /compatibility` returns policy status, runtime version, and package checks
   - Shares single source of truth with startup validation

5. **Integration Tests**
   - Added failing-then-passing tests for startup guard and endpoint behavior
   - 100% coverage maintained

6. **Direnv Configuration**
   - Added `UV_PYTHON="3.14"` pin to repo `.envrc`
   - Ensures direnv creates/recreates venv with correct Python version

### Validation

- ✅ ruff format/check
- ✅ pyright strict (0 errors)
- ✅ pytest: 8 passed, 100% coverage (>=95 gate)
- ✅ pydoclint: pass
- ✅ xdoctest: pass

### Notes

- SQLAlchemy/SQLModel introduction deferred to next pass per scope
- Compatibility logic currently validates fastapi/uvicorn pins (runtime deps)
- Ready for container build validation and Renovate bot testing

Co-authored-by: copilotcoder <copilotcoder@darkhelm.org>
Reviewed-on: #57
Co-authored-by: Cliff Hill <xlorep@darkhelm.org>
Co-committed-by: Cliff Hill <xlorep@darkhelm.org>
This commit was merged in pull request #57.
This commit is contained in:
Xlorep DarkHelm
2026-06-18 11:19:24 -04:00
committed by Xlorep DarkHelm
parent 5d74dd8d8f
commit d02039a22e
52 changed files with 4480 additions and 966 deletions
Download Patch File Download Diff File Expand all files Collapse all files

438
.gitea/workflows/cicd-checks.yaml Normal file
View File

@@ -0,0 +1,438 @@
name: CICD Checks
on:
workflow_dispatch:
inputs:
head_sha:
description: Commit SHA to process
required: false
source_workflow:
description: Upstream workflow name
required: false
trace_id:
description: Correlation id propagated across CICD dispatch chain
required: false
env:
GITEA_REGISTRY: kankali.darkhelm.lan:3001
GITEA_REGISTRY_IP: 10.18.75.2
GITEA_REGISTRY_HOST: kankali.darkhelm.lan
concurrency:
group: checks-${{ github.sha }}
cancel-in-progress: true
jobs:
setup:
name: Setup Checks Context
# Use the same stable runner label as the check matrix jobs.
runs-on: ubuntu-act
timeout-minutes: 8
outputs:
head_sha: ${{ steps.meta.outputs.head_sha }}
trace_id: ${{ steps.meta.outputs.trace_id }}
steps:
- name: Identify runner
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- name: Audit trigger context
env:
EVENT_NAME: ${{ github.event_name }}
SOURCE_WORKFLOW: ${{ github.event.inputs.source_workflow }}
HEAD_SHA_INPUT: ${{ github.event.inputs.head_sha }}
HEAD_SHA_FALLBACK: ${{ github.sha }}
REF: ${{ github.ref }}
REF_NAME: ${{ github.ref_name }}
HEAD_REF: ${{ github.head_ref }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
run: |
RESOLVED_HEAD_SHA="${HEAD_SHA_INPUT:-${HEAD_SHA_FALLBACK}}"
TRACE_ID="${TRACE_ID_INPUT:-cicd-checks-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${RESOLVED_HEAD_SHA:0:8}}"
echo "=== Dispatch Audit: CICD Checks ==="
echo "event_name=${EVENT_NAME}"
echo "source_workflow=${SOURCE_WORKFLOW}"
echo "head_sha_input=${HEAD_SHA_INPUT}"
echo "head_sha=${RESOLVED_HEAD_SHA}"
echo "ref=${REF}"
echo "ref_name=${REF_NAME}"
echo "head_ref=${HEAD_REF}"
echo "trace_id=${TRACE_ID}"
- name: Resolve head SHA
id: meta
env:
HEAD_SHA_INPUT: ${{ github.event.inputs.head_sha }}
HEAD_SHA_FALLBACK: ${{ github.sha }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
run: |
RESOLVED_HEAD_SHA="${HEAD_SHA_INPUT:-${HEAD_SHA_FALLBACK}}"
TRACE_ID="${TRACE_ID_INPUT:-cicd-checks-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${RESOLVED_HEAD_SHA:0:8}}"
echo "head_sha=${RESOLVED_HEAD_SHA}" >> "$GITHUB_OUTPUT"
echo "trace_id=${TRACE_ID}" >> "$GITHUB_OUTPUT"
- &failure_diagnostics_step
name: Failure diagnostics
if: failure()
run: |
echo "=== Failure Diagnostics ==="
date -u '+timestamp_utc=%Y-%m-%dT%H:%M:%SZ'
echo "runner_name=${RUNNER_NAME:-unknown}"
echo "runner_hostname=${HOSTNAME:-unknown}"
uname -a || true
cat /etc/os-release 2>/dev/null || true
df -h || true
free -h || true
ps aux --sort=-%mem | head -n 30 || true
if command -v docker >/dev/null 2>&1; then
echo "=== Docker Diagnostics ==="
docker version || true
docker info || true
docker ps -a || true
docker images --digests | head -n 50 || true
else
echo "docker not available on this runner"
fi
echo "=== Kernel Tail ==="
dmesg | tail -n 120 || true
run-check:
name: ${{ matrix.name }}
# Run checks across the full act runner pool for maximum parallelism.
runs-on: ubuntu-act
timeout-minutes: 20
needs: setup
strategy:
fail-fast: false
max-parallel: 6
matrix:
include:
- name: Trailing Whitespace Check
hook: trailing-whitespace
- name: YAML Syntax Check
hook: check-yaml
- name: End of File Check
hook: end-of-file-fixer
- name: TOML Syntax Check
hook: check-toml
- name: Mixed Line Ending Check
hook: mixed-line-ending
- name: TOML Formatting Check
hook: pretty-format-toml
- name: Ruff Linting
hook: ruff
- name: Ruff Format Check
hook: ruff-format
- name: Pyright Type Check
hook: pyright
- name: Pydoclint Docstring Check
hook: pydoclint
- name: No Docstring Types Check
hook: no-docstring-types
- name: ESLint Check
hook: eslint
- name: Prettier Format Check
hook: prettier
- name: TSDoc Lint Check
hook: tsdoc-lint
- name: TypeScript Type Check
hook: typescript-check
steps:
- name: Runner assignment audit
env:
TRACE_ID: ${{ needs.setup.outputs.trace_id }}
HOOK: ${{ matrix.hook }}
run: |
echo "=== Runner Assignment Audit ==="
echo "trace_id=${TRACE_ID}"
echo "hook=${HOOK}"
echo "run_id=${GITHUB_RUN_ID}"
echo "run_attempt=${GITHUB_RUN_ATTEMPT}"
echo "job=${GITHUB_JOB}"
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- name: Configure registry host resolution
run: |
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
- name: Ensure CICD image is available
env:
HEAD_SHA: ${{ needs.setup.outputs.head_sha }}
run: |
IMAGE="${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}"
if docker image inspect "${IMAGE}" >/dev/null 2>&1; then
echo "Using cached CICD image: ${IMAGE}"
else
echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login "http://${GITEA_REGISTRY}" -u "${{ github.actor }}" --password-stdin
pulled=false
for i in 1 2 3; do
echo "Pull attempt ${i}/3 for ${IMAGE}"
if docker pull "${IMAGE}"; then
pulled=true
break
fi
if [ "${i}" -lt 3 ]; then
sleep_seconds=$((5 * i))
echo "Pull failed; retrying in ${sleep_seconds}s"
sleep "${sleep_seconds}"
fi
done
if [ "${pulled}" != "true" ]; then
echo "❌ Failed to pull CICD image after 3 attempts: ${IMAGE}"
exit 1
fi
fi
- name: Validate CICD image provenance
env:
HEAD_SHA: ${{ needs.setup.outputs.head_sha }}
run: |
IMAGE="${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}"
SOURCE_SHA=$(docker run --rm --entrypoint /bin/sh "${IMAGE}" -c 'cat /workspace/.cicd-source-sha 2>/dev/null || true')
echo "requested_head_sha=${HEAD_SHA}"
echo "image=${IMAGE}"
echo "embedded_source_sha=${SOURCE_SHA:-missing}"
if [ -z "${SOURCE_SHA}" ]; then
echo "❌ Missing /workspace/.cicd-source-sha in ${IMAGE}"
exit 1
fi
case "${SOURCE_SHA}" in
${HEAD_SHA}*)
;;
*)
echo "❌ Source SHA mismatch: requested=${HEAD_SHA} actual=${SOURCE_SHA}"
exit 1
;;
esac
- name: Run pre-commit hook
env:
HEAD_SHA: ${{ needs.setup.outputs.head_sha }}
HOOK: ${{ matrix.hook }}
run: |
set -o pipefail
echo "=== Pre-commit Hook Start ==="
echo "hook=${HOOK}"
echo "head_sha=${HEAD_SHA}"
if [ "${HOOK}" = "prettier" ]; then
echo "=== Host Workspace Debug (prettier) ==="
echo "host_head_sha=${HEAD_SHA}"
git rev-parse --short HEAD || true
if [ -d frontend ]; then
(cd frontend && corepack yarn prettier --version && corepack yarn prettier --check src/main.test.ts src/main.ts src/test-setup.ts src/validation.test.ts src/validation.ts) || true
(cd frontend && sha256sum src/main.test.ts src/main.ts src/test-setup.ts src/validation.test.ts src/validation.ts) || true
else
echo "host_frontend_dir_missing=frontend"
fi
fi
LOG_FILE="$(mktemp)"
docker run --rm -e CI=true --entrypoint /bin/sh "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}" -c "
export HOME=/root &&
export PRE_COMMIT_HOME=/root/.cache/pre-commit &&
mkdir -p "$PRE_COMMIT_HOME" &&
echo 'running_hook=${HOOK}' &&
echo 'pre_commit_home='"$PRE_COMMIT_HOME" &&
echo 'container_shells=' && ls -l /bin/sh /bin/bash 2>/dev/null || true &&
echo 'tool_paths=' && command -v git /workspace/backend/.venv/bin/pre-commit python3 python 2>/dev/null || true &&
/workspace/backend/.venv/bin/pre-commit --version &&
cd /workspace &&
if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
echo 'No git repository found in /workspace; bootstrapping for pre-commit' &&
git config --global user.email 'ci@cicd' &&
git config --global user.name 'CICD' &&
git init /workspace &&
git -C /workspace config core.autocrlf false &&
git -C /workspace config core.eol lf &&
git -C /workspace add -A
fi &&
case "${HOOK}" in
eslint|prettier|tsdoc-lint|typescript-check)
echo 'JS hook detected; preparing frontend dependencies' &&
if [ ! -d /workspace/frontend ]; then
echo '❌ Missing /workspace/frontend for JS hook execution' &&
exit 1
fi &&
cd /workspace/frontend &&
install_ok=false &&
for i in 1 2 3; do
echo 'yarn install attempt' "\$i"'/3' &&
if timeout 1200 corepack yarn install --immutable --mode=skip-build; then
install_ok=true
break
fi
if [ "\$i" -lt 3 ]; then
echo 'yarn install failed; cleaning and retrying in 20s' &&
rm -rf node_modules .yarn/install-state.gz .yarn/cache &&
corepack yarn cache clean --all 2>/dev/null || true &&
sleep 20
fi
done &&
if [ "\$install_ok" != 'true' ]; then
echo '❌ Unable to prepare frontend dependencies for JS pre-commit hooks' &&
exit 1
fi &&
if [ "${HOOK}" = "prettier" ]; then
echo '=== Container Workspace Debug (prettier) ===' &&
corepack yarn prettier --version || true &&
sha256sum src/main.test.ts src/main.ts src/test-setup.ts src/validation.test.ts src/validation.ts || true &&
corepack yarn prettier --check src/main.test.ts src/main.ts src/test-setup.ts src/validation.test.ts src/validation.ts || true
fi &&
cd /workspace
;;
esac &&
/workspace/backend/.venv/bin/pre-commit run ${HOOK} --all-files --show-diff-on-failure
" 2>&1 | tee "${LOG_FILE}"
HOOK_STATUS=${PIPESTATUS[0]}
if [ "${HOOK_STATUS}" -ne 0 ]; then
echo "❌ Pre-commit hook failed: ${HOOK} (exit=${HOOK_STATUS})"
echo "--- Last 200 lines of hook output ---"
tail -n 200 "${LOG_FILE}" || true
if [ "${HOOK}" = "pydoclint" ]; then
echo "=== Additional pydoclint diagnostics ==="
docker run --rm -e CI=true --entrypoint /bin/sh "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}" -c "
set -e
cd /workspace/backend
echo 'python_version=' && python3 --version
echo 'uv_version=' && uv --version
echo 'pydoclint_version=' && uv run pydoclint --version
echo 'backend_py_count=' && find src -name '*.py' | wc -l
echo 'running_direct_pydoclint='
uv run pydoclint --config=pyproject.toml src/
" || true
fi
exit "${HOOK_STATUS}"
fi
echo "=== Pre-commit Hook End (success) ==="
- *failure_diagnostics_step
dispatch-tests:
name: Dispatch CICD Tests
# Keep dispatch on the same runner pool as setup/check jobs.
runs-on: ubuntu-act
timeout-minutes: 15
needs: [setup, run-check]
steps:
- name: Dispatch tests workflow
env:
ACTIONS_TRIGGER_TOKEN: ${{ secrets.ACTIONS_TRIGGER_TOKEN }}
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
HEAD_SHA: ${{ needs.setup.outputs.head_sha }}
REPO_FULL: ${{ github.repository }}
HEAD_REF: ${{ github.head_ref }}
REF_NAME: ${{ github.ref_name }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
run: |
set -e
DISPATCH_TOKEN="${ACTIONS_TRIGGER_TOKEN:-${PACKAGE_ACCESS_TOKEN:-}}"
if [ -z "${DISPATCH_TOKEN}" ]; then
echo "❌ Missing dispatch token. Set ACTIONS_TRIGGER_TOKEN (repo write scope) or ensure PACKAGE_ACCESS_TOKEN has Actions workflow-dispatch permissions."
exit 1
fi
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
REPO_OWNER="${REPO_FULL%/*}"
REPO_NAME="${REPO_FULL#*/}"
TARGET_REF="${HEAD_REF:-${REF_NAME}}"
TRACE_ID="${TRACE_ID_INPUT:-cicd-checks-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${HEAD_SHA:0:8}}"
echo "trace_id=${TRACE_ID}"
echo "target_ref=${TARGET_REF}"
CANDIDATE_API_BASES=()
if [ -n "${GITHUB_SERVER_URL:-}" ]; then
CANDIDATE_API_BASES+=("${GITHUB_SERVER_URL%/}/api/v1")
fi
CANDIDATE_API_BASES+=("http://${GITEA_REGISTRY_IP}:3001/api/v1")
CANDIDATE_API_BASES+=("http://${GITEA_REGISTRY_HOST}:3001/api/v1")
ensure_curl() {
if command -v curl >/dev/null 2>&1; then
return 0
fi
if command -v apt-get >/dev/null 2>&1; then
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq curl ca-certificates
fi
command -v curl >/dev/null 2>&1
}
ensure_curl || { echo "❌ curl unavailable for dispatch"; exit 1; }
HELPER_PATH="/tmp/dispatch-workflow.sh"
fetch_dispatch_helper() {
local helper_ref="$1"
local api_base
for api_base in "${CANDIDATE_API_BASES[@]}"; do
helper_url="${api_base}/repos/${REPO_OWNER}/${REPO_NAME}/raw/scripts/dispatch-workflow.sh?ref=${helper_ref}"
if curl -fsS --connect-timeout 5 --max-time 20 \
-H "Authorization: token ${DISPATCH_TOKEN}" \
-H "User-Agent: plex-playlist-cicd-checks" \
-o "${HELPER_PATH}" \
"${helper_url}"; then
chmod +x "${HELPER_PATH}"
return 0
fi
done
return 1
}
if ! fetch_dispatch_helper "${TARGET_REF}" && ! fetch_dispatch_helper "${HEAD_SHA}"; then
echo "❌ Failed to fetch scripts/dispatch-workflow.sh from repository"
exit 1
fi
DISPATCH_ARGS=(
--token "${DISPATCH_TOKEN}"
--repo "${REPO_FULL}"
--workflow "cicd-tests.yaml"
--ref "${TARGET_REF}"
--head-sha "${HEAD_SHA}"
--source-workflow "CICD Checks"
--trace-id "${TRACE_ID}"
)
for API_BASE in "${CANDIDATE_API_BASES[@]}"; do
DISPATCH_ARGS+=(--api-base "${API_BASE}")
done
"${HELPER_PATH}" "${DISPATCH_ARGS[@]}"
- *failure_diagnostics_step

230
.gitea/workflows/cicd-start.yaml Normal file
View File

@@ -0,0 +1,230 @@
name: CICD Start
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main, develop ]
workflow_dispatch:
inputs:
trace_id:
description: Correlation id propagated across CICD dispatch chain
required: false
env:
GITEA_SSH_HOST: kankali.darkhelm.lan
GITEA_SSH_PORT: "2222"
GITEA_REPO_SSH_URL: ssh://git@kankali.darkhelm.lan:2222/DarkHelm.org/plex-playlist.git
GITEA_REGISTRY: kankali.darkhelm.lan:3001
GITEA_REGISTRY_IP: 10.18.75.2
GITEA_REGISTRY_HOST: kankali.darkhelm.lan
concurrency:
group: start-${{ github.ref }}
cancel-in-progress: true
jobs:
sanity-and-routing:
name: Sanity and Base Decision
# Keep startup lightweight to avoid runner image setup stalls.
# Use the same stable self-hosted label as the downstream CICD jobs.
runs-on: ubuntu-act
timeout-minutes: 12
outputs:
base_needed: ${{ steps.base-decision.outputs.base_needed }}
head_sha: ${{ steps.base-decision.outputs.head_sha }}
base_hash: ${{ steps.base-decision.outputs.base_hash }}
steps:
- name: Identify runner
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- name: Audit trigger context
env:
EVENT_NAME: ${{ github.event_name }}
REPOSITORY: ${{ github.repository }}
ACTOR: ${{ github.actor }}
REF: ${{ github.ref }}
REF_NAME: ${{ github.ref_name }}
HEAD_REF: ${{ github.head_ref }}
SHA: ${{ github.sha }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
run: |
TRACE_ID="${TRACE_ID_INPUT:-cicd-start-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${SHA:0:8}}"
echo "=== Dispatch Audit: CICD Start ==="
echo "event_name=${EVENT_NAME}"
echo "repository=${REPOSITORY}"
echo "actor=${ACTOR}"
echo "ref=${REF}"
echo "ref_name=${REF_NAME}"
echo "head_ref=${HEAD_REF}"
echo "sha=${SHA}"
echo "trace_id=${TRACE_ID}"
- name: Configure registry host resolution
run: |
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
- name: Compute routing signal
id: base-decision
run: |
set -e
# Keep this workflow container-agnostic: do not rely on checkout/actions/node.
# docker-build-base handles canonical base hash/existence checks.
echo "head_sha=${GITHUB_SHA}" >> "${GITHUB_OUTPUT}"
echo "base_hash=deferred" >> "${GITHUB_OUTPUT}"
echo "base_needed=true" >> "${GITHUB_OUTPUT}"
echo "Routing via docker-build-base.yaml"
echo "Base hash delegated to Docker Build Base workflow"
- name: Dispatch downstream workflow
if: steps.base-decision.outcome == 'success' && steps.base-decision.outputs.head_sha != '' && steps.base-decision.outputs.base_needed != ''
env:
ACTIONS_TRIGGER_TOKEN: ${{ secrets.ACTIONS_TRIGGER_TOKEN }}
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
BASE_NEEDED: ${{ steps.base-decision.outputs.base_needed }}
HEAD_SHA: ${{ steps.base-decision.outputs.head_sha }}
BASE_HASH: ${{ steps.base-decision.outputs.base_hash }}
REPO_FULL: ${{ github.repository }}
HEAD_REF: ${{ github.head_ref }}
REF_NAME: ${{ github.ref_name }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
run: |
set -e
DISPATCH_TOKEN="${ACTIONS_TRIGGER_TOKEN:-${PACKAGE_ACCESS_TOKEN:-}}"
if [ -z "${DISPATCH_TOKEN}" ]; then
echo "❌ Missing dispatch token. Set ACTIONS_TRIGGER_TOKEN (repo write scope) or ensure PACKAGE_ACCESS_TOKEN has Actions workflow-dispatch permissions."
exit 1
fi
REPO_OWNER="${REPO_FULL%/*}"
REPO_NAME="${REPO_FULL#*/}"
TARGET_REF="${HEAD_REF:-${REF_NAME}}"
TRACE_ID="${TRACE_ID_INPUT:-cicd-start-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${HEAD_SHA:0:8}}"
echo "trace_id=${TRACE_ID}"
echo "target_ref=${TARGET_REF}"
if [ "${BASE_NEEDED}" = "true" ]; then
TARGET_WORKFLOW="docker-build-base.yaml"
else
TARGET_WORKFLOW="docker-build-main.yaml"
fi
CANDIDATE_API_BASES=()
if [ -n "${GITHUB_SERVER_URL:-}" ]; then
CANDIDATE_API_BASES+=("${GITHUB_SERVER_URL%/}/api/v1")
fi
if [ -n "${GITEA_SSH_HOST:-}" ]; then
CANDIDATE_API_BASES+=("http://${GITEA_SSH_HOST}:3001/api/v1")
fi
if [ -n "${GITEA_REGISTRY_HOST:-}" ]; then
CANDIDATE_API_BASES+=("http://${GITEA_REGISTRY_HOST}:3001/api/v1")
fi
if [ -n "${GITEA_REGISTRY_IP:-}" ]; then
CANDIDATE_API_BASES+=("http://${GITEA_REGISTRY_IP}:3001/api/v1")
fi
ensure_curl() {
if command -v curl >/dev/null 2>&1; then
return 0
fi
if command -v apt-get >/dev/null 2>&1; then
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq curl ca-certificates
fi
if command -v curl >/dev/null 2>&1; then
return 0
fi
echo "❌ curl is required for dispatch and could not be installed"
return 1
}
ensure_curl
HELPER_PATH="/tmp/dispatch-workflow.sh"
fetch_dispatch_helper() {
local helper_ref="$1"
local api_base
for api_base in "${CANDIDATE_API_BASES[@]}"; do
helper_url="${api_base}/repos/${REPO_OWNER}/${REPO_NAME}/raw/scripts/dispatch-workflow.sh?ref=${helper_ref}"
if curl -fsS --connect-timeout 5 --max-time 20 \
-H "Authorization: token ${DISPATCH_TOKEN}" \
-H "User-Agent: plex-playlist-cicd-start" \
-o "${HELPER_PATH}" \
"${helper_url}"; then
chmod +x "${HELPER_PATH}"
return 0
fi
done
return 1
}
if ! fetch_dispatch_helper "${TARGET_REF}" && ! fetch_dispatch_helper "${HEAD_SHA}"; then
echo "❌ Failed to fetch scripts/dispatch-workflow.sh from repository"
exit 1
fi
DISPATCH_ARGS=(
--token "${DISPATCH_TOKEN}"
--repo "${REPO_FULL}"
--workflow "${TARGET_WORKFLOW}"
--ref "${TARGET_REF}"
--head-sha "${HEAD_SHA}"
--source-workflow "CICD Start"
--trace-id "${TRACE_ID}"
--base-needed "${BASE_NEEDED}"
--base-hash "${BASE_HASH}"
)
for API_BASE in "${CANDIDATE_API_BASES[@]}"; do
DISPATCH_ARGS+=(--api-base "${API_BASE}")
done
"${HELPER_PATH}" "${DISPATCH_ARGS[@]}"
- name: Failure diagnostics
if: failure()
run: |
echo "=== Failure Diagnostics ==="
date -u '+timestamp_utc=%Y-%m-%dT%H:%M:%SZ'
echo "runner_name=${RUNNER_NAME:-unknown}"
echo "runner_hostname=${HOSTNAME:-unknown}"
uname -a || true
cat /etc/os-release 2>/dev/null || true
df -h || true
free -h || true
ps aux --sort=-%mem | head -n 30 || true
if command -v docker >/dev/null 2>&1; then
echo "=== Docker Diagnostics ==="
docker version || true
docker info || true
docker ps -a || true
docker images --digests | head -n 50 || true
else
echo "docker not available on this runner"
fi
echo "=== Kernel Tail ==="
dmesg | tail -n 120 || true

386
.gitea/workflows/cicd-tests.yaml Normal file
View File

@@ -0,0 +1,386 @@
name: CICD Tests
on:
workflow_dispatch:
inputs:
head_sha:
description: Commit SHA to process
required: false
source_workflow:
description: Upstream workflow name
required: false
trace_id:
description: Correlation id propagated across CICD dispatch chain
required: false
env:
GITEA_REGISTRY: kankali.darkhelm.lan:3001
GITEA_REGISTRY_IP: 10.18.75.2
GITEA_REGISTRY_HOST: kankali.darkhelm.lan
concurrency:
group: tests-${{ github.sha }}
cancel-in-progress: true
jobs:
setup:
name: Setup Tests Context
# Use the same stable runner label as the test jobs.
runs-on: ubuntu-act
timeout-minutes: 8
outputs:
head_sha: ${{ steps.meta.outputs.head_sha }}
steps:
- name: Identify runner
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- name: Audit trigger context
env:
EVENT_NAME: ${{ github.event_name }}
SOURCE_WORKFLOW: ${{ github.event.inputs.source_workflow }}
HEAD_SHA_INPUT: ${{ github.event.inputs.head_sha }}
HEAD_SHA_FALLBACK: ${{ github.sha }}
REF: ${{ github.ref }}
REF_NAME: ${{ github.ref_name }}
HEAD_REF: ${{ github.head_ref }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
run: |
RESOLVED_HEAD_SHA="${HEAD_SHA_INPUT:-${HEAD_SHA_FALLBACK}}"
TRACE_ID="${TRACE_ID_INPUT:-cicd-tests-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${RESOLVED_HEAD_SHA:0:8}}"
echo "=== Dispatch Audit: CICD Tests ==="
echo "event_name=${EVENT_NAME}"
echo "source_workflow=${SOURCE_WORKFLOW}"
echo "head_sha_input=${HEAD_SHA_INPUT}"
echo "head_sha=${RESOLVED_HEAD_SHA}"
echo "ref=${REF}"
echo "ref_name=${REF_NAME}"
echo "head_ref=${HEAD_REF}"
echo "trace_id=${TRACE_ID}"
- name: Resolve head SHA
id: meta
env:
HEAD_SHA_INPUT: ${{ github.event.inputs.head_sha }}
HEAD_SHA_FALLBACK: ${{ github.sha }}
run: |
RESOLVED_HEAD_SHA="${HEAD_SHA_INPUT:-${HEAD_SHA_FALLBACK}}"
echo "head_sha=${RESOLVED_HEAD_SHA}" >> "$GITHUB_OUTPUT"
- &failure_diagnostics_step
name: Failure diagnostics
if: failure()
run: |
echo "=== Failure Diagnostics ==="
date -u '+timestamp_utc=%Y-%m-%dT%H:%M:%SZ'
echo "runner_name=${RUNNER_NAME:-unknown}"
echo "runner_hostname=${HOSTNAME:-unknown}"
uname -a || true
cat /etc/os-release 2>/dev/null || true
df -h || true
free -h || true
ps aux --sort=-%mem | head -n 30 || true
if command -v docker >/dev/null 2>&1; then
echo "=== Docker Diagnostics ==="
docker version || true
docker info || true
docker ps -a || true
docker images --digests | head -n 50 || true
else
echo "docker not available on this runner"
fi
echo "=== Kernel Tail ==="
dmesg | tail -n 120 || true
backend-tests:
name: Backend Tests
# Use ubuntu-act runner pool for consistent availability.
runs-on: ubuntu-act
timeout-minutes: 25
needs: setup
steps:
- name: Identify runner
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- &configure_registry_host_step
name: Configure registry host resolution
run: |
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
- &ensure_cicd_image_step
name: Ensure CICD image is available
env:
HEAD_SHA: ${{ needs.setup.outputs.head_sha }}
run: |
IMAGE="${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}"
if docker image inspect "${IMAGE}" >/dev/null 2>&1; then
echo "Using cached CICD image: ${IMAGE}"
else
echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login "http://${GITEA_REGISTRY}" -u "${{ github.actor }}" --password-stdin
pulled=false
for i in 1 2 3; do
echo "Pull attempt ${i}/3 for ${IMAGE}"
if docker pull "${IMAGE}"; then
pulled=true
break
fi
if [ "${i}" -lt 3 ]; then
sleep_seconds=$((5 * i))
echo "Pull failed; retrying in ${sleep_seconds}s"
sleep "${sleep_seconds}"
fi
done
if [ "${pulled}" != "true" ]; then
echo "❌ Failed to pull CICD image after 3 attempts: ${IMAGE}"
exit 1
fi
fi
- name: Run backend tests with coverage
env:
HEAD_SHA: ${{ needs.setup.outputs.head_sha }}
run: |
set -o pipefail
LOG_FILE="$(mktemp)"
docker run --rm "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}" bash -c "
cd /workspace/backend &&
source .venv/bin/activate &&
uv run pytest -v --tb=short --cov=src --cov-report=term-missing --cov-fail-under=95
" 2>&1 | tee "${LOG_FILE}"
TEST_STATUS=${PIPESTATUS[0]}
if [ "${TEST_STATUS}" -ne 0 ]; then
echo "❌ Backend tests failed (exit=${TEST_STATUS})"
echo "--- Last 200 lines of backend test output ---"
tail -n 200 "${LOG_FILE}" || true
exit "${TEST_STATUS}"
fi
- *failure_diagnostics_step
frontend-tests:
name: Frontend Tests
# Use ubuntu-act runner pool for consistent availability.
runs-on: ubuntu-act
timeout-minutes: 25
needs: setup
steps:
- name: Identify runner
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- *configure_registry_host_step
- *ensure_cicd_image_step
- name: Run frontend tests with coverage
env:
HEAD_SHA: ${{ needs.setup.outputs.head_sha }}
run: |
set -o pipefail
LOG_FILE="$(mktemp)"
docker run --rm "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}" bash -c "
cd /workspace/frontend &&
yarn test:coverage --run --reporter=verbose --coverage.reporter=text --coverage.reporter=text-summary --coverage.thresholds.lines=85 --coverage.thresholds.functions=85 --coverage.thresholds.branches=85 --coverage.thresholds.statements=85
" 2>&1 | tee "${LOG_FILE}"
TEST_STATUS=${PIPESTATUS[0]}
if [ "${TEST_STATUS}" -ne 0 ]; then
echo "❌ Frontend tests failed (exit=${TEST_STATUS})"
echo "--- Last 200 lines of frontend test output ---"
tail -n 200 "${LOG_FILE}" || true
exit "${TEST_STATUS}"
fi
- *failure_diagnostics_step
xdoctest:
name: Backend Doctests
# Use ubuntu-act runner pool for consistent availability.
runs-on: ubuntu-act
timeout-minutes: 15
needs: setup
steps:
- name: Identify runner
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- *configure_registry_host_step
- *ensure_cicd_image_step
- name: Run backend doctests
env:
HEAD_SHA: ${{ needs.setup.outputs.head_sha }}
run: |
set -o pipefail
LOG_FILE="$(mktemp)"
docker run --rm "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}" bash -c "
cd /workspace/backend &&
source .venv/bin/activate &&
uv run xdoctest src/ --quiet
" 2>&1 | tee "${LOG_FILE}"
TEST_STATUS=${PIPESTATUS[0]}
if [ "${TEST_STATUS}" -ne 0 ]; then
echo "❌ Backend doctests failed (exit=${TEST_STATUS})"
echo "--- Last 200 lines of doctest output ---"
tail -n 200 "${LOG_FILE}" || true
exit "${TEST_STATUS}"
fi
- *failure_diagnostics_step
integration-tests:
name: Integration Tests
# Pin integration tests to high-memory worker to reduce setup-stage runner churn.
runs-on: ubuntu-act-8gb
timeout-minutes: 20
needs: [setup, backend-tests]
steps:
- name: Identify runner
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- *configure_registry_host_step
- *ensure_cicd_image_step
- name: Run integration tests
env:
HEAD_SHA: ${{ needs.setup.outputs.head_sha }}
run: |
set -o pipefail
LOG_FILE="$(mktemp)"
docker run --rm "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}" bash -c "
cd /workspace/backend &&
source .venv/bin/activate &&
if [ -d 'tests/integration' ]; then
uv run pytest tests/integration/ -v --tb=short
else
echo 'No integration tests found'
fi
" 2>&1 | tee "${LOG_FILE}"
TEST_STATUS=${PIPESTATUS[0]}
if [ "${TEST_STATUS}" -ne 0 ]; then
echo "❌ Integration tests failed (exit=${TEST_STATUS})"
echo "--- Last 200 lines of integration test output ---"
tail -n 200 "${LOG_FILE}" || true
exit "${TEST_STATUS}"
fi
- *failure_diagnostics_step
e2e-tests:
name: End-to-End Tests
# Use ubuntu-act runner pool for consistent availability.
runs-on: ubuntu-act
timeout-minutes: 30
needs: [setup, frontend-tests]
steps:
- name: Identify runner
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- *configure_registry_host_step
- *ensure_cicd_image_step
- name: Run E2E tests
env:
HEAD_SHA: ${{ needs.setup.outputs.head_sha }}
run: |
set -o pipefail
LOG_FILE="$(mktemp)"
docker run --rm -e CI=true "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}" bash -c "
cd /workspace/frontend &&
if [ -d 'tests/e2e' ] || grep -q 'playwright' package.json; then
yarn playwright --version &&
PW_BROWSER_PATH="\${PLAYWRIGHT_BROWSERS_PATH:-/root/.cache/ms-playwright}" &&
if find "\${PW_BROWSER_PATH}" -maxdepth 1 -type d -name 'chromium-*' | grep -q .; then
echo 'Using preinstalled Playwright Chromium from '"\${PW_BROWSER_PATH}"
else
browser_ok=false &&
for i in 1 2 3; do
echo 'Playwright browser install attempt' "\$i"'/3' &&
if timeout 1800 yarn playwright install chromium; then
browser_ok=true
break
fi
if [ "\$i" -lt 3 ]; then
echo 'Playwright install attempt failed; retrying in 20s'
sleep 20
fi
done &&
if [ "\$browser_ok" != 'true' ]; then
echo '❌ Playwright browser install failed after 3 attempts'
exit 1
fi
fi &&
yarn test:e2e --reporter=list --timeout=90000
else
echo 'No E2E tests found'
fi
" 2>&1 | tee "${LOG_FILE}"
TEST_STATUS=${PIPESTATUS[0]}
if [ "${TEST_STATUS}" -ne 0 ]; then
echo "❌ E2E tests failed (exit=${TEST_STATUS})"
echo "--- Last 200 lines of E2E output ---"
tail -n 200 "${LOG_FILE}" || true
exit "${TEST_STATUS}"
fi
- *failure_diagnostics_step

743
.gitea/workflows/cicd.yml
View File

@@ -1,658 +1,141 @@
name: Tests
name: CICD
on:
push:
branches: [ main, develop, feature/* ]
pull_request:
branches: [ main, develop ]
workflow_dispatch:
env:
GITEA_REGISTRY_HOST: kankali.darkhelm.lan
GITEA_REGISTRY_IP: 10.18.75.2
concurrency:
group: cicd-launch-${{ github.ref }}
cancel-in-progress: true
jobs:
setup-base:
name: Build and Push CICD Base Image
launch:
name: Launch CICD Start
# Use the same stable runner pool as the rest of CICD.
runs-on: ubuntu-act
timeout-minutes: 8
steps:
- name: Minimal checkout for base Dockerfile
env:
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
- name: Configure registry host resolution
run: |
echo "=== Minimal Repository Checkout for Base Dockerfile ==="
# Set up SSH key securely (temporary file approach)
if [ -n "${SSH_PRIVATE_KEY}" ]; then
mkdir -p ~/.ssh
echo "${SSH_PRIVATE_KEY}" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
ssh-keyscan -p 2222 dogar.darkhelm.org >> ~/.ssh/known_hosts 2>/dev/null
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
# Clone just enough to get the Dockerfile
GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" \
git clone --depth 1 --no-checkout \
ssh://git@dogar.darkhelm.org:2222/DarkHelm.org/plex-playlist.git .
# Checkout only the base Dockerfile and dockerignore
git checkout HEAD -- Dockerfile.cicd-base .dockerignore
# Clean up SSH key for security
rm -f ~/.ssh/id_rsa
echo "✓ Dockerfile.cicd-base ready for build"
- name: Check if base image needs rebuilding
id: check-base
- name: Dispatch CICD Start workflow
env:
ACTIONS_TRIGGER_TOKEN: ${{ secrets.ACTIONS_TRIGGER_TOKEN }}
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
REGISTRY_USER: ${{ secrets.REGISTRY_USER || github.actor }}
REPO_FULL: ${{ github.repository }}
HEAD_REF: ${{ github.head_ref }}
REF_NAME: ${{ github.ref_name }}
HEAD_SHA: ${{ github.sha }}
run: |
echo "=== Checking if CICD Base Image Needs Rebuilding ==="
set -e
# Login to registry to check for existing image
if echo "${PACKAGE_ACCESS_TOKEN}" | docker login dogar.darkhelm.org -u "${REGISTRY_USER}" --password-stdin; then
echo "✓ Successfully logged into registry"
else
echo "❌ Failed to login to registry, will force build"
echo "needs_build=true" >> $GITHUB_OUTPUT
exit 0
fi
DISPATCH_TOKEN="${ACTIONS_TRIGGER_TOKEN:-${PACKAGE_ACCESS_TOKEN:-}}"
# Calculate hash of base Dockerfile for cache key
BASE_HASH=$(sha256sum Dockerfile.cicd-base | cut -d' ' -f1 | head -c16)
echo "Base Dockerfile hash: ${BASE_HASH}"
echo "base_hash=${BASE_HASH}" >> $GITHUB_OUTPUT
# Try to pull existing base image with this hash (with timeout handling)
echo "Attempting to pull base image with hash: ${BASE_HASH}"
PULL_SUCCESS=false
# First try to pull the hash-specific image
if timeout 120 docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:${BASE_HASH} 2>/dev/null; then
echo "✓ Base image with hash ${BASE_HASH} found in registry"
PULL_SUCCESS=true
else
echo "Base image with hash ${BASE_HASH} not found, trying latest..."
# Fallback: try to pull latest and check if it matches our hash
if timeout 120 docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:latest 2>/dev/null; then
# Check if the pulled latest image is what we need (this is a simplified check)
echo "Pulled latest base image, will use it for now"
docker tag dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:latest \
dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:${BASE_HASH}
PULL_SUCCESS=true
fi
fi
if [ "$PULL_SUCCESS" = true ]; then
echo "✓ Base image available, skipping build"
echo "needs_build=false" >> $GITHUB_OUTPUT
# Ensure latest tag is available for dependent job
docker tag dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:${BASE_HASH} \
dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:latest 2>/dev/null || true
docker push dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:latest 2>/dev/null || \
echo "Warning: Could not push latest tag, but base image is available"
else
echo "Base image not available in registry - will build new image"
echo "This is normal for first run or when base dependencies change"
echo "needs_build=true" >> $GITHUB_OUTPUT
fi
- name: Build and push base image
if: steps.check-base.outputs.needs_build == 'true'
env:
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
REGISTRY_USER: ${{ secrets.REGISTRY_USER || github.actor }}
BASE_HASH: ${{ steps.check-base.outputs.base_hash }}
run: |
echo "=== Building CICD Base Image ==="
echo "Base hash: ${BASE_HASH}"
# Enable Docker BuildKit
export DOCKER_BUILDKIT=1
# Build base image (no secrets needed for base dependencies)
echo "Building base image..."
if docker build -f Dockerfile.cicd-base \
--build-arg BASE_IMAGE_VERSION="v1.0.0-${BASE_HASH}" \
-t cicd-base:latest .; then
echo "✓ Base image built successfully"
else
echo "❌ Failed to build base image"
if [ -z "${DISPATCH_TOKEN}" ]; then
echo "❌ Missing dispatch token. Set ACTIONS_TRIGGER_TOKEN (repo write scope) or ensure PACKAGE_ACCESS_TOKEN has Actions workflow-dispatch permissions."
exit 1
fi
# Tag for registry with hash and latest
docker tag cicd-base:latest dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:${BASE_HASH}
docker tag cicd-base:latest dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:latest
# Push to registry with retry
echo "Pushing base images to registry..."
for i in 1 2 3; do
echo "Push attempt $i..."
if docker push dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:${BASE_HASH} && \
docker push dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:latest; then
echo "✓ CICD base image built and pushed with hash ${BASE_HASH}"
break
else
echo "❌ Push attempt $i failed"
if [ $i -eq 3 ]; then
echo "❌ All push attempts failed, but base image is built locally"
echo "The complete image job can use the local base image"
else
sleep 10
fi
fi
done
setup:
name: Build and Push CICD Complete Image
runs-on: ubuntu-act
needs: setup-base
steps:
- name: Minimal checkout for Dockerfile
env:
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
run: |
echo "=== Minimal Repository Checkout for Complete Dockerfile ==="
# Set up SSH key securely (temporary file approach)
if [ -n "${SSH_PRIVATE_KEY}" ]; then
mkdir -p ~/.ssh
echo "${SSH_PRIVATE_KEY}" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
ssh-keyscan -p 2222 dogar.darkhelm.org >> ~/.ssh/known_hosts 2>/dev/null
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
# Clone just enough to get the Dockerfiles
GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" \
git clone --depth 1 --no-checkout \
ssh://git@dogar.darkhelm.org:2222/DarkHelm.org/plex-playlist.git .
REPO_OWNER="${REPO_FULL%/*}"
REPO_NAME="${REPO_FULL#*/}"
TARGET_REF="${HEAD_REF:-${REF_NAME}}"
TRACE_ID="cicd-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${HEAD_SHA:0:8}"
# Checkout Dockerfiles and dockerignore (include base for fallback)
git checkout HEAD -- Dockerfile.cicd Dockerfile.cicd-base .dockerignore
echo "trace_id=${TRACE_ID}"
echo "target_ref=${TARGET_REF}"
CANDIDATE_API_BASES=()
if [ -n "${GITHUB_SERVER_URL:-}" ]; then
CANDIDATE_API_BASES+=("${GITHUB_SERVER_URL%/}/api/v1")
fi
CANDIDATE_API_BASES+=("http://${GITEA_REGISTRY_IP}:3001/api/v1")
CANDIDATE_API_BASES+=("http://${GITEA_REGISTRY_HOST}:3001/api/v1")
# Clean up SSH key for security
rm -f ~/.ssh/id_rsa
ensure_curl() {
if command -v curl >/dev/null 2>&1; then
return 0
fi
if command -v apt-get >/dev/null 2>&1; then
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq curl ca-certificates
fi
command -v curl >/dev/null 2>&1
}
echo "✓ Dockerfile.cicd and fallback base ready for secure build"
ensure_curl || { echo "❌ curl unavailable for dispatch"; exit 1; }
- name: Build and push complete CICD image
env:
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
GITHUB_SHA: ${{ github.sha }}
REGISTRY_USER: ${{ secrets.REGISTRY_USER || github.actor }}
HELPER_PATH="/tmp/dispatch-workflow.sh"
fetch_dispatch_helper() {
local helper_ref="$1"
local api_base
for api_base in "${CANDIDATE_API_BASES[@]}"; do
helper_url="${api_base}/repos/${REPO_OWNER}/${REPO_NAME}/raw/scripts/dispatch-workflow.sh?ref=${helper_ref}"
if curl -fsS --connect-timeout 5 --max-time 20 \
-H "Authorization: token ${DISPATCH_TOKEN}" \
-H "User-Agent: plex-playlist-cicd" \
-o "${HELPER_PATH}" \
"${helper_url}"; then
chmod +x "${HELPER_PATH}"
return 0
fi
done
return 1
}
if ! fetch_dispatch_helper "${TARGET_REF}" && ! fetch_dispatch_helper "${HEAD_SHA}"; then
echo "❌ Failed to fetch scripts/dispatch-workflow.sh from repository"
exit 1
fi
DISPATCH_ARGS=(
--token "${DISPATCH_TOKEN}"
--repo "${REPO_FULL}"
--workflow "cicd-start.yaml"
--ref "${TARGET_REF}"
--head-sha "${HEAD_SHA}"
--source-workflow "CICD"
--trace-id "${TRACE_ID}"
)
for API_BASE in "${CANDIDATE_API_BASES[@]}"; do
DISPATCH_ARGS+=(--api-base "${API_BASE}")
done
"${HELPER_PATH}" "${DISPATCH_ARGS[@]}"
- name: Failure diagnostics
if: failure()
run: |
echo "=== Building Complete CICD Image with Secure Secrets ==="
echo "=== Failure Diagnostics ==="
date -u '+timestamp_utc=%Y-%m-%dT%H:%M:%SZ'
echo "runner_name=${RUNNER_NAME:-unknown}"
echo "runner_hostname=${HOSTNAME:-unknown}"
uname -a || true
cat /etc/os-release 2>/dev/null || true
df -h || true
free -h || true
ps aux --sort=-%mem | head -n 30 || true
# Login to registry
echo "${PACKAGE_ACCESS_TOKEN}" | docker login dogar.darkhelm.org -u "${REGISTRY_USER}" --password-stdin
# Verify base image availability with fallback strategy
BASE_IMAGE="dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd-base:latest"
echo "Checking base image availability: ${BASE_IMAGE}"
if docker pull "${BASE_IMAGE}" 2>/dev/null; then
echo "✓ Base image pulled successfully from registry"
if command -v docker >/dev/null 2>&1; then
echo "=== Docker Diagnostics ==="
docker version || true
docker info || true
docker ps -a || true
docker images --digests | head -n 50 || true
else
echo "❌ Failed to pull base image from registry"
echo "This might be the first run - checking if we need to build base locally..."
# Check if base Dockerfile exists and build it locally as fallback
if [ -f "Dockerfile.cicd-base" ]; then
echo "Building base image locally as fallback..."
export DOCKER_BUILDKIT=1
docker build -f Dockerfile.cicd-base -t cicd-base-local:latest .
BASE_IMAGE="cicd-base-local:latest"
echo "✓ Base image built locally: ${BASE_IMAGE}"
else
echo "❌ Cannot find Dockerfile.cicd-base for fallback build"
exit 1
fi
echo "docker not available on this runner"
fi
# Create temporary SSH key file for BuildKit secrets
echo "${SSH_PRIVATE_KEY}" > /tmp/ssh_key
chmod 600 /tmp/ssh_key
# Enable Docker BuildKit for secrets support
export DOCKER_BUILDKIT=1
# Build complete CICD image using secure BuildKit secrets, inheriting from base
# SSH key is mounted securely and never stored in image layers
echo "Building complete image with base: ${BASE_IMAGE}"
docker build -f Dockerfile.cicd \
--secret id=ssh_private_key,src=/tmp/ssh_key \
--build-arg GITHUB_SHA="$GITHUB_SHA" \
--build-arg CICD_BASE_IMAGE="${BASE_IMAGE}" \
-t cicd:latest .
# Clean up temporary SSH key file
rm -f /tmp/ssh_key
# Tag for Gitea container registry
docker tag cicd:latest dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:latest
docker tag cicd:latest dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
# Push to registry
echo "Pushing complete CICD images to registry..."
docker push dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:latest
docker push dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
echo "✓ Complete CICD image built and pushed to registry"
# Pre-commit style checks - General file formatting
trailing-whitespace:
name: Trailing Whitespace Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Check trailing whitespace with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run trailing-whitespace --all-files --show-diff-on-failure
"
end-of-file-fixer:
name: End of File Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Check end of file with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run end-of-file-fixer --all-files --show-diff-on-failure
"
check-yaml:
name: YAML Syntax Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Check YAML files with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run check-yaml --all-files --show-diff-on-failure
"
check-toml:
name: TOML Syntax Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Check TOML files with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run check-toml --all-files --show-diff-on-failure
"
mixed-line-ending:
name: Mixed Line Ending Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Check line endings with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run mixed-line-ending --all-files --show-diff-on-failure
"
toml-lint:
name: TOML Formatting Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: |
echo "=== Network-Resilient Docker Registry Login ==="
for i in 1 2 3 4 5; do
echo "Login attempt $i/5..."
if echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | timeout 60 docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin; then
echo "✓ Login successful"
break
else
if [ $i -eq 5 ]; then
echo "❌ All login attempts failed after network timeouts"
exit 1
fi
echo "⚠ Login attempt $i failed, waiting 15s before retry..."
sleep 15
fi
done
- name: Check TOML formatting with pre-commit
run: |
echo "=== Network-Resilient Docker Operations ==="
# Resilient docker pull with retries
for i in 1 2 3; do
echo "Docker pull attempt $i/3..."
if timeout 300 docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}; then
echo "✓ Docker pull successful"
break
else
if [ $i -eq 3 ]; then
echo "❌ All docker pull attempts failed"
exit 1
fi
echo "⚠ Docker pull attempt $i failed, waiting 20s before retry..."
sleep 20
fi
done
# Run the actual test
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run pretty-format-toml --all-files --show-diff-on-failure
"
# Backend Python checks
ruff-lint:
name: Ruff Linting
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run ruff linting with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run ruff --all-files --show-diff-on-failure
"
ruff-format:
name: Ruff Format Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Check ruff formatting with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run ruff-format --all-files --show-diff-on-failure
"
pyright:
name: Pyright Type Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run pyright type checking with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run pyright --all-files --show-diff-on-failure
"
darglint:
name: Darglint Docstring Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run darglint docstring linting with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run darglint --all-files --show-diff-on-failure
"
no-docstring-types:
name: No Docstring Types Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run no docstring types check with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run no-docstring-types --all-files --show-diff-on-failure
"
# Frontend checks
eslint:
name: ESLint Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run ESLint with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run eslint --all-files --show-diff-on-failure
"
prettier:
name: Prettier Format Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Check Prettier formatting with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run prettier --all-files --show-diff-on-failure
"
typescript-check:
name: TypeScript Type Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run TypeScript type checking with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run typescript-check --all-files --show-diff-on-failure
"
tsdoc-lint:
name: TSDoc Lint Check
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run TSDoc linting with pre-commit
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace &&
pre-commit run tsdoc-lint --all-files --show-diff-on-failure
"
# Unit tests with coverage
backend-tests:
name: Backend Tests
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run backend tests with coverage
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace/backend &&
source .venv/bin/activate &&
uv run pytest -v --tb=short --cov=src --cov-report=term-missing --cov-fail-under=95
"
frontend-tests:
name: Frontend Tests
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run frontend tests with coverage
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace/frontend &&
yarn test:coverage --run --reporter=verbose --coverage.reporter=text --coverage.reporter=text-summary --coverage.thresholds.lines=85 --coverage.thresholds.functions=85 --coverage.thresholds.branches=85 --coverage.thresholds.statements=85
"
# Doctest for backend
xdoctest:
name: Backend Doctests
runs-on: ubuntu-act
needs: setup
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run backend doctests
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace/backend &&
source .venv/bin/activate &&
echo 'Running doctests...' &&
if uv run xdoctest src/ --quiet; then
echo '✓ All doctests passed'
else
echo 'ℹ No doctests found or some doctests failed'
# Don't fail the build for missing doctests, only for failed ones
if uv run xdoctest src/ --quiet --verbose 2>&1 | grep -q 'FAILED'; then
exit 1
fi
fi
"
# Integration and E2E tests (run after unit tests complete)
integration-tests:
name: Integration Tests
runs-on: ubuntu-act
needs: [backend-tests, frontend-tests]
steps:
- name: Login to Gitea Container Registry
run: echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin
- name: Run integration tests
run: |
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
docker run --rm dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace/backend &&
source .venv/bin/activate &&
if [ -d 'tests/integration' ]; then
uv run pytest tests/integration/ -v --tb=short
else
echo 'ℹ No integration tests found'
fi
"
e2e-tests:
name: End-to-End Tests
runs-on: ubuntu-act
needs: [backend-tests, frontend-tests]
steps:
- name: Login to Gitea Container Registry
run: |
echo "=== Network-Resilient Docker Registry Login ==="
for i in 1 2 3 4 5; do
echo "Login attempt $i/5..."
if echo "${{ secrets.PACKAGE_ACCESS_TOKEN }}" | timeout 60 docker login dogar.darkhelm.org -u ${{ github.actor }} --password-stdin; then
echo "✓ Login successful"
break
else
if [ $i -eq 5 ]; then
echo "❌ All login attempts failed after network timeouts"
exit 1
fi
echo "⚠ Login attempt $i failed, waiting 15s before retry..."
sleep 15
fi
done
- name: Run E2E tests
run: |
echo "=== Network-Resilient E2E Test Execution ==="
# Use the same simple approach as all other successful tests
docker pull dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest}
# Run E2E tests with network resilience
docker run --rm -e CI=true dogar.darkhelm.org/darkhelm.org/plex-playlist/cicd:${GITHUB_SHA:-latest} bash -c "
cd /workspace/frontend &&
if [ -d 'tests/e2e' ] || grep -q 'playwright' package.json; then
echo 'Running E2E tests with Playwright (Network Resilient)...' &&
export CI=true &&
export NODE_ENV=test &&
# Enhanced network-resilient Playwright setup
echo 'Verifying Playwright installation...' &&
yarn playwright --version &&
echo 'Installing Playwright browser binaries with enhanced retries...' &&
for i in 1 2 3 4; do
echo \"Enhanced browser install attempt \$i/4...\" &&
# Use longer timeout and Chromium-only for CI reliability
if timeout 900 yarn playwright install --with-deps chromium; then
echo \"✓ Playwright Chromium browser installed successfully\" &&
break
else
if [ \$i -eq 4 ]; then
echo \"❌ All enhanced browser install attempts failed\" &&
echo \"Attempting emergency fallback without dependencies...\" &&
if timeout 600 yarn playwright install chromium; then
echo \"✓ Emergency fallback browser install successful\" &&
break
else
echo \"❌ Emergency fallback also failed\" &&
exit 1
fi
fi
echo \"⚠ Enhanced browser install attempt \$i failed, waiting 45s before retry...\" &&
# Clear any partial browser downloads
rm -rf ~/.cache/ms-playwright 2>/dev/null || true &&
sleep 45
fi
done &&
echo 'Running E2E tests with network resilience...' &&
# Set additional network timeout environment variables
export PLAYWRIGHT_TIMEOUT=90000 &&
export NODE_TLS_REJECT_UNAUTHORIZED=0 &&
yarn test:e2e --reporter=list --timeout=90000
else
echo 'ℹ No E2E tests found'
fi
"
echo "=== Kernel Tail ==="
dmesg | tail -n 120 || true

537
.gitea/workflows/docker-build-base.yaml Normal file
View File

@@ -0,0 +1,537 @@
name: Docker Build Base
on:
workflow_dispatch:
inputs:
force_rebuild:
description: Force a rebuild even when the immutable base tag already exists
required: false
default: "false"
head_sha:
description: Commit SHA to process
required: false
source_workflow:
description: Upstream workflow name
required: false
trace_id:
description: Correlation id propagated across CICD dispatch chain
required: false
concurrency:
group: base-${{ github.sha }}
cancel-in-progress: true
env:
GITEA_SSH_HOST: kankali.darkhelm.lan
GITEA_SSH_PORT: "2222"
GITEA_REPO_SSH_URL: ssh://git@kankali.darkhelm.lan:2222/DarkHelm.org/plex-playlist.git
GITEA_REGISTRY: kankali.darkhelm.lan:3001
GITEA_REGISTRY_IP: 10.18.75.2
GITEA_REGISTRY_HOST: kankali.darkhelm.lan
defaults:
run:
shell: bash
jobs:
publish-base:
name: Build and Publish CICD Base Image
# Use ubuntu-act runner pool which is known to be available.
runs-on: ubuntu-act
timeout-minutes: 35
outputs:
base_hash: ${{ steps.base-state.outputs.base_hash }}
head_sha: ${{ steps.meta.outputs.head_sha }}
steps:
- name: Identify runner
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- name: Emit startup diagnostics
shell: sh
env:
EVENT_NAME: ${{ github.event_name }}
SOURCE_WORKFLOW: ${{ github.event.inputs.source_workflow }}
FORCE_REBUILD: ${{ github.event.inputs.force_rebuild }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
HEAD_SHA_INPUT: ${{ github.event.inputs.head_sha }}
HEAD_SHA_FALLBACK: ${{ github.sha }}
REF: ${{ github.ref }}
REF_NAME: ${{ github.ref_name }}
HEAD_REF: ${{ github.head_ref }}
TARGET_LABEL: ubuntu-act
run: |
RESOLVED_HEAD_SHA="${HEAD_SHA_INPUT:-${HEAD_SHA_FALLBACK}}"
TRACE_SUFFIX="$(printf '%s' "${RESOLVED_HEAD_SHA}" | cut -c1-8)"
TRACE_ID="${TRACE_ID_INPUT:-cicd-base-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${TRACE_SUFFIX}}"
echo "=== Base Workflow Startup Audit ==="
echo "event_name=${EVENT_NAME}"
echo "source_workflow=${SOURCE_WORKFLOW}"
echo "force_rebuild=${FORCE_REBUILD}"
echo "head_sha_input=${HEAD_SHA_INPUT}"
echo "head_sha=${RESOLVED_HEAD_SHA}"
echo "ref=${REF}"
echo "ref_name=${REF_NAME}"
echo "head_ref=${HEAD_REF}"
echo "trace_id=${TRACE_ID}"
echo "target_runner_label=${TARGET_LABEL}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo "startup_audit=ok"
- name: Preflight Docker and registry connectivity
env:
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
REGISTRY_USER: ${{ github.actor }}
run: |
set -e
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
echo "=== Docker Preflight ==="
command -v docker
docker --version
docker info >/tmp/docker-info.log 2>&1 || (echo "❌ docker info failed" && tail -n 40 /tmp/docker-info.log && exit 1)
grep -qi "${GITEA_REGISTRY}" /tmp/docker-info.log || echo "⚠ Registry not listed in docker info insecure registries: ${GITEA_REGISTRY}"
echo "=== Registry Login Preflight ==="
if [ -z "${PACKAGE_ACCESS_TOKEN}" ]; then
echo "❌ PACKAGE_ACCESS_TOKEN is empty"
exit 1
fi
echo "${PACKAGE_ACCESS_TOKEN}" | docker login "http://${GITEA_REGISTRY}" -u "${REGISTRY_USER}" --password-stdin >/tmp/docker-login.log 2>&1 || (echo "❌ Registry login preflight failed" && tail -n 40 /tmp/docker-login.log && exit 1)
echo "registry_login_preflight=ok"
- name: Audit trigger context
env:
EVENT_NAME: ${{ github.event_name }}
SOURCE_WORKFLOW: ${{ github.event.inputs.source_workflow }}
FORCE_REBUILD: ${{ github.event.inputs.force_rebuild }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
HEAD_SHA_INPUT: ${{ github.event.inputs.head_sha }}
HEAD_SHA_FALLBACK: ${{ github.sha }}
REF: ${{ github.ref }}
REF_NAME: ${{ github.ref_name }}
HEAD_REF: ${{ github.head_ref }}
run: |
RESOLVED_HEAD_SHA="${HEAD_SHA_INPUT:-${HEAD_SHA_FALLBACK}}"
TRACE_ID="${TRACE_ID_INPUT:-cicd-base-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${RESOLVED_HEAD_SHA:0:8}}"
echo "=== Dispatch Audit: CICD Base Image ==="
echo "event_name=${EVENT_NAME}"
echo "source_workflow=${SOURCE_WORKFLOW}"
echo "force_rebuild=${FORCE_REBUILD}"
echo "head_sha_input=${HEAD_SHA_INPUT}"
echo "head_sha=${RESOLVED_HEAD_SHA}"
echo "ref=${REF}"
echo "ref_name=${REF_NAME}"
echo "head_ref=${HEAD_REF}"
echo "trace_id=${TRACE_ID}"
- name: Resolve head SHA
id: meta
env:
HEAD_SHA_INPUT: ${{ github.event.inputs.head_sha }}
HEAD_SHA_FALLBACK: ${{ github.sha }}
run: |
RESOLVED_HEAD_SHA="${HEAD_SHA_INPUT:-${HEAD_SHA_FALLBACK}}"
echo "head_sha=${RESOLVED_HEAD_SHA}" >> "$GITHUB_OUTPUT"
- name: Minimal checkout for base inputs
env:
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
GITHUB_SHA: ${{ steps.meta.outputs.head_sha }}
run: |
umask 077
trap 'rm -f ~/.ssh/id_rsa' EXIT
echo "=== Minimal Repository Checkout for Base Inputs ==="
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
if [ -n "${SSH_PRIVATE_KEY}" ]; then
mkdir -p ~/.ssh
echo "${SSH_PRIVATE_KEY}" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
echo "Loaded SSH key fingerprint: $(ssh-keygen -lf ~/.ssh/id_rsa | awk '{print $2}')"
ssh-keyscan -p "${GITEA_SSH_PORT}" "${GITEA_SSH_HOST}" >> ~/.ssh/known_hosts 2>/dev/null
else
echo "❌ SSH_PRIVATE_KEY is empty"
exit 1
fi
GIT_SSH_COMMAND="ssh -i ~/.ssh/id_rsa -o IdentitiesOnly=yes -o StrictHostKeyChecking=no" \
git clone --depth 1 --no-checkout \
"${GITEA_REPO_SSH_URL}" .
if [ -n "${GITHUB_SHA}" ] && \
GIT_SSH_COMMAND="ssh -i ~/.ssh/id_rsa -o IdentitiesOnly=yes -o StrictHostKeyChecking=no" \
git fetch --depth 1 origin "${GITHUB_SHA}" >/dev/null 2>&1; then
git checkout FETCH_HEAD -- Dockerfile.cicd-base .dockerignore scripts/compute-cicd-base-hash.sh
echo "✓ Checked out base inputs from commit ${GITHUB_SHA}"
else
git checkout HEAD -- Dockerfile.cicd-base .dockerignore scripts/compute-cicd-base-hash.sh
echo "⚠ Falling back to default branch HEAD for base inputs checkout"
fi
chmod +x scripts/compute-cicd-base-hash.sh
rm -f ~/.ssh/id_rsa
- name: Compute base hash and inspect registry
id: base-state
env:
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
REGISTRY_USER: ${{ github.actor }}
FORCE_REBUILD: ${{ github.event.inputs.force_rebuild }}
run: |
echo "=== Computing CICD Base Image Hash ==="
BASE_HASH=$(./scripts/compute-cicd-base-hash.sh)
BASE_REF_HASH="${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd-base:${BASE_HASH}"
BASE_REF_LATEST="${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd-base:latest"
echo "Base hash: ${BASE_HASH}"
echo "base_hash=${BASE_HASH}" >> $GITHUB_OUTPUT
echo "base_ref_hash=${BASE_REF_HASH}" >> $GITHUB_OUTPUT
echo "base_ref_latest=${BASE_REF_LATEST}" >> $GITHUB_OUTPUT
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
if ! echo "${PACKAGE_ACCESS_TOKEN}" | docker login "http://${GITEA_REGISTRY}" -u "${REGISTRY_USER}" --password-stdin >/dev/null 2>&1; then
echo "❌ Registry login failed during base inspection"
exit 1
fi
if [ "${FORCE_REBUILD}" = "true" ]; then
echo "Manual force rebuild requested"
echo "needs_build=true" >> $GITHUB_OUTPUT
exit 0
fi
for i in 1 2 3; do
echo "Existing base check ${i}/3 for ${BASE_REF_HASH}..."
# Manifest inspect is unreliable on some runners; use pull as truth.
if timeout 900 docker pull "${BASE_REF_HASH}" >/tmp/base-state-pull.log 2>&1; then
echo "✓ Immutable base image already exists and is pullable: ${BASE_REF_HASH}"
echo "needs_build=false" >> $GITHUB_OUTPUT
exit 0
else
pull_exit=$?
echo "Existing base pull check failed with exit code ${pull_exit}"
if [ "${pull_exit}" -eq 124 ]; then
echo "Existing base pull timed out after 900s"
fi
if [ -s /tmp/base-state-pull.log ]; then
tail -n 20 /tmp/base-state-pull.log || true
fi
fi
if [ "${i}" -lt 3 ]; then
sleep 10
fi
done
echo "ℹ Immutable base image missing or not yet pullable: ${BASE_REF_HASH}"
echo "needs_build=true" >> $GITHUB_OUTPUT
- name: Write registry push helpers
run: |
cat > /tmp/registry-push-helpers.sh <<'EOF'
#!/bin/bash
ensure_skopeo() {
if command -v skopeo >/dev/null 2>&1; then
return 0
fi
if command -v apt-get >/dev/null 2>&1; then
apt-get update && apt-get install -y skopeo
elif command -v apk >/dev/null 2>&1; then
apk add --no-cache skopeo
fi
if ! command -v skopeo >/dev/null 2>&1; then
echo "❌ skopeo not available for fallback push"
return 1
fi
}
push_ref_with_fallback() {
ref="$1"
local_image="$2"
archive_path="$3"
registry_user="$4"
package_access_token="$5"
for i in 1 2 3; do
echo "Docker push attempt ${i}/3 for ${ref}..."
if docker push "${ref}"; then
echo "✓ Docker push succeeded for ${ref}"
return 0
fi
if [ "${i}" -lt 3 ]; then
sleep 10
fi
done
echo "⚠ Docker push failed for ${ref}; trying skopeo fallback"
if ! ensure_skopeo; then
return 1
fi
if [ ! -f "${archive_path}" ]; then
echo "Creating local image archive for fallback push..."
if ! docker save "${local_image}" -o "${archive_path}"; then
echo "❌ Failed to create docker archive for fallback push"
return 1
fi
fi
if skopeo copy \
--dest-creds "${registry_user}:${package_access_token}" \
--dest-tls-verify=false \
"docker-archive:${archive_path}" \
"docker://${ref}"; then
echo "✓ Skopeo fallback push succeeded for ${ref}"
return 0
fi
echo "❌ Skopeo fallback push failed for ${ref}"
return 1
}
EOF
chmod 700 /tmp/registry-push-helpers.sh
- name: Build and push base image
if: steps.base-state.outputs.needs_build == 'true'
env:
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
REGISTRY_USER: ${{ github.actor }}
BASE_HASH: ${{ steps.base-state.outputs.base_hash }}
BASE_REF_HASH: ${{ steps.base-state.outputs.base_ref_hash }}
BASE_REF_LATEST: ${{ steps.base-state.outputs.base_ref_latest }}
run: |
echo "=== Building CICD Base Image ==="
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
if ! echo "${PACKAGE_ACCESS_TOKEN}" | docker login "http://${GITEA_REGISTRY}" -u "${REGISTRY_USER}" --password-stdin; then
echo "❌ Registry login failed before build"
exit 1
fi
PLAYWRIGHT_BROWSERS_MIRROR_TAG="${GITEA_REGISTRY}/darkhelm.org/playwright-browsers:v1.56.1-jammy"
echo "playwright_browsers_mirror_tag=${PLAYWRIGHT_BROWSERS_MIRROR_TAG}"
if ! docker pull "${PLAYWRIGHT_BROWSERS_MIRROR_TAG}" >/tmp/playwright-mirror-pull.log 2>&1; then
echo "❌ Required internal Playwright browsers mirror image is missing: ${PLAYWRIGHT_BROWSERS_MIRROR_TAG}"
echo "Mirror this image into your internal registry before running base builds."
tail -n 80 /tmp/playwright-mirror-pull.log || true
exit 1
fi
PLAYWRIGHT_BROWSERS_IMAGE=$(docker image inspect --format='{{index .RepoDigests 0}}' "${PLAYWRIGHT_BROWSERS_MIRROR_TAG}" 2>/dev/null || true)
if [ -z "${PLAYWRIGHT_BROWSERS_IMAGE}" ]; then
echo "❌ Failed to resolve immutable digest for ${PLAYWRIGHT_BROWSERS_MIRROR_TAG}"
exit 1
fi
echo "playwright_browsers_image=${PLAYWRIGHT_BROWSERS_IMAGE}"
export DOCKER_BUILDKIT=1
BUILD_TIMEOUT_SECONDS=5400
BUILD_START_EPOCH=$(date +%s)
echo "docker_build_timeout_seconds=${BUILD_TIMEOUT_SECONDS}"
echo "docker_build_started_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
timeout "${BUILD_TIMEOUT_SECONDS}" docker build --progress=plain -f Dockerfile.cicd-base \
--build-arg BASE_IMAGE_VERSION="v1.0.0-${BASE_HASH}" \
--build-arg BASE_IMAGE_HASH="${BASE_HASH}" \
--build-arg PLAYWRIGHT_BROWSERS_IMAGE="${PLAYWRIGHT_BROWSERS_IMAGE}" \
-t cicd-base:latest .
BUILD_EXIT_CODE=$?
if [ "${BUILD_EXIT_CODE}" -ne 0 ]; then
if [ "${BUILD_EXIT_CODE}" -eq 124 ]; then
echo "❌ docker build timed out after ${BUILD_TIMEOUT_SECONDS}s"
else
echo "❌ docker build failed with exit code ${BUILD_EXIT_CODE}"
fi
exit "${BUILD_EXIT_CODE}"
fi
BUILD_END_EPOCH=$(date +%s)
BUILD_ELAPSED=$((BUILD_END_EPOCH - BUILD_START_EPOCH))
echo "docker_build_elapsed_seconds=${BUILD_ELAPSED}"
ARCHIVE_PATH="/tmp/cicd-base.tar"
source /tmp/registry-push-helpers.sh
docker tag cicd-base:latest "${BASE_REF_HASH}"
docker tag cicd-base:latest "${BASE_REF_LATEST}"
push_ref_with_fallback "${BASE_REF_HASH}" "cicd-base:latest" "${ARCHIVE_PATH}" "${REGISTRY_USER}" "${PACKAGE_ACCESS_TOKEN}"
push_ref_with_fallback "${BASE_REF_LATEST}" "cicd-base:latest" "${ARCHIVE_PATH}" "${REGISTRY_USER}" "${PACKAGE_ACCESS_TOKEN}"
- name: Verify published base image
env:
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
REGISTRY_USER: ${{ github.actor }}
BASE_REF_HASH: ${{ steps.base-state.outputs.base_ref_hash }}
run: |
echo "=== Verifying Published CICD Base Image ==="
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
if ! echo "${PACKAGE_ACCESS_TOKEN}" | docker login "http://${GITEA_REGISTRY}" -u "${REGISTRY_USER}" --password-stdin; then
echo "❌ Registry login failed during verification"
exit 1
fi
for i in 1 2 3 4 5 6; do
echo "Verification attempt ${i}/6 for ${BASE_REF_HASH}..."
if docker pull "${BASE_REF_HASH}"; then
echo "✓ Published base image is pullable: ${BASE_REF_HASH}"
exit 0
fi
if [ "${i}" -lt 6 ]; then
echo "⚠ Pull failed; waiting 20s before retry"
sleep 20
fi
done
echo "❌ Published base image could not be pulled after 6 attempts: ${BASE_REF_HASH}"
exit 1
- name: Dispatch main build workflow
env:
ACTIONS_TRIGGER_TOKEN: ${{ secrets.ACTIONS_TRIGGER_TOKEN }}
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
HEAD_SHA: ${{ steps.meta.outputs.head_sha }}
BASE_HASH: ${{ steps.base-state.outputs.base_hash }}
REPO_FULL: ${{ github.repository }}
HEAD_REF: ${{ github.head_ref }}
REF_NAME: ${{ github.ref_name }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
run: |
set -e
DISPATCH_TOKEN="${ACTIONS_TRIGGER_TOKEN:-${PACKAGE_ACCESS_TOKEN:-}}"
if [ -z "${DISPATCH_TOKEN}" ]; then
echo "❌ Missing dispatch token. Set ACTIONS_TRIGGER_TOKEN (repo write scope) or ensure PACKAGE_ACCESS_TOKEN has Actions workflow-dispatch permissions."
exit 1
fi
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
REPO_OWNER="${REPO_FULL%/*}"
REPO_NAME="${REPO_FULL#*/}"
TARGET_REF="${HEAD_REF:-${REF_NAME}}"
TRACE_ID="${TRACE_ID_INPUT:-cicd-base-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${HEAD_SHA:0:8}}"
echo "trace_id=${TRACE_ID}"
echo "target_ref=${TARGET_REF}"
CANDIDATE_API_BASES=()
if [ -n "${GITHUB_SERVER_URL:-}" ]; then
CANDIDATE_API_BASES+=("${GITHUB_SERVER_URL%/}/api/v1")
fi
CANDIDATE_API_BASES+=("http://${GITEA_REGISTRY_IP}:3001/api/v1")
CANDIDATE_API_BASES+=("http://${GITEA_REGISTRY_HOST}:3001/api/v1")
if ! command -v curl >/dev/null 2>&1; then
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq curl ca-certificates
fi
HELPER_PATH="/tmp/dispatch-workflow.sh"
fetch_dispatch_helper() {
local helper_ref="$1"
local api_base
for api_base in "${CANDIDATE_API_BASES[@]}"; do
helper_url="${api_base}/repos/${REPO_OWNER}/${REPO_NAME}/raw/scripts/dispatch-workflow.sh?ref=${helper_ref}"
if curl -fsS --connect-timeout 5 --max-time 20 \
-H "Authorization: token ${DISPATCH_TOKEN}" \
-H "User-Agent: plex-playlist-cicd-base" \
-o "${HELPER_PATH}" \
"${helper_url}"; then
chmod +x "${HELPER_PATH}"
return 0
fi
done
return 1
}
if ! fetch_dispatch_helper "${TARGET_REF}" && ! fetch_dispatch_helper "${HEAD_SHA}"; then
echo "❌ Failed to fetch scripts/dispatch-workflow.sh from repository"
exit 1
fi
DISPATCH_ARGS=(
--token "${DISPATCH_TOKEN}"
--repo "${REPO_FULL}"
--workflow "docker-build-main.yaml"
--ref "${TARGET_REF}"
--head-sha "${HEAD_SHA}"
--source-workflow "CICD Base Image"
--trace-id "${TRACE_ID}"
--base-needed "true"
--base-hash "${BASE_HASH}"
)
for API_BASE in "${CANDIDATE_API_BASES[@]}"; do
DISPATCH_ARGS+=(--api-base "${API_BASE}")
done
"${HELPER_PATH}" "${DISPATCH_ARGS[@]}"
- name: Failure diagnostics
if: failure()
run: |
echo "=== Failure Diagnostics ==="
date -u '+timestamp_utc=%Y-%m-%dT%H:%M:%SZ'
echo "runner_name=${RUNNER_NAME:-unknown}"
echo "runner_hostname=${HOSTNAME:-unknown}"
uname -a || true
cat /etc/os-release 2>/dev/null || true
df -h || true
free -h || true
ps aux --sort=-%mem | head -n 30 || true
if command -v docker >/dev/null 2>&1; then
echo "=== Docker Diagnostics ==="
docker version || true
docker info || true
docker ps -a || true
docker images --digests | head -n 50 || true
else
echo "docker not available on this runner"
fi
echo "=== Kernel Tail ==="
dmesg | tail -n 120 || true

401
.gitea/workflows/docker-build-main.yaml Normal file
View File

@@ -0,0 +1,401 @@
name: Docker Build Main
on:
workflow_dispatch:
inputs:
head_sha:
description: Commit SHA to process
required: false
base_hash:
description: Immutable base hash to use for CICD base image
required: false
source_workflow:
description: Upstream workflow name
required: false
base_needed:
description: Whether base rebuild was required
required: false
trace_id:
description: Correlation id propagated across CICD dispatch chain
required: false
env:
GITEA_SSH_HOST: kankali.darkhelm.lan
GITEA_SSH_PORT: "2222"
GITEA_REPO_SSH_URL: ssh://git@kankali.darkhelm.lan:2222/DarkHelm.org/plex-playlist.git
GITEA_REGISTRY: kankali.darkhelm.lan:3001
GITEA_REGISTRY_IP: 10.18.75.2
GITEA_REGISTRY_HOST: kankali.darkhelm.lan
defaults:
run:
shell: bash
concurrency:
group: main-build-${{ github.sha }}
cancel-in-progress: true
jobs:
startup-audit:
name: Main Workflow Startup Audit
# Pin startup audit to high-memory worker to avoid setup-stage runner churn.
runs-on: ubuntu-act-8gb
timeout-minutes: 5
steps:
- name: Identify runner
shell: sh
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- name: Emit startup diagnostics
shell: sh
env:
EVENT_NAME: ${{ github.event_name }}
SOURCE_WORKFLOW: ${{ github.event.inputs.source_workflow }}
BASE_NEEDED: ${{ github.event.inputs.base_needed }}
HEAD_SHA_INPUT: ${{ github.event.inputs.head_sha }}
HEAD_SHA_FALLBACK: ${{ github.sha }}
BASE_HASH_INPUT: ${{ github.event.inputs.base_hash }}
REF: ${{ github.ref }}
REF_NAME: ${{ github.ref_name }}
HEAD_REF: ${{ github.head_ref }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
TARGET_LABEL: ubuntu-act-8gb
run: |
RESOLVED_HEAD_SHA="${HEAD_SHA_INPUT:-${HEAD_SHA_FALLBACK}}"
TRACE_SUFFIX="$(printf '%s' "${RESOLVED_HEAD_SHA}" | cut -c1-8)"
TRACE_ID="${TRACE_ID_INPUT:-cicd-main-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${TRACE_SUFFIX}}"
echo "=== Main Workflow Startup Audit ==="
echo "event_name=${EVENT_NAME}"
echo "source_workflow=${SOURCE_WORKFLOW}"
echo "base_needed=${BASE_NEEDED}"
echo "head_sha_input=${HEAD_SHA_INPUT}"
echo "head_sha=${RESOLVED_HEAD_SHA}"
echo "base_hash_input=${BASE_HASH_INPUT}"
echo "ref=${REF}"
echo "ref_name=${REF_NAME}"
echo "head_ref=${HEAD_REF}"
echo "trace_id=${TRACE_ID}"
echo "target_runner_label=${TARGET_LABEL}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo "startup_audit=ok"
- name: Check control-plane reachability
shell: sh
run: |
SERVER_URL="${GITHUB_SERVER_URL:-}"
if command -v curl >/dev/null 2>&1; then
if [ -n "${SERVER_URL}" ] && curl -fsS --connect-timeout 5 --max-time 10 "${SERVER_URL%/}/api/v1/version" >/tmp/gitea-version.json 2>/dev/null; then
echo "gitea_api_reachable=true"
cat /tmp/gitea-version.json || true
else
echo "gitea_api_reachable=false"
fi
else
echo "curl unavailable; skipping reachability check"
fi
- &failure_diagnostics_step
name: Failure diagnostics
if: failure()
run: |
echo "=== Failure Diagnostics ==="
date -u '+timestamp_utc=%Y-%m-%dT%H:%M:%SZ'
echo "runner_name=${RUNNER_NAME:-unknown}"
echo "runner_hostname=${HOSTNAME:-unknown}"
uname -a || true
cat /etc/os-release 2>/dev/null || true
df -h || true
free -h || true
ps aux --sort=-%mem | head -n 30 || true
if command -v docker >/dev/null 2>&1; then
echo "=== Docker Diagnostics ==="
docker version || true
docker info || true
docker ps -a || true
docker images --digests | head -n 50 || true
else
echo "docker not available on this runner"
fi
echo "=== Kernel Tail ==="
dmesg | tail -n 120 || true
build:
name: Build and Push CICD Complete Image
# Pin main image build to high-memory worker to reduce setup-time failures.
runs-on: ubuntu-act-8gb
needs: startup-audit
timeout-minutes: 60
outputs:
head_sha: ${{ steps.meta.outputs.head_sha }}
steps:
- name: Identify runner
run: |
echo "=== Runner Identity ==="
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_os=${RUNNER_OS:-unknown}"
echo "runner_arch=${RUNNER_ARCH:-unknown}"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- name: Audit trigger context
env:
EVENT_NAME: ${{ github.event_name }}
SOURCE_WORKFLOW: ${{ github.event.inputs.source_workflow }}
BASE_NEEDED: ${{ github.event.inputs.base_needed }}
HEAD_SHA_INPUT: ${{ github.event.inputs.head_sha }}
HEAD_SHA_FALLBACK: ${{ github.sha }}
BASE_HASH_INPUT: ${{ github.event.inputs.base_hash }}
REF: ${{ github.ref }}
REF_NAME: ${{ github.ref_name }}
HEAD_REF: ${{ github.head_ref }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
run: |
RESOLVED_HEAD_SHA="${HEAD_SHA_INPUT:-${HEAD_SHA_FALLBACK}}"
TRACE_ID="${TRACE_ID_INPUT:-cicd-main-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${RESOLVED_HEAD_SHA:0:8}}"
echo "=== Dispatch Audit: CICD Main Build ==="
echo "event_name=${EVENT_NAME}"
echo "source_workflow=${SOURCE_WORKFLOW}"
echo "base_needed=${BASE_NEEDED}"
echo "head_sha_input=${HEAD_SHA_INPUT}"
echo "head_sha=${RESOLVED_HEAD_SHA}"
echo "base_hash_input=${BASE_HASH_INPUT}"
echo "ref=${REF}"
echo "ref_name=${REF_NAME}"
echo "head_ref=${HEAD_REF}"
echo "trace_id=${TRACE_ID}"
- name: Resolve head SHA
id: meta
env:
HEAD_SHA_INPUT: ${{ github.event.inputs.head_sha }}
HEAD_SHA_FALLBACK: ${{ github.sha }}
run: |
RESOLVED_HEAD_SHA="${HEAD_SHA_INPUT:-${HEAD_SHA_FALLBACK}}"
echo "head_sha=${RESOLVED_HEAD_SHA}" >> "$GITHUB_OUTPUT"
- name: Minimal checkout for build inputs
env:
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
HEAD_SHA: ${{ steps.meta.outputs.head_sha }}
run: |
set -e
umask 077
trap 'rm -f ~/.ssh/id_rsa' EXIT
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
mkdir -p ~/.ssh
echo "${SSH_PRIVATE_KEY}" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
ssh-keyscan -p "${GITEA_SSH_PORT}" "${GITEA_SSH_HOST}" >> ~/.ssh/known_hosts 2>/dev/null
GIT_SSH_COMMAND="ssh -i ~/.ssh/id_rsa -o IdentitiesOnly=yes -o StrictHostKeyChecking=no" \
git clone --depth 1 --no-checkout "${GITEA_REPO_SSH_URL}" .
GIT_SSH_COMMAND="ssh -i ~/.ssh/id_rsa -o IdentitiesOnly=yes -o StrictHostKeyChecking=no" \
git fetch --depth 1 origin "${HEAD_SHA}" >/dev/null 2>&1 || true
git checkout FETCH_HEAD -- Dockerfile.cicd Dockerfile.cicd-base .dockerignore scripts/compute-cicd-base-hash.sh
chmod +x scripts/compute-cicd-base-hash.sh
- name: Build and push complete CICD image
env:
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
REGISTRY_USER: ${{ github.actor }}
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
HEAD_SHA: ${{ steps.meta.outputs.head_sha }}
BASE_HASH_INPUT: ${{ github.event.inputs.base_hash }}
run: |
set -e
umask 077
trap 'rm -f /tmp/ssh_key' EXIT
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
echo "${PACKAGE_ACCESS_TOKEN}" | docker login "http://${GITEA_REGISTRY}" -u "${REGISTRY_USER}" --password-stdin
if [ -n "${BASE_HASH_INPUT}" ]; then
BASE_HASH="${BASE_HASH_INPUT}"
echo "Using provided immutable base hash from upstream workflow dispatch: ${BASE_HASH}"
else
BASE_HASH=$(./scripts/compute-cicd-base-hash.sh)
echo "No base_hash input provided; computed base hash locally: ${BASE_HASH}"
fi
BASE_IMAGE="${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd-base:${BASE_HASH}"
verify_base_image() {
# Fast path: base image already present locally.
if docker image inspect "${BASE_IMAGE}" >/dev/null 2>&1; then
return 0
fi
# Manifest inspect unreliable on these runners; use pull as truth.
timeout 1800 docker pull "${BASE_IMAGE}" >/tmp/base-image-pull.log 2>&1
}
max_attempts=2
sleep_seconds=15
for i in $(seq 1 "${max_attempts}"); do
echo "Base availability check ${i}/${max_attempts}..."
if verify_base_image; then
echo "✓ Base image available and hash matched: ${BASE_IMAGE}"
break
else
pull_exit=$?
echo "Base pull attempt ${i} failed with exit code ${pull_exit}"
if [ "${pull_exit}" -eq 124 ]; then
echo "Pull timed out after 1800s while downloading base image"
fi
if [ -s /tmp/base-image-pull.log ]; then
tail -n 40 /tmp/base-image-pull.log || true
fi
if [ "${i}" -eq "${max_attempts}" ]; then
echo "❌ Required immutable base image is not available or mismatched: ${BASE_IMAGE}"
exit 1
fi
sleep "${sleep_seconds}"
fi
done
echo "✓ Base image ready: ${BASE_IMAGE}"
echo "${SSH_PRIVATE_KEY}" > /tmp/ssh_key
chmod 600 /tmp/ssh_key
export DOCKER_BUILDKIT=1
docker build -f Dockerfile.cicd \
--secret id=ssh_private_key,src=/tmp/ssh_key \
--add-host "${GITEA_SSH_HOST}:${GITEA_REGISTRY_IP}" \
--build-arg GITHUB_SHA="${HEAD_SHA}" \
--build-arg CICD_BASE_IMAGE="${BASE_IMAGE}" \
-t cicd:latest .
docker tag cicd:latest "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:latest"
docker tag cicd:latest "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}"
docker push "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:latest"
docker push "${GITEA_REGISTRY}/darkhelm.org/plex-playlist-cicd:${HEAD_SHA}"
- *failure_diagnostics_step
dispatch-checks:
name: Dispatch CICD Checks
runs-on: ubuntu-act
timeout-minutes: 15
needs: build
if: needs.build.result == 'success'
steps:
- name: Dispatch checks workflow
env:
ACTIONS_TRIGGER_TOKEN: ${{ secrets.ACTIONS_TRIGGER_TOKEN }}
PACKAGE_ACCESS_TOKEN: ${{ secrets.PACKAGE_ACCESS_TOKEN }}
HEAD_SHA: ${{ needs.build.outputs.head_sha }}
REPO_FULL: ${{ github.repository }}
HEAD_REF: ${{ github.head_ref }}
REF_NAME: ${{ github.ref_name }}
TRACE_ID_INPUT: ${{ github.event.inputs.trace_id }}
run: |
set -e
DISPATCH_TOKEN="${ACTIONS_TRIGGER_TOKEN:-${PACKAGE_ACCESS_TOKEN:-}}"
if [ -z "${DISPATCH_TOKEN}" ]; then
echo "❌ Missing dispatch token. Set ACTIONS_TRIGGER_TOKEN (repo write scope) or ensure PACKAGE_ACCESS_TOKEN has Actions workflow-dispatch permissions."
exit 1
fi
if ! grep -q "${GITEA_REGISTRY_HOST}" /etc/hosts; then
echo "${GITEA_REGISTRY_IP} ${GITEA_REGISTRY_HOST}" >> /etc/hosts
fi
REPO_OWNER="${REPO_FULL%/*}"
REPO_NAME="${REPO_FULL#*/}"
TARGET_REF="${HEAD_REF:-${REF_NAME}}"
TRACE_ID="${TRACE_ID_INPUT:-cicd-main-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}-${HEAD_SHA:0:8}}"
echo "trace_id=${TRACE_ID}"
echo "target_ref=${TARGET_REF}"
CANDIDATE_API_BASES=()
if [ -n "${GITHUB_SERVER_URL:-}" ]; then
CANDIDATE_API_BASES+=("${GITHUB_SERVER_URL%/}/api/v1")
fi
CANDIDATE_API_BASES+=("http://${GITEA_REGISTRY_IP}:3001/api/v1")
CANDIDATE_API_BASES+=("http://${GITEA_REGISTRY_HOST}:3001/api/v1")
ensure_curl() {
if command -v curl >/dev/null 2>&1; then
return 0
fi
if command -v apt-get >/dev/null 2>&1; then
export DEBIAN_FRONTEND=noninteractive
apt-get update -qq
apt-get install -y -qq curl ca-certificates
fi
command -v curl >/dev/null 2>&1
}
ensure_curl || { echo "❌ curl unavailable for dispatch"; exit 1; }
HELPER_PATH="/tmp/dispatch-workflow.sh"
fetch_dispatch_helper() {
local helper_ref="$1"
local api_base
for api_base in "${CANDIDATE_API_BASES[@]}"; do
helper_url="${api_base}/repos/${REPO_OWNER}/${REPO_NAME}/raw/scripts/dispatch-workflow.sh?ref=${helper_ref}"
if curl -fsS --connect-timeout 5 --max-time 20 \
-H "Authorization: token ${DISPATCH_TOKEN}" \
-H "User-Agent: plex-playlist-cicd-main" \
-o "${HELPER_PATH}" \
"${helper_url}"; then
chmod +x "${HELPER_PATH}"
return 0
fi
done
return 1
}
if ! fetch_dispatch_helper "${TARGET_REF}" && ! fetch_dispatch_helper "${HEAD_SHA}"; then
echo "❌ Failed to fetch scripts/dispatch-workflow.sh from repository"
exit 1
fi
DISPATCH_ARGS=(
--token "${DISPATCH_TOKEN}"
--repo "${REPO_FULL}"
--workflow "cicd-checks.yaml"
--ref "${TARGET_REF}"
--head-sha "${HEAD_SHA}"
--source-workflow "CICD Main Build"
--trace-id "${TRACE_ID}"
)
for API_BASE in "${CANDIDATE_API_BASES[@]}"; do
DISPATCH_ARGS+=(--api-base "${API_BASE}")
done
"${HELPER_PATH}" "${DISPATCH_ARGS[@]}"
- *failure_diagnostics_step

55
.gitea/workflows/renovate.yml
View File

@@ -15,33 +15,11 @@ on:
jobs:
renovate:
name: Renovate Dependencies
# Non-heavy workflow: allow any host exposing the generic ubuntu-act label.
runs-on: ubuntu-act
timeout-minutes: 90
steps:
- name: Checkout repository
env:
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
run: |
echo "=== Repository Checkout for Renovate ==="
# Set up SSH key securely
if [ -n "${SSH_PRIVATE_KEY}" ]; then
mkdir -p ~/.ssh
echo "${SSH_PRIVATE_KEY}" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
ssh-keyscan -p 2222 dogar.darkhelm.org >> ~/.ssh/known_hosts 2>/dev/null
fi
# Clone repository
GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" \
git clone --depth 1 \
ssh://git@dogar.darkhelm.org:2222/DarkHelm.org/plex-playlist.git .
# Clean up SSH key
rm -f ~/.ssh/id_rsa
echo "✓ Repository checked out for Renovate processing"
- name: Setup Node.js for Renovate
run: |
echo "=== Setting up Node.js 24 for Renovate ==="
@@ -266,6 +244,7 @@ jobs:
env:
RENOVATE_TOKEN: ${{ secrets.RENOVATE_TOKEN }}
RENOVATE_DRY_RUN: ${{ inputs.dry_run }}
RENOVATE_CONFIG_FILE: renovate-config.js
LOG_LEVEL: info
run: |
echo "=== Running Renovate Bot ==="
@@ -282,10 +261,36 @@ jobs:
echo "🔍 Running in DRY-RUN mode (no changes will be made)"
fi
renovate --config-file=renovate-config.js DarkHelm.org/plex-playlist
renovate DarkHelm.org/plex-playlist
echo "✓ Renovate execution completed"
- name: Failure diagnostics
if: failure()
run: |
echo "=== Failure Diagnostics ==="
date -u '+timestamp_utc=%Y-%m-%dT%H:%M:%SZ'
echo "runner_name=${RUNNER_NAME:-unknown}"
echo "runner_hostname=${HOSTNAME:-unknown}"
uname -a || true
cat /etc/os-release 2>/dev/null || true
df -h || true
free -h || true
ps aux --sort=-%mem | head -n 30 || true
if command -v docker >/dev/null 2>&1; then
echo "=== Docker Diagnostics ==="
docker version || true
docker info || true
docker ps -a || true
docker images --digests | head -n 50 || true
else
echo "docker not available on this runner"
fi
echo "=== Kernel Tail ==="
dmesg | tail -n 120 || true
- name: Upload Renovate logs
if: always()
run: |

270
.gitea/workflows/runner-canary.yaml Normal file
View File

@@ -0,0 +1,270 @@
name: Runner Canary
on:
schedule:
- cron: '*/15 * * * *'
workflow_dispatch:
inputs:
target_label:
description: Runner label to probe for setup stability
required: false
default: ubuntu-act
probe_count:
description: Number of canary probe jobs to launch (max 20)
required: false
default: "12"
include_heavy:
description: Include heavy label probes (ubuntu-act-8gb and ubuntu-act-4gb)
required: false
default: "false"
jobs:
canary-scheduled:
name: Canary (${{ matrix.label }})
if: github.event_name == 'schedule'
runs-on: ${{ matrix.label }}
timeout-minutes: 6
strategy:
fail-fast: false
matrix:
label:
- ubuntu-act
- ubuntu-latest
steps:
- name: Runner startup audit
env:
LABEL: ${{ matrix.label }}
RUN_ID: ${{ github.run_id }}
RUN_ATTEMPT: ${{ github.run_attempt }}
REPOSITORY: ${{ github.repository }}
EVENT_NAME: ${{ github.event_name }}
REF: ${{ github.ref }}
run: |
echo "=== Runner Canary ==="
echo "label=${LABEL}"
echo "run_id=${RUN_ID}"
echo "run_attempt=${RUN_ATTEMPT}"
echo "repository=${REPOSITORY}"
echo "event_name=${EVENT_NAME}"
echo "ref=${REF}"
date -u '+timestamp_utc=%Y-%m-%dT%H:%M:%SZ'
uname -a
- name: Basic tool availability
run: |
set -e
for tool in bash sh date; do
command -v "${tool}" >/dev/null 2>&1
echo "tool_ok=${tool}"
done
- name: Docker and registry preflight
env:
LABEL: ${{ matrix.label }}
run: |
set -e
if [ "${LABEL}" = "ubuntu-act" ]; then
command -v docker >/dev/null 2>&1
echo "docker_ok=true"
if docker info 2>/dev/null | grep -qi "kankali.darkhelm.lan:3001"; then
echo "registry_config_ok=true"
else
echo "registry_config_ok=false"
echo "Missing Docker insecure registry config for kankali.darkhelm.lan:3001"
exit 1
fi
if docker pull kankali.darkhelm.lan:3001/darkhelm.org/act-ubuntu:act-latest >/tmp/canary-mirror-pull.log 2>&1; then
echo "mirror_pull_ok=true"
else
if grep -qi "http response to https client" /tmp/canary-mirror-pull.log; then
echo "mirror_pull_ok=false"
echo "Detected HTTPS/HTTP registry mismatch for kankali.darkhelm.lan:3001"
fi
tail -n 20 /tmp/canary-mirror-pull.log || true
exit 1
fi
else
echo "docker_preflight=skipped_for_${LABEL}"
fi
- &failure_diagnostics_step
name: Failure diagnostics
if: failure()
run: |
echo "=== Failure Diagnostics ==="
date -u '+timestamp_utc=%Y-%m-%dT%H:%M:%SZ'
echo "runner_name=${RUNNER_NAME:-unknown}"
echo "runner_hostname=${HOSTNAME:-unknown}"
uname -a || true
cat /etc/os-release 2>/dev/null || true
df -h || true
free -h || true
ps aux --sort=-%mem | head -n 30 || true
if command -v docker >/dev/null 2>&1; then
echo "=== Docker Diagnostics ==="
docker version || true
docker info || true
docker ps -a || true
docker images --digests | head -n 50 || true
else
echo "docker not available on this runner"
fi
echo "=== Kernel Tail ==="
dmesg | tail -n 120 || true
canary-manual-heavy:
name: Canary Heavy (${{ matrix.label }})
if: github.event_name == 'workflow_dispatch' && github.event.inputs.include_heavy == 'true'
runs-on: ${{ matrix.label }}
timeout-minutes: 8
strategy:
fail-fast: false
matrix:
label:
- ubuntu-act-8gb
- ubuntu-act-4gb
steps:
- name: Runner startup audit
env:
LABEL: ${{ matrix.label }}
RUN_ID: ${{ github.run_id }}
RUN_ATTEMPT: ${{ github.run_attempt }}
REPOSITORY: ${{ github.repository }}
EVENT_NAME: ${{ github.event_name }}
REF: ${{ github.ref }}
run: |
echo "=== Runner Canary Heavy ==="
echo "label=${LABEL}"
echo "run_id=${RUN_ID}"
echo "run_attempt=${RUN_ATTEMPT}"
echo "repository=${REPOSITORY}"
echo "event_name=${EVENT_NAME}"
echo "ref=${REF}"
date -u '+timestamp_utc=%Y-%m-%dT%H:%M:%SZ'
uname -a
- name: Basic tool availability
run: |
set -e
for tool in bash sh date; do
command -v "${tool}" >/dev/null 2>&1
echo "tool_ok=${tool}"
done
- name: Docker and registry preflight
env:
LABEL: ${{ matrix.label }}
run: |
set -e
command -v docker >/dev/null 2>&1
echo "docker_ok=true"
if docker info 2>/dev/null | grep -qi "kankali.darkhelm.lan:3001"; then
echo "registry_config_ok=true"
else
echo "registry_config_ok=false"
echo "Missing Docker insecure registry config for kankali.darkhelm.lan:3001"
exit 1
fi
if docker pull kankali.darkhelm.lan:3001/darkhelm.org/act-ubuntu:act-latest >/tmp/canary-mirror-pull.log 2>&1; then
echo "mirror_pull_ok=true"
else
if grep -qi "http response to https client" /tmp/canary-mirror-pull.log; then
echo "mirror_pull_ok=false"
echo "Detected HTTPS/HTTP registry mismatch for kankali.darkhelm.lan:3001"
fi
tail -n 20 /tmp/canary-mirror-pull.log || true
exit 1
fi
- *failure_diagnostics_step
canary-targeted-burst:
name: Canary Burst (${{ github.event.inputs.target_label || 'ubuntu-act' }} #${{ matrix.probe }})
if: github.event_name == 'workflow_dispatch' || github.event_name == 'schedule'
runs-on: ${{ github.event.inputs.target_label || 'ubuntu-act' }}
timeout-minutes: 6
strategy:
fail-fast: false
max-parallel: 4
matrix:
probe: ["01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20"]
steps:
- name: Skip probes above requested count
env:
PROBE: ${{ matrix.probe }}
PROBE_COUNT_INPUT: ${{ github.event.inputs.probe_count }}
EVENT_NAME: ${{ github.event_name }}
run: |
if [ "${EVENT_NAME}" = "schedule" ]; then
# Keep scheduled load light while still sampling runner stability.
REQUESTED_COUNT="6"
else
REQUESTED_COUNT="${PROBE_COUNT_INPUT:-12}"
fi
if [ "${REQUESTED_COUNT}" -lt 1 ] || [ "${REQUESTED_COUNT}" -gt 20 ]; then
echo "❌ probe_count must be between 1 and 20; received ${REQUESTED_COUNT}"
exit 1
fi
if [ "${PROBE#0}" -gt "${REQUESTED_COUNT}" ]; then
echo "Skipping probe ${PROBE}; requested_count=${REQUESTED_COUNT}"
exit 0
fi
- name: Runner fingerprint audit
env:
TARGET_LABEL: ${{ github.event.inputs.target_label || 'ubuntu-act' }}
PROBE: ${{ matrix.probe }}
RUN_ID: ${{ github.run_id }}
RUN_ATTEMPT: ${{ github.run_attempt }}
JOB: ${{ github.job }}
run: |
TRACE_ID="canary-${RUN_ID}-${RUN_ATTEMPT}-${TARGET_LABEL}-${PROBE}"
echo "=== Runner Canary Burst ==="
echo "trace_id=${TRACE_ID}"
echo "job=${JOB}"
echo "target_label=${TARGET_LABEL}"
echo "probe=${PROBE}"
echo "runner_name=${RUNNER_NAME:-}"
echo "runner_name_hint=${GITEA_RUNNER_NAME:-${ACT_RUNNER_NAME:-${RUNNER_NAME:-unknown}}}"
echo "runner_hostname_env=${HOSTNAME:-unknown}"
echo "runner_uname_n=$(uname -n 2>/dev/null || echo unknown)"
echo "runner_etc_hostname=$(cat /etc/hostname 2>/dev/null || echo unknown)"
echo "runner_machine_id=$(cat /etc/machine-id 2>/dev/null || echo unknown)"
echo "runner_kernel=$(uname -r 2>/dev/null || echo unknown)"
echo "runner_uptime_seconds=$(cut -d' ' -f1 /proc/uptime 2>/dev/null || echo unknown)"
echo "runner_ipv4=$(hostname -I 2>/dev/null | awk '{print $1}' || echo unknown)"
echo "timestamp_utc=$(date -u +%Y-%m-%dT%H:%M:%SZ)"
- name: Docker quick health
run: |
set -e
if command -v docker >/dev/null 2>&1; then
docker version --format '{{.Server.Version}}' || docker version || true
docker info --format 'docker_driver={{.Driver}} docker_cgroup={{.CgroupDriver}}' || true
docker ps --format 'container={{.Names}} status={{.Status}}' | head -n 20 || true
else
echo "docker_unavailable=true"
fi
- name: Registry smoke pull
run: |
set -e
if command -v docker >/dev/null 2>&1; then
if docker pull kankali.darkhelm.lan:3001/darkhelm.org/act-ubuntu:act-latest >/tmp/canary-burst-pull.log 2>&1; then
echo "mirror_pull_ok=true"
else
echo "mirror_pull_ok=false"
tail -n 20 /tmp/canary-burst-pull.log || true
exit 1
fi
fi
- *failure_diagnostics_step