Public Access
feature/pp-58-runtime-image-contract (#68)
## Summary This PR tightens repository quality enforcement around markdown and documentation. It adds `markdownlint` to the `cicd-checks` workflow, expands pre-commit coverage so markdown files are checked repo-wide, and cleans up the PP-58 documentation set to keep it aligned with the new policy. ## What changed - Added a `Markdownlint Check` entry to `.gitea/workflows/cicd-checks.yaml` - Added `markdownlint` to pre-commit and widened prettier coverage to include markdown files across the repo - Updated `README.md` to satisfy markdownlint line-length rules - Normalized the PP-58 documentation set: - `docs/DEPLOYABLE_RUNTIME_CONTRACT.md` - `docs/adr/ADR003-deployable_runtime_image_contract.md` - `docs/DEVELOPMENT.md` - `docs/CICD_MULTI_STAGE_BUILD.md` - `docs/CICD_TROUBLESHOOTING_GUIDE.md` - `docs/SECURE_DOCKER_CICD.md` ## Validation - `pre-commit run markdownlint --files README.md docs/DEPLOYABLE_RUNTIME_CONTRACT.md` - `pre-commit run prettier --files README.md docs/DEPLOYABLE_RUNTIME_CONTRACT.md` - Workflow YAML validation returned no errors ## Notes This change does not alter application runtime behavior. It only strengthens CI and documentation quality enforcement. Co-authored-by: copilotcoder <copilotcoder@darkhelm.org> Reviewed-on: #68
This commit was merged in pull request #68.
This commit is contained in:
@@ -30,6 +30,7 @@ RUN git clone full_repo && merge_preserving_deps # ✅ Source changes don't bus
|
||||
**Technical Challenges & Solutions**:
|
||||
|
||||
1. **Local Package Build Error**: `OSError: Readme file does not exist: ../README.md`
|
||||
|
||||
```dockerfile
|
||||
# Fix: Create minimal structure for package build
|
||||
RUN mkdir -p src/backend && \
|
||||
@@ -39,6 +40,7 @@ RUN git clone full_repo && merge_preserving_deps # ✅ Source changes don't bus
|
||||
```
|
||||
|
||||
2. **Dependency Preservation**: Need to preserve installed packages when copying source
|
||||
|
||||
```dockerfile
|
||||
# Fix: Backup/restore strategy
|
||||
RUN if [ -d "/workspace/backend/.venv" ]; then mv /workspace/backend/.venv /tmp/venv_backup; fi && \
|
||||
@@ -47,13 +49,15 @@ RUN git clone full_repo && merge_preserving_deps # ✅ Source changes don't bus
|
||||
```
|
||||
|
||||
3. **No rsync Available**: Base image doesn't include rsync for selective copying
|
||||
```dockerfile
|
||||
# Fix: Use standard cp with backup strategy instead of rsync
|
||||
# rsync -av --exclude='node_modules' /tmp/fullrepo/ /workspace/ # ❌ Not available
|
||||
# Standard cp with manual exclusions # ✅ Works everywhere
|
||||
```
|
||||
|
||||
```dockerfile
|
||||
# Fix: Use standard cp with backup strategy instead of rsync
|
||||
# rsync -av --exclude='node_modules' /tmp/fullrepo/ /workspace/ # ❌ Not available
|
||||
# Standard cp with manual exclusions # ✅ Works everywhere
|
||||
```
|
||||
|
||||
**Metrics**:
|
||||
|
||||
- Dependency cache hit rate: ~95% (only miss when pyproject.toml/package.json change)
|
||||
- Average build time reduction: 12-17 minutes saved per build
|
||||
- Resource efficiency: Better CPU/memory utilization on Raspberry Pi workers
|
||||
@@ -65,6 +69,7 @@ RUN git clone full_repo && merge_preserving_deps # ✅ Source changes don't bus
|
||||
**Problem**: Firefox and WebKit browsers failing consistently in Docker CI environment.
|
||||
|
||||
**Root Cause Analysis**:
|
||||
|
||||
- **Firefox**: Sandbox restrictions in Docker containers, requires `--no-sandbox` and security compromises
|
||||
- **WebKit**: Content loading timeout issues, navigation reliability problems in headless mode
|
||||
- **Docker Environment**: Limited resources (RPi 4GB) exacerbate browser compatibility issues
|
||||
@@ -77,26 +82,28 @@ const projects = process.env.CI
|
||||
? [
|
||||
// CI: Only Chromium (most reliable in Docker)
|
||||
{
|
||||
name: 'chromium',
|
||||
use: { ...devices['Desktop Chrome'] },
|
||||
}
|
||||
name: "chromium",
|
||||
use: { ...devices["Desktop Chrome"] },
|
||||
},
|
||||
]
|
||||
: [
|
||||
// Local: Full browser coverage
|
||||
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
|
||||
{ name: 'firefox', use: { ...devices['Desktop Firefox'] } },
|
||||
{ name: 'webkit', use: { ...devices['Desktop Safari'] } },
|
||||
{ name: "chromium", use: { ...devices["Desktop Chrome"] } },
|
||||
{ name: "firefox", use: { ...devices["Desktop Firefox"] } },
|
||||
{ name: "webkit", use: { ...devices["Desktop Safari"] } },
|
||||
];
|
||||
```
|
||||
|
||||
**Rationale**:
|
||||
|
||||
- Chromium engine powers 95%+ of web browsers (Chrome, Edge, Opera, Brave)
|
||||
- Excellent Docker compatibility and resource efficiency
|
||||
- Core functionality testing coverage maintained
|
||||
- Full browser testing available for local development
|
||||
|
||||
**Error Examples Resolved**:
|
||||
```
|
||||
|
||||
```text
|
||||
Firefox: error: unknown option '--headed=false'
|
||||
WebKit: Test timeout 30000ms exceeded... waiting for navigation
|
||||
Firefox: browserType.launch: Executable doesn't exist
|
||||
@@ -113,8 +120,9 @@ Firefox: browserType.launch: Executable doesn't exist
|
||||
**Solution**: Multi-level retry logic with exponential backoff:
|
||||
|
||||
#### Docker Registry Operations
|
||||
|
||||
```yaml
|
||||
# .gitea/workflows/cicd.yml
|
||||
# .gitea/workflows/cicd-checks.yaml
|
||||
- name: Login to Container Registry (with retry)
|
||||
run: |
|
||||
for attempt in {1..5}; do
|
||||
@@ -135,6 +143,7 @@ Firefox: browserType.launch: Executable doesn't exist
|
||||
```
|
||||
|
||||
#### Playwright Browser Installation
|
||||
|
||||
```yaml
|
||||
- name: Install Playwright Browsers (with retry)
|
||||
run: |
|
||||
@@ -151,14 +160,19 @@ Firefox: browserType.launch: Executable doesn't exist
|
||||
```
|
||||
|
||||
#### E2E Test Navigation Resilience
|
||||
|
||||
```typescript
|
||||
// frontend/tests/e2e/app.spec.ts
|
||||
async function navigateWithRetry(page: Page, url: string, maxRetries: number = 3): Promise<void> {
|
||||
async function navigateWithRetry(
|
||||
page: Page,
|
||||
url: string,
|
||||
maxRetries: number = 3,
|
||||
): Promise<void> {
|
||||
for (let attempt = 1; attempt <= maxRetries; attempt++) {
|
||||
try {
|
||||
await page.goto(url, {
|
||||
waitUntil: 'networkidle',
|
||||
timeout: 90000 // Extended timeout
|
||||
waitUntil: "networkidle",
|
||||
timeout: 90000, // Extended timeout
|
||||
});
|
||||
return;
|
||||
} catch (error) {
|
||||
@@ -171,6 +185,7 @@ async function navigateWithRetry(page: Page, url: string, maxRetries: number = 3
|
||||
```
|
||||
|
||||
**Configuration Enhancements**:
|
||||
|
||||
```typescript
|
||||
// playwright.config.ts - CI optimizations
|
||||
use: {
|
||||
@@ -182,6 +197,7 @@ use: {
|
||||
```
|
||||
|
||||
**Results**:
|
||||
|
||||
- CI success rate: 40% → 95%
|
||||
- Average retry overhead: +30 seconds per build
|
||||
- Network timeout elimination: 100% of Docker operations now succeed
|
||||
@@ -193,7 +209,8 @@ use: {
|
||||
**Problem**: Production base image missing pre-installed Python dev tools optimization.
|
||||
|
||||
**Symptom**:
|
||||
```
|
||||
|
||||
```text
|
||||
⚠ Pre-installed Python dev tools not found - fresh installation
|
||||
Base image may need rebuild for optimal caching
|
||||
```
|
||||
@@ -201,6 +218,7 @@ Base image may need rebuild for optimal caching
|
||||
**Impact**: +15-20 seconds build time (acceptable degradation vs failure)
|
||||
|
||||
**Solution**: Graceful fallback detection:
|
||||
|
||||
```dockerfile
|
||||
# Dockerfile.cicd - Resilient optimization detection
|
||||
RUN echo "=== Base Image Optimization Status ===" && \
|
||||
@@ -220,7 +238,8 @@ RUN echo "=== Base Image Optimization Status ===" && \
|
||||
### Missing Immutable Base Image
|
||||
|
||||
**Symptom**:
|
||||
```
|
||||
|
||||
```text
|
||||
❌ Required immutable base image is not available: kankali.darkhelm.lan:3001/darkhelm.org/plex-playlist-cicd-base:<hash>
|
||||
Publish the base image via the CICD Base Image workflow before rerunning main CI.
|
||||
```
|
||||
@@ -229,11 +248,13 @@ Publish the base image via the CICD Base Image workflow before rerunning main CI
|
||||
dedicated base-image workflow has not published that immutable tag yet.
|
||||
|
||||
**Checks**:
|
||||
|
||||
1. Confirm whether `Dockerfile.cicd-base`, `.dockerignore`, or `scripts/compute-cicd-base-hash.sh` changed in the branch.
|
||||
2. Check the `CICD Base Image` workflow for the same commit or PR.
|
||||
3. Verify the registry contains `plex-playlist-cicd-base:<hash>`.
|
||||
|
||||
**Resolution**:
|
||||
|
||||
1. If the base workflow is still running, rerun main CI after it completes.
|
||||
2. If the base workflow did not trigger, run it manually with `force_rebuild=false`.
|
||||
3. If the tag should be republished despite already existing, run it manually with `force_rebuild=true`.
|
||||
@@ -245,47 +266,60 @@ publish-once/consume-many design.
|
||||
### Docker Build Failures
|
||||
|
||||
#### 1. rsync Command Not Found
|
||||
```
|
||||
|
||||
```text
|
||||
/bin/bash: line 1: rsync: command not found
|
||||
```
|
||||
|
||||
**Fix**: Replace with standard cp commands and backup strategy (implemented)
|
||||
|
||||
#### 2. README.md Not Found During uv sync
|
||||
```
|
||||
|
||||
```text
|
||||
OSError: Readme file does not exist: ../README.md
|
||||
```
|
||||
|
||||
**Fix**: Create dummy README.md during dependency installation phase (implemented)
|
||||
|
||||
#### 3. Dependency Cache Invalidation
|
||||
|
||||
**Symptom**: Dependencies rebuilding on every commit
|
||||
**Fix**: Verify dependency-first build pattern is correctly implemented
|
||||
|
||||
### E2E Test Failures
|
||||
|
||||
#### 1. Browser Not Found
|
||||
```
|
||||
|
||||
```text
|
||||
Executable doesn't exist at /root/.cache/ms-playwright/chromium-*/
|
||||
```
|
||||
|
||||
**Fix**: Ensure `yarn playwright install --with-deps` runs before tests
|
||||
|
||||
#### 2. Navigation Timeouts
|
||||
```
|
||||
|
||||
```text
|
||||
Test timeout 30000ms exceeded
|
||||
```
|
||||
|
||||
**Fix**: Use `navigateWithRetry` helper with extended timeouts
|
||||
|
||||
#### 3. Multi-browser Failures in CI
|
||||
|
||||
**Fix**: Use Chromium-only configuration for CI environments
|
||||
|
||||
### Network-Related Issues
|
||||
|
||||
#### 1. Docker Registry Timeouts
|
||||
|
||||
**Fix**: Retry logic with exponential backoff (5 attempts, 15s intervals)
|
||||
|
||||
#### 2. Package Download Failures
|
||||
|
||||
**Fix**: Increase timeouts and add retry mechanisms
|
||||
|
||||
#### 3. SSL Certificate Issues
|
||||
|
||||
**Fix**: Set `ignoreHTTPSErrors: true` and `NODE_TLS_REJECT_UNAUTHORIZED=0`
|
||||
|
||||
## Performance Monitoring
|
||||
@@ -320,12 +354,13 @@ Test timeout 30000ms exceeded
|
||||
**🎉 MILESTONE ACHIEVED**: First fully successful CI/CD workflow completion with all optimizations working together.
|
||||
|
||||
**Final Performance Metrics**:
|
||||
|
||||
- **Total Pipeline Time**: ~3-5 minutes (down from 15-25 minutes)
|
||||
- **Success Rate**: 100% (all test phases passing)
|
||||
- **Build Optimization**: 85% time reduction achieved
|
||||
- **E2E Test Reliability**: 100% (simplified Docker approach)
|
||||
|
||||
### **Key Issues Resolved in Final Sprint**:
|
||||
### **Key Issues Resolved in Final Sprint**
|
||||
|
||||
1. **✅ README.md Dependency Fix**: Dummy file creation for dependency-only builds
|
||||
2. **✅ Rsync Replacement**: Standard cp commands with backup/restore strategy
|
||||
@@ -333,7 +368,8 @@ Test timeout 30000ms exceeded
|
||||
4. **✅ E2E Test Simplification**: Removed unnecessary complex retry logic
|
||||
5. **✅ Memory Management**: Proper swap configuration and Node.js memory limits
|
||||
|
||||
### **Validated Working Components**:
|
||||
### **Validated Working Components**
|
||||
|
||||
- **Multi-stage Docker builds** with optimal layer caching
|
||||
- **Dependency-first build pattern** preventing cache invalidation
|
||||
- **Network-resilient Playwright setup** with Chromium-only CI testing
|
||||
@@ -341,8 +377,10 @@ Test timeout 30000ms exceeded
|
||||
- **SSH-based secure repository access** with proper key management
|
||||
- **Comprehensive test coverage** (linting, unit tests, integration, E2E)
|
||||
|
||||
### **Architecture Stability**:
|
||||
### **Architecture Stability**
|
||||
|
||||
All components now work cohesively:
|
||||
|
||||
- Base image caching (cicd-base) ↔️ Complete image building (cicd)
|
||||
- Python dependency management (uv) ↔️ Backend source integration
|
||||
- Frontend dependency management (Yarn PnP) ↔️ Source code preservation
|
||||
|
||||
Reference in New Issue
Block a user