plex-playlist/docs/DEVELOPMENT.md

Development Environment Setup and Workflow

This document outlines how to set up your development environment and work with the plex-playlist project.

Table of Contents

1. Quick Start
2. Development Environment Setup
3. Poe the Poet Task Runner
4. Git Workflow
5. Pre-commit Hooks
6. Manual Tool Usage
7. CI/CD Pipeline
8. Branch Protection and Merge Requirements

Related Documentation

• Poe Task Reference - Complete guide to unified development tasks
• CI/CD Multi-Stage Build Architecture - Technical details of the optimized build system
• CI/CD Troubleshooting - Current CI lane failures and remediation paths
• Secure Docker CI/CD - Security considerations and practices
• Deployable Runtime Contract - Canonical backend/frontend runtime artifact contract and exclusion rules
• ADR003: Deployable Runtime Image Contract Boundaries - Decision record for deployable runtime boundaries
• ADR004: Registry Image Resolution and Auth Resilience Policy - Cross-workflow reliability policy for registry/auth/image fallback behavior

Deployable Runtime Artifacts

When changing deployment behavior or image composition, treat DEPLOYABLE_RUNTIME_CONTRACT.md as the source of truth for:

• Runtime entrypoint and exposed ports.
• Health and startup expectations.
• Runtime environment contract.
• Disallowed non-runtime tooling classes in final deployable images.

Scope boundary:

• Contract definition lives in DEPLOYABLE_RUNTIME_CONTRACT.md.
• CI enforcement now includes Dockerfile boundary checks and deployable image purity checks in .gitea/workflows/cicd.yaml.
• Broader workflow redesign and deployment wiring remain out of scope for this repository's runtime contract document and belong to follow-up work under epic #66.

Runtime vs Validation Enforcement

Deployable runtime artifacts and validation environments are intentionally separated.

• Validation tools (lint/typecheck/test/browser tooling) belong to CICD image paths (Dockerfile.cicd-base, Dockerfile.cicd) and CI validation workflows.
• Deployable runtime paths (Dockerfile.backend, Dockerfile.frontend production target) must remain independent of CI-only tooling layers.

Automated checks:

• scripts/check-dockerfile-boundaries.sh validates deployable Dockerfiles do not depend on CICD images or install disallowed CI-only tools.
• scripts/verify-deployable-image-purity.sh inspects built deployable images and fails if CI/development binaries or package metadata artifacts are present.

These checks run in .gitea/workflows/cicd.yaml before publishing the complete CICD image.

Quick Start

# Clone the repository
git clone ssh://git@dogar.darkhelm.org:2222/DarkHelm.org/plex-playlist.git
cd plex-playlist

# Complete setup with Poe the Poet (recommended)
cd backend && uv sync --dev && cd ..
poe setup

# OR manual setup
docker compose -f compose.dev.yml up -d
pip install pre-commit && pre-commit install

# Create a feature branch and start developing
git checkout -b feature/your-feature-name

# Use Poe for all development tasks
poe format lint type-check test-unit

Development Environment Setup

Local Development with Docker Compose

The project uses compose.dev.yml for local development, which provides:

• Backend: FastAPI application with hot reload
• Frontend: Vite development server with hot module replacement
• Database: PostgreSQL with persistent data
• Development Tools: Pre-configured with all necessary dependencies

# Start all services
docker compose -f compose.dev.yml up -d

# View logs
docker compose -f compose.dev.yml logs -f

# Stop services
docker compose -f compose.dev.yml down

# Rebuild after dependency changes
docker compose -f compose.dev.yml up -d --build

Service Access

• Frontend: http://localhost:3000
• Backend API: http://localhost:8000
• API Documentation: http://localhost:8000/docs
• Database: localhost:5432 (user: plex, password: see secrets/postgres_password)

Poe the Poet Task Runner

This project uses Poe the Poet as a unified task runner to simplify and standardize development workflows. Instead of remembering different commands for backend and frontend, you can use consistent poe commands from anywhere in the project.

Quick Start with Poe

# Complete development setup (new developers)
poe setup

# Start development environment
poe dev

# Run all quality checks (format, lint, type-check)
poe ci-quick

# Run all tests
poe test-all

# See all available tasks
poe --help

Key Benefits

• Unified Interface: Same commands work for backend (Python) and frontend (TypeScript)
• Parallel Execution: Tasks run in parallel for better performance
• Smart Workflows: Conditional and chained task execution
• Developer Friendly: Interactive task selection and helpful descriptions
• CI Integration: Local simulation of CI/CD pipeline

Common Workflows

# Daily development workflow
poe format lint type-check test-unit

# Pre-commit workflow (faster than full CI)
poe ci-quick

# Complete quality gate (like CI pipeline)
poe quality-gate

# Build and test Docker images locally
poe build-cicd

For the complete list of tasks and detailed usage, see Poe Task Reference.

Git Workflow

Branch Strategy

The project follows a feature branch workflow with strict main branch protection:

1. Main Branch: Always deployable, protected from direct commits
2. Feature Branches: All work done in feature branches (feature/feature-name)
3. Pull Requests: Required for all changes to main

Creating a Feature Branch

# Ensure you're on main and up to date
git checkout main
git pull origin main

# Create and switch to feature branch
git checkout -b feature/your-feature-name

# Push branch to remote
git push -u origin feature/your-feature-name

Making Changes

# Make your changes
# ... edit files ...

# Stage and commit
git add .
git commit -m "feat: add new playlist functionality"

# Push changes
git push

Creating a Pull Request

1. Push your feature branch to the remote repository
2. Navigate to the Gitea web interface
3. Create a Pull Request from your feature branch to main
4. Ensure all CI checks pass (100% green required)
5. Request review from team members
6. Merge only after approval and passing CI

Pre-commit Hooks

Why Pre-commit is Recommended

Pre-commit hooks provide immediate feedback and prevent CI failures by running the same validation locally that runs in CI. Benefits:

• Fast Feedback: Catch issues before pushing
• Consistent Quality: Same tools and configurations as CI
• Time Saving: Avoid CI failure cycles
• Team Standards: Automatic code formatting and linting

Installation and Setup

# Install pre-commit (if not already installed)
pip install pre-commit

# Install hooks for this repository
pre-commit install

# Optionally, run on all files initially
pre-commit run --all-files

How It Works

Pre-commit automatically runs on every git commit and will:

• Format code (Prettier, Ruff)
• Check syntax (ESLint, Pyright)
• Validate files (YAML, TOML, trailing whitespace)
• Run quick tests and linting

If any hook fails, the commit is blocked until issues are fixed.

Pre-commit is Optional

While strongly recommended, pre-commit is not required because:

• CI Validation: All the same checks run in CI
• Developer Choice: Some prefer manual tool usage
• Learning: Developers can run tools individually to understand them

However, without pre-commit, you'll need to manually run tools and may face CI failures.

Manual Tool Usage

If you prefer not to use pre-commit, here's how to run each tool manually:

Backend Tools (Python)

Navigate to the backend/ directory for all backend commands.

Backend Code Formatting

# Format code with Ruff
uv run ruff format .

# Fix auto-fixable linting issues
uv run ruff check . --fix

Backend Type Checking

# Run Pyright type checker
uv run pyright

Backend Testing

# Run unit tests
uv run pytest

# Run tests with coverage
uv run pytest --cov=src --cov-report=html

# Run integration tests
uv run pytest tests/integration/

# Run doctests
uv run xdoctest src/

Backend Documentation

# Check docstring style
uv run pydoclint --config=pyproject.toml src/

Frontend Tools (TypeScript/Vue)

Navigate to the frontend/ directory for all frontend commands.

Frontend Code Formatting

# Format code with Prettier
yarn format

# Check formatting
yarn format:check

Linting

# Run ESLint
yarn lint

# Fix auto-fixable ESLint issues
yarn lint:fix

# Check TypeScript documentation
yarn lint:tsdoc

Frontend Type Checking

# Run Vue TypeScript compiler
yarn type-check

Frontend Testing

# Run unit tests
yarn test

# Run unit tests with coverage
yarn test:coverage

# Run E2E tests (Playwright)
yarn test:e2e

# Run E2E tests in headless mode (CI-like, default)
yarn playwright test

# Run E2E tests with specific reporter
yarn playwright test --reporter=list

# Run E2E tests for specific browser
yarn playwright test --project=chromium

Project-wide Tools

YAML/TOML Validation

# Check YAML files
pre-commit run check-yaml --all-files

# Check TOML files
pre-commit run check-toml --all-files

File Quality

# Fix trailing whitespace
pre-commit run trailing-whitespace --all-files

# Fix end-of-file issues
pre-commit run end-of-file-fixer --all-files

CI/CD Pipeline

Pipeline Overview

The canonical CI workflow is .gitea/workflows/cicd.yaml.

Current triggers:

• Push on main and develop
• Pull requests targeting main and develop
• Manual workflow_dispatch

Current dispatch inputs:

• head_sha: commit SHA to process
• force_rebuild_base: force base image publication

Current Job Topology

The pipeline is intentionally staged so expensive image jobs run only after source checks and required gates pass:

1. Publish base image lane: Build and Publish CICD Base Image computes base hash, checks registry, and conditionally builds/pushes immutable + latest tags.
2. CICD image lane: Build and Push CICD Image consumes base hash and publishes the shared CICD image.
3. Source lanes: Source Checks, Frontend Dependency Audit, Backend Dependency Audit, then CICD Tests Complete gate.
4. Runtime image lanes: backend base/frontend base, backend main/frontend main, integration tester, e2e tester, then Production Images Complete gate.
5. Runtime validation lanes: Runtime Black-Box Integration Tests and End-to-End Tests.
6. Postmortem lanes: targeted postmortem jobs run when key lanes fail to capture diagnostics even when primary jobs fail early.

Reliability and Traceability Behavior

Current workflow behavior includes:

• registry auth realm host pinning from WWW-Authenticate challenge when registry tokens are issued from a different host
• bounded retry logic for docker login/pull/push operations in image lanes
• digest/tag contract checks for deployable image references before runtime black-box tests
• context hydration for image-build lanes by copying /workspace from the published CICD image
• runner split between ubuntu-act and ubuntu-act-8gb based on lane resource requirements

For details of base/complete image build strategy, see CI/CD Multi-Stage Build Architecture. For incident handling, see CI/CD Troubleshooting.

Operator Quick Reference

Use workflow dispatch when you need deterministic reruns on a specific commit:

# Example: rerun CI against an explicit commit
# input head_sha=<commit>

# Example: force base image republish
# input force_rebuild_base=true

Recommended rerun order during flaky infrastructure incidents:

1. Build and Publish CICD Base Image
2. Build and Push CICD Image
3. failing runtime image lane (Build Integration Tester Image, Build Frontend Main Image, etc.)
4. downstream integration/e2e lanes

Local CI/CD Testing

Build and test CI/CD images locally:

# Build both base and complete images
./scripts/build-cicd-local.sh

# Build only base image
./scripts/build-cicd-local.sh --base-only

# Build only complete image (requires existing base)
./scripts/build-cicd-local.sh --complete-only

# Force rebuild with no cache
./scripts/build-cicd-local.sh --force --no-cache

# Test with custom SSH key
./scripts/build-cicd-local.sh --ssh-key ~/.ssh/custom_key

CI/CD Design Principles

• Multi-Stage Optimization: Separate stable dependencies from project code
• Intelligent Caching: Base image cached when unchanged (hash-based detection)
• Single Source of Truth: All CI jobs use the same pre-commit hooks as local development
• Parallel Execution: Maximum efficiency with concurrent job execution
• Fast Feedback: Jobs fail fast on first error
• Memory Efficiency: Optimized for 4GB Raspberry Pi workers
• Comprehensive Coverage: Every aspect of code quality is validated

Viewing CI Results

1. Navigate to your pull request in Gitea
2. Check the "Checks" tab for detailed results
3. Click on individual job names to see logs
4. All jobs must pass (100% green) before merging

Branch Protection and Merge Requirements

Main Branch Protection

The main branch is protected with the following requirements:

1. No Direct Pushes: All changes must come through pull requests
2. CI Must Pass: All CI/CD jobs must be 100% green
3. Review Required: At least one team member approval needed
4. Up-to-date Branch: Feature branch must be current with main

Merge Process

1. Create pull request from feature branch
2. Wait for all CI checks to complete successfully
3. Address any CI failures by pushing fixes to the feature branch
4. Request and receive code review approval
5. Ensure branch is up-to-date with main
6. Merge pull request (only available when all requirements met)

If CI Fails

When CI fails:

1. Check Logs: Review the failed job logs in Gitea
2. Fix Locally: Make corrections in your feature branch
3. Test Locally: Run the same tools locally or use pre-commit
4. Push Fix: Commit and push the fix
5. Wait for CI: New CI run will start automatically

Common CI Failure Resolutions

# Format code issues
pre-commit run --all-files

# Type errors (backend)
cd backend && uv run pyright

# Type errors (frontend)
cd frontend && yarn type-check

# Test failures (backend)
cd backend && uv run pytest

# Test failures (frontend)
cd frontend && yarn test:coverage

# E2E test failures (frontend)
cd frontend && yarn test:e2e

Best Practices

Development Workflow

1. Always use feature branches - never commit directly to main
2. Keep branches small - easier to review and merge
3. Write descriptive commit messages - helps with project history
4. Run tests locally - catch issues before pushing
5. Use pre-commit - saves time and ensures quality

Code Quality

1. Follow existing patterns - maintain consistency
2. Write tests for new features - maintain coverage thresholds
3. Add docstrings - especially for public APIs
4. Type annotations - required for all Python functions
5. Meaningful variable names - self-documenting code

Performance Tips

1. Use Docker efficiently - leverage layer caching
2. Run specific tests - don't always run full suite during development
3. Parallel development - multiple developers can work simultaneously
4. Resource monitoring - watch Docker memory usage on constrained systems

---

For questions or issues with the development environment, please create an issue in the project repository or contact the development team.