Testing Guide

Core Principle: Local-First Testing

All tests MUST run locally without external dependencies.

• Unit tests: MUST NOT use network, Docker, or databases
• Integration tests: MUST use testcontainers or local services
• Tests MUST NOT require cloud accounts or remote APIs
• External services MUST be mocked; test data MUST use fixtures
• CI MUST reproduce local test environment exactly

Unit Tests

Test individual functions/methods in isolation².

• MUST be fast: Milliseconds per test
• MUST be isolated: No external dependencies, no I/O
• MUST be repeatable: Same result every time
• MUST be self-validating: Pass or fail, no manual checking

Test Markers/Tags

Use markers to categorize and selectively run tests:

# Python
pytest -m unit
pytest -m "not slow"

# Go
go test -run TestUnit
go test -short

# JavaScript (Vitest): filter by test name
vitest run -t "unit"

# JavaScript (Vitest): filter by named project
vitest run --project unit

Vitest has no marker system and rejects --grep, so a category filter needs a naming or configuration convention. You MUST choose one of these:

• Encode the category in the test name and filter with -t (--testNamePattern), which matches the full name as a regular expression.
• Declare named projects in vitest.config.ts and filter with --project, which binds the category to a file glob and leaves the test name free to describe behavior.

// vitest.config.ts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    projects: [
      { test: { name: 'unit', include: ['tests/unit/**/*.test.ts'] } },
      { test: { name: 'integration', include: ['tests/integration/**/*.test.ts'] } },
    ],
  },
});

CI Strategy

Every job MUST select a runner, check out the repository, install a pinned runtime, and install locked dependencies before it runs a test command.

name: Tests

on:
  push:
    branches: [main]
  pull_request:

jobs:
  # Fast feedback first
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
        with:
          python-version: "3.14"
      - run: uv sync --locked
      - run: uv run ruff check .

  unit:
    needs: lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
        with:
          python-version: "3.14"
      - run: uv sync --locked
      - run: uv run pytest -m unit -n auto

  integration:
    needs: unit
    runs-on: ubuntu-latest
    services:
      postgres:
        # postgres:16; registry manifest digest.
        image: postgres@sha256:ca0bd484cb98bf4b24eb1010e73fb3fcbd6714d240fbc1a10eea5b7dbecb641d
        env:
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: test
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
        with:
          python-version: "3.14"
      - run: uv sync --locked
      - run: uv run pytest -m integration
        env:
          DATABASE_URL: postgres://postgres:postgres@localhost:5432/test

  e2e:
    needs: integration
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
        with:
          python-version: "3.14"
      - run: uv sync --locked
      - run: uv run playwright install --with-deps chromium
      - run: uv run pytest tests/e2e/ --tracing retain-on-failure --output test-results
      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        if: failure()
        with:
          name: e2e-traces
          path: test-results/

  # Slow tests on main only
  performance:
    if: github.ref == 'refs/heads/main'
    needs: e2e
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
        with:
          python-version: "3.14"
      - run: uv sync --locked
      - run: uv run pytest tests/performance/

The postgres service MUST set POSTGRES_PASSWORD: the official image refuses to initialize without it, so the job has no database to reach. The service MUST declare a health check, which is what makes the runner wait until PostgreSQL accepts connections instead of merely until the container starts. Because these jobs run directly on the runner rather than in a job container, the service MUST map 5432:5432 and clients MUST connect via localhost. The e2e job uploads Playwright traces on failure so a red run is diagnosable without a rerun.

Flaky Test Policy

Flaky tests³³ undermine confidence. Handle them aggressively:

1. MUST quarantine immediately: Move to separate suite
2. MUST fix within 48 hours or delete
3. MUST NOT ignore: A flaky test is worse than no test
4. SHOULD track patterns: Network? Timing? Order-dependent?

Test Data Management

• Fixtures³⁴: Static data files for deterministic tests
• Factories³⁵: Generate test data programmatically
• Builders³⁶: Fluent API for complex objects
• Fakers³⁷: Random but realistic data

You SHOULD prefer factories over fixtures for flexibility. You SHOULD use fixtures for complex scenarios that need exact reproduction.