UI End-to-End Tests (Playwright)

tests/ui/ is the framework-bootstrap home for browser-driven regression guards against the dashboard's rendered DOM (mission playwright-ui-e2e-bootstrap-01KWX72W, issue #1008). It exists because backend and API tests cannot catch a bug where the server responds correctly but the browser renders it wrong — exactly what shipped in PR #970: 338 backend tests and every architectural test passed while the dashboard's WP-card click-through modal silently dropped the agent identity, because no test layer ever exercised the actual click-then-render flow.

CLAUDE.md's "Never claim the frontend works without Playwright proof" rule is enforced by this suite, not aspirational — the runnable proof is tests/ui/test_dashboard_wp_modal.py.

Running the suite locally

# One-time: install the headless Chromium binary (never committed; cached
# under ~/.cache/ms-playwright).
uv run playwright install chromium

# Run the suite headless, matching CI exactly.
PWHEADLESS=1 uv run pytest tests/ui/ -q

No real ~/.spec-kitty, no network access, and no git repository are required — every test boots a hermetic, synthetic project root on an ephemeral port (see below).

How the hermetic fixture works

tests/ui/conftest.py provides two fixtures every test in this suite builds on:

  • synthetic_project_root — materializes a temp directory containing one synthetic Mission and work package (kitty-specs/<slug>/tasks/WP01.md + a seeded status.events.jsonl) using the exact frontmatter shape dashboard/scanner.py::_process_wp_file decomposes: a dict agent: {tool, model} plus top-level agent_profile/role keys — not the colon-string agent: 'tool:model:profile:role' form, which the scanner does not decompose.
  • dashboard — boots the real dashboard app in-thread against that synthetic root via start_dashboard(project_dir, port, background_process=False) (src/specify_cli/dashboard/server.py:110). This is the hermetic seam: it runs on a daemon thread bound to an ephemeral find_free_port() port and dies with the pytest process — no PID file, no sibling processes killed. It returns a dict with base_url plus the exact identity values the fixture wrote, so a test can assert against known values instead of guessing.

Deliberately not used: cli/commands/dashboard.py's ensure_dashboard_running singleton. That path detaches a PID'd child process and kills siblings on the shared dashboard port range — the wrong tool for a hermetic, parallel-safe test.

Writing another e2e test (copy-template)

  1. Reuse the fixtures. Add a test function to a new or existing module under tests/ui/ that takes page: Page (from pytest-playwright) and dashboard: dict[str, str] as parameters — no manual server boot needed.
  2. Navigate and wait for the real state, never a fixed sleep. 
    page.goto(dashboard["base_url"])
  3. Assert the pre-interaction baseline first. If your flow reveals or changes a hidden element, assert it starts hidden — this is what proves the later "populated" assertion is a real transition, not a coincidentally-passing check against an always-visible element.
  4. Scope every assertion to the specific container you are testing, never page-global. test_dashboard_wp_modal.py's core lesson: several identity fields also render as always-visible card badges before any click, so a page-global expect(page).to_contain_text(...) would still pass even if the feature under test (the modal) dropped the field entirely — a non-vacuous test must scope its locator to the specific container (page.locator("#prompt-modal .agent-identity-section"), for example), not the whole page.
  5. If your scenario needs different fixture data (a second work package, a different lane, a different agent identity), extend tests/ui/conftest.py rather than hand-rolling fixture data inline — keep the synthetic-mission shape as the single source of truth so every test in the suite stays consistent with what the scanner actually reads.
  6. Mark it @pytest.mark.e2e (already applied at module scope in the existing test file via pytestmark = pytest.mark.e2e) so it is correctly selected/excluded by suites that filter on that marker.
  7. Prove non-vacuity before you commit. Temporarily break the render path your test exercises (not the fixture data — breaking the data also reds the always-visible card badge and proves nothing about your new assertion) and confirm your test fails cleanly, then revert.

CI

.github/workflows/ui-e2e.yml runs this suite headless in its own scoped job: install (frozen), cache + install the Chromium binary, then PWHEADLESS=1 uv run pytest tests/ui/ -q. It is deliberately its own workflow file rather than a job inside ci-quality.yml so it stays bounded — one test, one browser install, not a dependency every unrelated CI shard has to wait on (NFR-001).

Edit this page