Testing the Spec Kitty codebase
Everything about running the suite and writing tests that stay honest: the never-retry-to-green flakiness policy, the one correct parallel-run command, local mutation testing, browser regressions, and time-dependent test discipline.
- CLI acceptance scenarios — Gherkin scenarios run nightly against the built CLI on three operating systems.
- Test-pyramid rebalance plan — which layer a test belongs in, and how integration and e2e coverage moves into the scenarios.
- Integration and e2e test inventory — the 2026-10-15 per-file audit behind the plan.
- Test-flakiness handling policy — detection tiers and the never-retry-to-green rule.
- Running the test suite in parallel — the parallel-run workflow and volume gates.
- Run mutation tests locally —
mutmut-based assertion-quality checks. - UI end-to-end tests (Playwright) — the browser-regression pattern (its dashboard targets were removed with the bundled dashboard).
- Write time-dependent tests — inject stable clocks; avoid wall-clock reads in assertions.
- Write permission-denied tests — inject
PermissionErrorat the seam so a test gives the same verdict as root and as a regular user.