Mission Specification: Golden-Path NFR Budget
Mission Branch: issue-4213-golden-path-nfr-budget Created: 2026-09-22 Status: Draft Input: User description: "Work #4213 (canonical tracker) and #4211 (same defect, budget framing) — test_charter_epic_golden_path exceeds its NFR-007 120s budget in every environment post-un-quarantine. Un-skip the test, close the budget gap without loosening the cap, and close both issues from one PR."
Clarifications
The constraints below come from different sources; each is attributed to its real origin rather than presented as one undifferentiated block of operator decisions. The operator decisions are genuinely the operator's own calls. Two attribution tiers apply, and they are formatted differently on purpose: Scope and Lever below are direct blockquotes because they are verbatim from the original dispatch — quoted exactly as the operator said them. Ruling 1 and Ruling 2 are NOT verbatim quotes — they are the orchestrator's faithful summary of the operator's ruling as recorded in reviews/spec.ruling.md (which itself separates the orchestrator's "Question put" framing from the operator's own short "Operator answer"); they are presented below as plain labeled paragraphs, not blockquotes, so the distinction from Scope/Lever's verbatim quotes is visible at a glance, not just stated in this sentence. The remaining constraints come from the issues' own acceptance text and the readiness probe, not from the operator directly. Ruling 1 and Ruling 2 were recorded 2026-09-23 resolving a round-2 adversarial-review HALT (see reviews/spec.ruling.md); they replace the original acceptance bar for the findings that triggered that HALT.
Operator decisions
> Scope: "Work #4213 as the canonical tracker; one PR closes both #4213 and #4211."
> Lever: "A+C — redesign the fresh_e2e_project fixture (~29s of setup: git init, spec-kitty init, commit; single consumer, so a real redesign, not amortisation) AND trim the residual ~0.2s/call Typer command-surface construction that #4417 deferred." Lever B (reduce installed-CLI subprocess call count / in-process substitution) was explicitly rejected as part of this same lever decision. Lever D (loosening the 120s cap or moving the test's cadence) was also explicitly considered and rejected as part of this same lever decision — see C-001 (no cap loosening) and C-002 (no cadence move), which independently enforce this rejection as binding constraints.
Ruling 1 — skip-regression gate (2026-09-23) (operator ruling, summarized from reviews/spec.ruling.md; not a verbatim quote): the operator's ruling was to cut the previously-drafted FR-006/SC-006 CI gate against re-applying @pytest.mark.skip/xfail to budget-guarded tests. The "never silently re-skip or re-quarantine" rule stays as prose only, in C-005; the active 120s timeout is what makes a budget miss loud, and the PR stays a test + CLI-startup change.
Ruling 2 — closure margin (2026-09-23) (operator ruling, summarized from reviews/spec.ruling.md; not a verbatim quote): the operator's ruling was that closure requires a hard ≤110s measurement on a real GitHub Actions run, with the 120s timeout active. There is no exception band — a measurement in the 110-120s band is not closable by a documented exception; it stops the mission and returns it to the operator for a new, explicitly-scoped decision.
Ruling 6 — lever C scope (2026-09-23) (operator ruling, summarized from reviews/plan.ruling.md; not a verbatim quote): raised during the plan phase, after profiling found that most of the golden path's CLI-startup cost is not the #4417-deferred Typer command-surface construction the original Lever quote above named, but ensure_global_agent_commands()/assess_global_agent_commands() (src/specify_cli/runtime/agent_commands.py), which re-renders every configured agent's command templates on every non-fast-pathed CLI call. The operator's ruling was both, freshness check first: lever C's primary mechanism is now a freshness pre-check on ensure_global_agent_commands()/assess_global_agent_commands() that skips the redundant global agent-command render when nothing has changed; the #4417-deferred Typer command-surface trim stays in scope as a secondary, complementary change. Both are in this mission and this PR. Binding condition: the freshness check must be proven red-first not to cause silent staleness — there must be tests that fail if the check wrongly skips a render it should not skip, at minimum (1) when a command template changes, (2) when the spec-kitty version changes, and (3) when the rendered output on disk is missing or was altered. A missed refresh must never pass silently. The original Lever quote above stays intact and attributed as originally recorded; Ruling 6 expands, rather than replaces, what FR-003 covers (see FR-003 below).
From the issues' own acceptance text / the readiness probe (not operator decisions)
- The 120s NFR-007 cap is NOT loosened. Do not propose raising it.
- The test stays on its current gate/cadence — no moving it off the per-push
tests (e2e)shard. - The
@pytest.mark.skipdecorator citing #4213 (currently at lines ~797-805 oftests/e2e/test_charter_epic_golden_path.py) is removed as part of this mission. - The in-process-substitution route for the 12 installed-CLI subprocess calls in the golden path (lever B) is rejected — per the issue trail and robertDouglass's own #4213 comment — the test must keep exercising the real installed CLI via subprocess, not swap in in-process calls.
- Closure requires a real GitHub Actions runner measurement, not only a local measurement — historical CI/VM numbers ran 12-32s slower than local, so a bare "under 120s locally" is not sufficient evidence of closure. Ruling 2 above turns this into the concrete ≤110s hard rule with no exception band.
Glossary note — "installed CLI": the operator quotes above use "installed CLI" to mean the real spec-kitty CLI invoked through a subprocess boundary, as opposed to an in-process/library-level call (the rejected lever B) — not literally a pip-installed package. Mechanically, every CLI invocation in the golden path runs against the source tree via PYTHONPATH override — isolation by construction, not by active detection: tests/conftest.py's isolated_env fixture (tests/conftest.py:1277-1315) strips any inherited PYTHONPATH (tests/conftest.py:1293) and sets it to the source tree only (tests/conftest.py:1299), so a pip-installed version is never importable in the first place — there is nothing to "detect and fail" because the alternative package path is simply not on the interpreter's search path for the subprocess. (A separate helper, assert_test_isolation — tests/test_isolation_helpers.py:70-90 — genuinely does detect-and-fail on a version mismatch, but it is unrelated to the golden path: its only caller in the whole tree is tests/cross_cutting/test_version_isolation_integration.py, a test of the helper itself; neither isolated_env nor the run_cli fixture it backs ever calls it.) The rest of this spec (User Story 1, C-003, SC-005, etc.) uses the more precise "CLI via subprocess (source tree, isolated from any pip-installed package)" phrasing for exactly this reason; both phrasings refer to the same binding decision.
User Scenarios & Testing (mandatory)
User Story 1 - Reliable, in-budget golden-path signal whenever the e2e shard runs (Priority: P1)
A push whose diff touches tests/e2e/ or tests/cross_cutting/ — the CI router's e2e path-filter group (.github/workflows/ci-router.yml:178-180) — selects the per-push tests (e2e) shard (gated at .github/workflows/ci-router.yml:675 on needs.changes.outputs.e2e == 'true'), which runs test_charter_epic_golden_path: the one test that drives the full operator journey (charter synthesize → mission scaffold → next lifecycle → retrospect) through the CLI via a real subprocess boundary — source tree, isolated from any pip-installed package (see C-003) — exactly as a real operator would invoke it. An ordinary push that changes only CLI source (src/specify_cli/cli/) does not select this shard: cli is itself a named router filter group (.github/workflows/ci-router.yml:126-127), so such a change neither joins the e2e/cross_cutting path set nor trips the router's fail-closed unmatched catch-all (.github/workflows/ci-router.yml:202-240), which only forces a full run when a src/ change matches no named group at all. Whenever the shard does run, the contributor needs this signal to be present (not silently skipped), to complete within its budget, and to be trustworthy — a red result means a real regression, not noise from an underpowered CI runner or a flaky wall-clock assertion.
Why this priority: This is the only P1 because it is the entire mission — #4213/#4211 exist because this exact signal was quarantined (@pytest.mark.skip citing #4213) after being un-skipped by #4017 and found over-budget. Every other requirement in this spec exists to make this story true again, durably. This P1's scope is what the golden-path test actually is: the acceptance mechanism for the NFR-007 budget itself (see SC-001), on the pushes that select its shard (path-scoped, above) — it is not, and this spec does not claim it to be, the pre-merge catch for an ordinary CLI-source-only regression. That catch is FR-005's structural/deterministic startup-regression guard, which runs regardless of which shard a given push selects (see FR-005, SC-002, Acceptance Scenario 3 below).
Independent Test: Remove the skip decorator, run pytest tests/e2e/test_charter_epic_golden_path.py::test_charter_epic_golden_path with @pytest.mark.timeout(120) active (not disabled), on a real GitHub Actions tests (e2e) job. The test must pass, and the recorded wall-clock time must be at or under the NFR-002 hard ≤110s rule. A measurement over 110s — even if still under the 120s timeout — does not satisfy this Independent Test: per NFR-002/SC-003 it is a STOP condition, not a pass, and is not closable through any documented exception or sign-off.
Acceptance Scenarios:
1. Given the skip decorator is removed and the fixture/CLI-startup levers are applied, When the per-push tests (e2e) shard runs on a real GitHub Actions runner, Then test_charter_epic_golden_path passes within its unmodified 120s @pytest.mark.timeout budget, with a recorded runner-measured wall-clock time at or under the NFR-002 hard ≤110s rule (see NFR-002 for the rationale behind that specific number). A measurement strictly over 110s and under 120s does not satisfy this scenario — per NFR-002/SC-003 it is a STOP condition, not a pass, regardless of how comfortably it clears the separate 120s timeout. 2. Given a future change regresses the golden path's cost back over budget, When the per-push tests (e2e) shard runs, Then the test fails loudly (the @pytest.mark.timeout(120) marker fires, the job goes red) — it is never silently re-skipped or re-quarantined as a workaround (see C-005). 3. Given a future change regresses CLI startup cost (the lever-C surface: the ensure_global_agent_commands()/assess_global_agent_commands() freshness pre-check, primary, and the Typer command-surface construction trim, secondary — see Clarifications, Ruling 6), When the dedicated startup-time regression guard runs in its designated tier, Then it fails deterministically and non-flakily, without relying on a hard wall-clock assertion inside a fast unit-test shard that could flake under CI contention (see FR-003, SC-002). 4. Given the fresh_e2e_project fixture is redesigned per FR-002, When test_charter_epic_golden_path runs against the redesigned fixture, Then the produced project still contains no .kittify content copied from the source checkout (the fixture's own docstring contract holds, and the existing pre-condition assertion at tests/e2e/test_charter_epic_golden_path.py:485 — assert not doctrine_path.exists() — keeps passing unmodified), and no other tests/e2e/ test is observed to have gained or lost state as a result of whatever the redesign shares (see C-006).
Edge Cases
- What happens if the
fresh_e2e_projectfixture redesign (lever A) changes fixture-scoped state in a way that breaks the single consumer's isolation assumptions (e.g. a redesigned fixture-sharing mechanism reused in a way that leaks state between tests)? The redesign must preservefresh_e2e_project's documented contract — a truly clean, from-scratch project per test invocation (per its own docstring: "does NOT copy.kittifyfrom the source checkout... exercises the operator path from a truly clean starting state (FR-003, FR-020)" intests/e2e/conftest.py) — even while replacing its per-invocationgit/spec-kitty initsubprocess costs with a cheaper mechanism (see FR-002). This preservation requirement is a testable constraint here, not only prose — see Constraint C-006 and Acceptance Scenario 4 under User Story 1. - How does the mission handle the case where lever A + lever C together are insufficient to clear the Actions-runner margin? This is a real risk surfaced in the readiness report (historical CI/VM numbers ran ~12-32s slower than the 127.55s local baseline — see NFR-002 for the precise per-run figures). The mission must re-measure on a real Actions runner before declaring closure — a local pass under 120s is necessary but not sufficient evidence (see NFR-002, SC-003). If that real Actions-runner re-measurement still does not clear the NFR-002 ≤110s hard rule, the mission has no implicit fourth lever to fall back on — C-001, C-002, and C-003 already forbid loosening the cap, moving cadence, or in-process substitution. Per NFR-002's hard rule (no exception band — a 110-120s measurement is not closable by any documented exception), the mission stops and returns to the operator for a new, explicitly-scoped decision, rather than defaulting to any of the three already-rejected levers.
- What happens to the
#4017runtime-side "Global asset input changed" concurrency defect if it resurfaces during this mission's work? It is explicitly out of scope (a separate, already-claimed sibling mission under epic #1931) and must not be folded in or worked around here; if evidence of it appears, it is a tracer-tooling-friction note and an escalation, not a fix target.
Requirements (mandatory)
Functional Requirements
| ID | Title | User Story | Priority | Status |
|---|---|---|---|---|
| FR-001 | Un-skip the golden-path test | As a contributor, I want test_charter_epic_golden_path to actually run in the per-push e2e shard (not silently skipped) so that a regression in the operator journey is caught before merge. | High | Open |
| FR-002 | Redesign fresh_e2e_project fixture cost (lever A) | As a contributor, I want the ~29s setup cost of the fixture (bare git init + two git config calls + spec-kitty init via subprocess + git add/commit — tests/e2e/conftest.py lines 390-452) reduced by real restructuring of what the fixture's own per-invocation subprocess calls do — replacing the git init/config subprocess calls with a faster/leaner mechanism (the plan phase chooses the concrete mechanism), and/or reducing what the spec-kitty init subprocess call itself does for this fixture's specific needs (see C-003 — this call must remain a real subprocess invocation of the installed/source-tree CLI; only its internal cost may be reduced, never converted to an in-process/library call) — rather than a relabeling, so that the golden path's total wall-clock time drops meaningfully. This fixture has exactly one consumer (test_charter_epic_golden_path.py) and builds no venv or editable install per invocation, so neither "amortize across the e2e shard" nor "share venv/editable-install groundwork" (as variously framed on the issue thread) is a real lever here — the redesign must target the fixture's actual git-subprocess and spec-kitty init costs directly, while preserving fixture isolation (see Edge Cases and C-006). | High | Open |
| FR-003 | Cut CLI command-surface construction/render cost (lever C, expanded per Ruling 6) | As a contributor, I want lever C's cost reduced through two mechanisms, primary then secondary (see Clarifications — Ruling 6): primary — a freshness pre-check on ensure_global_agent_commands()/assess_global_agent_commands() (src/specify_cli/runtime/agent_commands.py) that skips the redundant re-render of all configured agents' command templates on a non-fast-pathed call when nothing has changed; secondary — the ~0.2s/call Typer command-surface construction cost that PR #4417 explicitly deferred (its own "Deferred" section: "the init command still imports the whole charter/doctrine provisioning stack at module scope... Making the command modules lazy would shave more"), so that each of the golden path's 12 subprocess CLI calls costs less. Both are in scope; the freshness check must be proven red-first per Ruling 6's binding condition (see Clarifications). | High | Open |
| FR-004 | One PR closes both #4213 and #4211 | As the maintainer, I want a single pull request out of this mission that resolves the shared root cause and is written up to close both #4213 (canonical tracker, "hang" framing empirically disproved — see evidence below) and #4211 (budget framing, the accurate one per robertDouglass's own #4213 comment), so that the tracker does not carry two open issues for one fixed defect. | High | Open |
| FR-005 | Startup-time regression guard for lever C | As a maintainer, I want a regression guard that catches a future reintroduction of the lever-C cost, following the same two-tier pattern #4409/#4417 already established (tests/performance/test_cli_startup_budget_4409.py): a structural/deterministic check that runs everywhere (e.g. no reintroduction of the specific eager-construction pattern this mission removes), plus a wall-clock check scoped to the performance-marked nightly lane, not the per-PR fast-unit tier. | Medium | Open |
Non-Functional Requirements
| ID | Title | Requirement | Category | Priority | Status |
|---|---|---|---|---|---|
| NFR-001 | 120s NFR-007 cap unchanged | test_charter_epic_golden_path keeps its existing @pytest.mark.timeout(120) marker at its current value (120 seconds), unmodified by this mission. This is the NFR-007 budget the test enforces; it is the acceptance mechanism itself (see SC-001), not a target to relax. | Reliability | High | Open |
| NFR-002 | CI-runner-verified margin | The golden path's measured wall-clock time on a real GitHub Actions tests (e2e) job must be ≤110s, with no exception band. Rationale for this specific number: the readiness-session local baseline (post-#4417, skip removed, timeout plugin disabled) measured 127.55s on a quiet local machine (readiness-session measured-results log) — this is pytest's own total-session wall-clock figure, not the same quantity as the test's narrower 102.97s setup+call+teardown execution-only sub-total (see "Evidence carried into this spec" below for the basis correction), while historical CI/VM measurements for the same test ran slower than that same current local baseline (139.7s implementer VM is ~12s slower than 127.55s; 159.5s merge-agent worktree is ~32s slower than 127.55s — all three figures are total-session numbers, the same basis; compared here against the current post-#4417 local baseline, the same comparison basis "Evidence carried into this spec" below uses; not against the different, pre-#4417 ~156s local number that was current in that same PR #4207/#4209 era). Note also that the 139.7s/159.5s CI/VM figures themselves predate PR #4417 (merged 2026-09-15) — they are from that same PR #4207/#4209-era measurement (2026-09-11 or earlier) — so this comparison only corrects the local-baseline side of the gap to post-#4417; the true post-#4417 CI-vs-local gap on a real Actions runner is unmeasured and could differ materially (in either direction) from the cited ~12s/~32s figures. This does not change the hard ≤110s rule below, which depends on a real future Actions measurement, not on this rationale. A bare "passes locally under 120s" would not be durable evidence against that spread; ≤110s on the actual Actions runner is the number this mission must demonstrate, not infer. Hard rule, no exception band (operator Ruling 2, 2026-09-23 — see Clarifications): a real Actions measurement at or under 110s satisfies this NFR and SC-003/closure. A measurement strictly over 110s and under 120s does not satisfy this NFR and is not closable through any documented exception, sign-off, or other self-authorized path — it is a STOP condition: the mission stops and returns to the operator for a new, explicitly-scoped decision, rather than a sign-off that closes the mission as-is. A measurement at or over 120s is an outright failure — the @pytest.mark.timeout(120) marker fires. | Performance | High | Open |
| NFR-003 | CLI-operation latency context for lever C | The charter's §Performance standard ("CLI operations must complete in < 2 seconds for typical projects" — .kittify/charter/charter.md) is supporting rationale for FR-003's scope, not a new requirement beyond what #4211/#4213 already ask for: every subprocess CLI call the golden path makes is a spec-kitty invocation subject to that standard, so trimming lever-C's per-call construction/render cost serves both this mission's budget and the charter's general CLI-latency expectation. | Performance | Low | Open |
Constraints
| ID | Title | Constraint | Category | Priority | Status |
|---|---|---|---|---|---|
| C-001 | No cap loosening | The 120s NFR-007 cap (the @pytest.mark.timeout(120) value) must not be raised, removed, or otherwise loosened by this mission, under any circumstance, including as an interim measure. | Technical | High | Open |
| C-002 | No cadence move | test_charter_epic_golden_path stays on its current gate — the per-push tests (e2e) shard, which the CI router (.github/workflows/ci-router.yml:178-180,675) selects only when a push's diff touches tests/e2e/ or tests/cross_cutting/ (or, rarely, when a src/ change matches no named filter group at all — the fail-closed unmatched catch-all, .github/workflows/ci-router.yml:202-240; an ordinary CLI-source-only change does not hit this, since src/specify_cli/cli/ is itself the named cli group, .github/workflows/ci-router.yml:126-127). This mission must not move it to the nightly lane, mark it performance-only, or otherwise change when/how often it runs, even temporarily; this mission does not alter the router's path filters or gate conditions. This re-litigates and reaffirms the controller correction on PR #4209 (2026-09-11) that reverted an earlier un-skip back to skip rather than moving cadence. | Business | High | Open |
| C-003 | No in-process substitution (lever B rejected) | The golden path must continue to exercise the real spec-kitty CLI via a real subprocess boundary — source tree, isolated from any pip-installed package per tests/conftest.py's isolated_env fixture (tests/conftest.py:1277-1315) — for all 12 of its CLI invocations. These 12 calls run through the run_cli fixture's inner _run_cli helper, defined at tests/conftest.py:1318-1338 (not tests/e2e/conftest.py); the fixture's own spec-kitty init setup call separately uses run_cli_subprocess, defined at tests/test_isolation_helpers.py:93-124 and merely imported into tests/e2e/conftest.py at line 22 — both build the same underlying subprocess command. Reducing subprocess call count or substituting any in-process/library-level call for a subprocess call is explicitly rejected as a lever for this mission — per the issue trail and robertDouglass's own #4213 investigation comment, the test's documented purpose is exercising the CLI across a real subprocess boundary, and weakening that would undermine the signal this mission is trying to make trustworthy again. This subprocess-boundary requirement binds the fixture's own spec-kitty init setup call too, not only the 12 test-body invocations: FR-002's license to reduce "what the spec-kitty init subprocess call itself does" may only trim the work performed inside that one subprocess invocation (e.g. narrowing what init provisions for this fixture's specific needs) — it must remain a real subprocess invocation of the installed/source-tree CLI via run_cli_subprocess, never converted to an in-process/library call. This applies the same lever-B rejection to every CLI invocation the golden path makes, fixture-setup and test-body alike, not just the 12. | Technical | High | Open |
| C-004 | Skip decorator removed | The @pytest.mark.skip(reason="#4213: ...") decorator currently at tests/e2e/test_charter_epic_golden_path.py lines 797-805 (the decorator only — opening at line 797, closing at its closing paren on line 805) is removed as part of this mission's changes — not left in place, not narrowed, not converted to xfail. Line 806 (@pytest.mark.timeout(120)), immediately below the removed decorator, and line 807 (def test_charter_epic_golden_path) must remain untouched — see NFR-001/C-001; only the skip decorator itself is removed. | Technical | High | Open |
| C-005 | Silent success is forbidden — loud failure only | If, after this mission ships, the golden path's cost regresses past its 120s budget on CI, the test must fail loudly: the @pytest.mark.timeout(120) marker fires and the tests (e2e) job goes red. It must never be silently re-skipped, re-quarantined, or have its budget quietly raised as a reflexive response to that red — the defect class this mission fixes (@pytest.mark.skip citing #4213, added by PR #4209's "controller correction" after #4017's un-quarantine exposed the real cost) is exactly a prior instance of exactly that silent-success failure mode, and it must not recur. Any future budget breach is a new, explicitly-scoped decision (per the charter's §Pre-existing Failure Reporting Rule and §Red-main & release discipline standing order), not a default reaction. | Regulatory | High | Open |
| C-006 | Fixture isolation preserved | The redesigned fresh_e2e_project fixture (FR-002) must still produce a truly clean, from-scratch project per test invocation, such that spec-kitty init runs against it under the same precondition it observes today: a repository with no commits and no .kittify content. Explicitly, both halves are required: (1) no .kittify content copied from the source checkout (the existing pre-condition assertion at tests/e2e/test_charter_epic_golden_path.py:485, assert not doctrine_path.exists(), keeps passing unmodified), and (2) no commit made before spec-kitty init runs — today's fixture performs a bare git init plus two git config calls only, then invokes spec-kitty init; the git add/commit happens only afterward, so init itself always observes a zero-commit repository. Whatever mechanism the plan phase chooses to redesign the fixture with must preserve both halves of this precondition, and any newly-shared or restructured groundwork must be verified not to leak state into or out of any other tests/e2e/ test. See Edge Cases and Acceptance Scenario 4 (User Story 1). | Technical | High | Open |
Key Entities
test_charter_epic_golden_path(tests/e2e/test_charter_epic_golden_path.py): the single end-to-end test driving the full operator journey —_run_charter_flow→_scaffold_minimal_mission→_run_next_and_assert_lifecycle→_run_retrospect— through 12 subprocess CLI invocations, each routed through therun_clifixture's inner_run_clihelper (tests/conftest.py:1318-1338— see C-003; not defined intests/e2e/conftest.py). Carries@pytest.mark.timeout(120)(NFR-007's enforcement mechanism) at line 806 and, currently, the@pytest.mark.skipdecorator this mission removes (lines 797-805 — see C-004).fresh_e2e_projectfixture (tests/e2e/conftest.py, lines 390-452): builds a from-scratch Spec Kitty project (baregit init+ twogit configcalls +spec-kitty initvia therun_cli_subprocesshelper —tests/test_isolation_helpers.py:93-124, imported intotests/e2e/conftest.pyat line 22 — +git add/commit) for the golden-path test's exclusive use. Currently costs ~29s of the test's ~127.55s measured local total (the "setup" phase). Single consumer of this fixture in the whole test suite. Must still satisfy its own from-scratch/no-.kittify-copy contract after redesign — see C-006.- CLI command-surface construction/render cost (lever C, expanded per Ruling 6): two mechanisms, primary then secondary. Primary:
ensure_global_agent_commands()/assess_global_agent_commands()(src/specify_cli/runtime/agent_commands.py) unconditionally re-renders every configured agent's command templates on every non-fast-pathed CLI call, with no freshness pre-check today. Secondary: the ~0.2s/call residual cost PR #4417 (#4409) explicitly deferred — Typer app/command-tree construction and eager module-scope imports (e.g. theinitcommand's charter/doctrine provisioning stack) that happen after package import but before a command does real work. Distinct from thejsonschemaeager-import cost #4417 already fixed. - NFR-007 budget (
@pytest.mark.timeout(120)): the 120-second cap this mission must make the golden path fit inside, with margin, without loosening.
Success Criteria (mandatory)
Measurable Outcomes
- SC-001: The
@pytest.mark.timeout(120)marker ontest_charter_epic_golden_pathis itself the falsifiable acceptance mechanism for the budget requirement (per the charter's ATDD-first / C-011 discipline: a test that fails loud when the change it pins is reverted). With the skip decorator removed and the marker active, the test passes on a real Actions run; reverting the mission's fixture/CLI-startup changes must make it fail (timeout) again, proving the marker is load-bearing rather than incidentally green. This is the acceptance mechanism for the NFR-007 budget itself, on the pushes that select this shard (see User Story 1's path-scoped trigger condition) — it is not a claimed pre-merge catch for a lever-C regression on an ordinary CLI-source-only push; that catch is FR-005's structural guard (see SC-002). - SC-002: A startup-time regression guard for the lever-C portion exists, following the #4409/#4417 precedent's two-tier pattern unconditionally (matching FR-005): a structural check (deterministic, runs in the fast per-PR tier, asserts against the specific construction pattern this mission removes — not a wall-clock number) plus a wall-clock check, scoped to the
performance-marked nightly lane only, never asserted as a hard wall-clock threshold inside a fast unit-test shard where CI contention could flake it. - SC-003: Closure is verified on a real GitHub Actions runner — not only local measurement. The mission's evidence must include an actual
tests (e2e)job run showingtest_charter_epic_golden_pathpassing, with its reported wall-clock duration recorded and compared against the hard ≤110s rule from NFR-002 (not merely "under 120s"). Apply NFR-002's hard rule with no exception band: at or under 110s satisfies this criterion and the mission is closable on this point; a measurement strictly over 110s and under 120s does not satisfy this criterion under any documented exception — it is a STOP condition that returns the mission to the operator for a new, explicitly-scoped decision, not a sign-off that closes the mission as-is; at or over 120s is an outright failure, not evidence of closure. - SC-004: The resulting PR's description explicitly closes both #4213 and #4211 (e.g.
Closes #4213,Closes #4211), and both issues' shared root cause (the budget overrun, not a hang — see evidence below) is stated in the PR so a reviewer or future reader does not need to re-derive which framing was correct. - SC-005: No regression to the golden path's e2e fidelity: all 12 subprocess CLI invocations remain real subprocess calls against the real, source-tree
spec-kittyCLI — isolated from any pip-installed package, per C-003 — not converted to in-process/library-level calls (verifies C-003 was honored, not just declared).
Evidence carried into this spec (not re-derived; measured in the readiness session)
- No hang exists. On
main@6b4164dbf(post-#4417 merge), with the@pytest.mark.skipremoved and-p no:timeout(pytest-timeout plugin disabled),test_charter_epic_golden_pathpassed in 127.55s — this is pytest's own reported total session wall-clock time (the "in Xs" figure from pytest's final summary line for that invocation), which includes test collection/module-import overhead in addition to the test's own execution. The readiness session separately recorded this one test's setup/call/teardown phase breakdown — 73.58s call + 29.36s setup + 0.03s teardown, summing to 102.97s — which is not the same quantity as the 127.55s total: the phase breakdown covers only this test's own execution (fixture setup through teardown), not the ~24.6s of collection/module-import overhead pytest pays before any phase timer starts. Basis correction (post-tasks analysis, finding B1): an earlier draft of this section presented the three phase numbers as if they summed to 127.55s; they do not, and this was not an arithmetic slip in this spec but a mislabeling of two genuinely different pytest-reported quantities as one. Both numbers are carried as-is from the readiness session's own log, consistent with pytest's documented reporting model (session total = collection time + every collected test's own setup+call+teardown + session teardown) — this spec treats 127.55s as the total-session figure and 102.97s as a narrower, execution-only sub-total, not as two measurements of the same thing. Source-file note (post-tasks analysis, finding C2): the readiness session's own working files,_readiness/4211/measured-results.mdand_readiness/4211/golden_path_run.log, were ephemeral design-phase investigation scratch — they were never committed to this repository and do not exist in this checkout or its git history. They are cited here only as the provenance of the numbers above (carried forward from that investigation, not re-derived), not as files a later reader can open; do not go looking for them. This independently corroborates robertDouglass's 2026-09-14 finding on #4213 (different machine, same conclusion: the reported "hang" stack waspytest-timeoutkilling the run wherever it happened to be inside arun_clisubprocess call, and the reported kill-site moved between runs —_run_charter_flow:488in the original filing vs._scaffold_minimal_mission:589in his re-run — which a genuine block would not do). spec-kitty --helpmeasured at 0.83s warm this session, confirming PR #4417 (merged 2026-09-15) landed and works as claimed (down from pre-fix ~3.2s).- Historical CI/VM runs were slower than local: 139.7s (implementer VM, PR #4207/#4209 era), 159.5s (merge-agent worktree adjudication) — both red against the 120s cap, both higher than this session's local 127.55s. All three of these numbers (127.55s, 139.7s, 159.5s) are total-session pytest wall-clock figures, the same basis (see the correction above) — the comparison is session-total to session-total throughout, not session-total against the narrower 102.97s execution-only sub-total. This is why NFR-002 sets a stated margin (≤110s) rather than accepting a bare "passes locally."
fresh_e2e_projecthas exactly one consumer — confirmed via grep acrosstests/; onlytest_charter_epic_golden_path.pyuses it. "Amortize across the e2e shard" as originally framed on the issue thread is not literally available without a real redesign of what gets shared (FR-002, Edge Cases).- No open-PR lane-scope conflict: none of the four PRs open at readiness time (#4942, #4938, #4936, #4932) touch
tests/e2e/test_charter_epic_golden_path.py,tests/e2e/conftest.py, or CLI-startup paths. - #4017 is out of scope: a separate runtime-side sibling defect (
ensure_runtime()'s "Global asset input changed" asset-recheck under concurrency), already claimed under a different mission (Mission B, epic #1931, assignee stijn-dejongh). This session's clean golden-path pass showed no asset-recheck symptom, so #4017 is not currently entangled with this mission's failure mode.