Spec Kitty

└─ kitty-specs
   └─ CLI Acceptance Scenarios & Cucumber-JVM Nightly

Mission Run:

📚 Docs ↗

Research & Grounding Note: CLI Acceptance Scenarios & Cucumber-JVM Nightly

_Grounded 2026-10-11 against a local working tree, branch cli-acceptance-bdd-nightly-01M4NZRJ. Read-only; nothing modified._

Mission: cli-acceptance-bdd-nightly-01M4NZRJ. Spec: kitty-specs/cli-acceptance-bdd-nightly-01M4NZRJ/spec.md. Governing ADR: docs/adr/4.x/2026-10-11-3-acceptance-testing-and-bdd-support.md (Accepted 2026-10-11).

0. Headline findings (read these first)

The ADR-as-accepted contradicts the spec on three points the FR-001 amendment must reconcile. The accepted ADR mandates: tests/scenarios/promise-index.json (public slice of QA's index), a per-PR stdlib check tests/architectural/test_scenario_promise_tags.py (asserts @promise ids are ratified, one broken fixture per rule), the scenarios ci-router group gating that architectural job, and a PROMISE_INDEX_SYNC_TOKEN org sync (Open Question 1). The owner ruling (2026-10-11) drops all four from core → spec-kitty-qa. None of these artifacts exist in the tree yet (test_scenario_promise_tags.py, promise-index.json — absent), so the amendment is recording a narrowing that was never built, not deleting code. The amendment must restate: scenarios gates no job (was "gates the architectural job"); acceptance_jvm still gates no job (unchanged); @promise/@defect/@qa inert in core.
The single trickiest green-keeping edit is tests/architectural/test_workflow_coherence.py:310 — _DELIBERATELY_UNGATED_FILTER_GROUPS = frozenset({"any_src", "ci"}). FR-003b (test_every_restored_filter_group_is_consumed_live) reds for any dorny filter group that gates no job unless it is in this set. The two new groups scenarios + acceptance_jvm MUST be added here.
FR-003c (test_every_restored_filter_glob_is_live) forces an ordering/atomicity constraint: tests/scenarios/ and acceptance-jvm/ globs must each match ≥1 git-tracked path or the router change reds. The seed .feature files and the acceptance-jvm/ subproject must be committed in (or before) the same change that adds the router groups.
contracts/ pins Gradle via pins.json + install_tools.py, NOT a committed gradle-wrapper.properties. There is no wrapper jar and no wrapper properties file in the tree. CI installs the pinned, sha256-checked Gradle zip; CI sets up JDK via actions/setup-java (temurin, currently java-version: '21' — ADR mandates Java 25 for acceptance-jvm). verification-metadata.xml is Gradle-generated (origin="Generated by Gradle"), hand-edit forbidden.
contracts/ runs per-PR (contracts.yml, on: pull_request path-scoped to contracts/).** acceptance-jvm must NOT get an analogous per-PR workflow (NFR-004: 0 per-PR JDK cost) — it runs nightly only.
acceptance-jvm/ is NOT folded into the existing corpus group (which owns contracts/ and DOES gate per-PR jobs). It is its own non-gating group.
Both ci-router.yml (54 commits/90d) and ci-nightly.yml (40 commits/90d) are hot files — high rebase/merge risk. pyproject.toml is extremely hot (244/90d) but its sdist section is stable. contracts/ is stable (2 commits, all from #5558).

1. Target files — sizes + responsibility map

Path	Lines	Owner / responsibility	Mission action
`.github/workflows/ci-nightly.yml`	1443	Nightly full-mode suites; every current job runs on `ubuntu-24.04` (single-OS); `nightly-summary` aggregates `needs:` and echoes results (never fails the run).	ADD `acceptance-bdd` job, 3-OS matrix (FR-010).
`.github/workflows/ci-router.yml`	1164	Two-authority path router. Authority #1 = dorny `changes` filter block (group→globs); authority #2 = job `if: needs.changes.outputs.<group>` (group→job). `scripts/ci/gate_selection.py` PARSES both.	ADD `scenarios` + `acceptance_jvm` filter groups + `changes.outputs` entries; gate NO job (FR-012, C-005).
`.github/ci-module-registry.yml`	957	Single data source for module set (module names, roots, test_dirs); consumed by `gate_selection._registry_rows`.	No change needed (new groups are non-src, no module/shard).
`scripts/ci/gate_selection.py`	~260	The ONE importable gate-selection authority (#2476). `_job_gates` reads each job's `if:` for `needs.changes.outputs.<grp>`; a group with no job `if:` reference gates nothing. `PROBE_GROUPS={"any_src"}`.	No change — it parses, doesn't enumerate.
`tests/cross_cutting/packaging/test_packaging_safety.py`	167	Wheel/sdist packaging invariants. Uses session fixture `build_artifacts` (wheel+sdist). Tests marked `@pytest.mark.slow` + `integration`.	ADD sdist-exclusion assertion for `acceptance-jvm/` + `tests/scenarios/`, paired with a positive control (FR-013).
`pyproject.toml` `[tool.hatch.build.targets.sdist]` (L207–225)	—	`include=["src//","packs/built-in/",...]`; `exclude=[".kittify/`*`",".kittify/active-mission","src/specify_cli//tests/"]`. Wheel force-includes `packs/built-in` only.	ADD `"acceptance-jvm/"` + `"tests/scenarios/``"` to sdist `exclude` (FR-013, C-004).
`src/specify_cli/cli/commands/init_help.py`	57	`INIT_COMMAND_DOC` — source of truth for the two seed promises.	Read-only source for FR-003/FR-004 scenarios.
`src/specify_cli/cli/commands/init.py`	—	The actual `init` command the doc describes.	Behaviour under test (black-box; steps never import it).

The two seed promises, verbatim from init_help.py:

FR-003 (idempotent restore): _"Re-running init in an already-initialized directory is idempotent for project state: it verifies the configured agents' skill surfaces, additively restoring missing per-agent skill roots (e.g. .claude/skills/) through the canonical installer..."_
FR-004 (no git): _"Creates project files only. Does not initialize a git repository. Does not create any commits."_ + _"init never touches git state regardless of flags."_

2. The `contracts/` Gradle precedent — exactly how to mirror it

Files: contracts/build.gradle (2940 B), contracts/settings.gradle (638 B), contracts/gradle/verification-metadata.xml (1032 lines / 63517 B), contracts/tools/pins.json, contracts/tools/install_tools.py, contracts/tools/gradle_pin_check.py, contracts/README.md (25220 B). No gradle/wrapper/ dir, no gradle-wrapper.properties, no gradlew/gradlew.bat, no wrapper jar exist (find contracts -name "*.properties" → empty).

Distribution pinning + checksum (the mechanics to copy)

Pinned in contracts/tools/pins.json, entry name: "gradle", kind: "archive", version: "8.14.5", url: https://services.gradle.org/distributions/gradle-8.14.5-bin.zip, sha256: 6f74b601422d6d6fc4e1f9a1ab6522f642c2fdcbc15ae33ebd30ba3d7198e854, plus published + advisory_feed_checked provenance strings.
contracts/tools/install_tools.py (stdlib only) downloads the zip, refuses to unpack or execute until the bytes match the pinned sha256 (codes: CHECKSUM_MISMATCH, CHECKSUM_MISSING, NOT_HTTPS, UNSAFE_ARCHIVE, UNSUPPORTED_INTERPRETER, BINARY_MISSING), then appends the tool's bin/ to $GITHUB_PATH. Invoked in CI as python contracts/tools/install_tools.py --pins contracts/tools/pins.json --dest "$RUNNER_TEMP/tools" --only gradle.
contracts/tools/gradle_pin_check.py (stdlib + PyYAML) cross-checks that the versions the build USES equal the pins: the id 'org.openapi.generator' version '...' line of build.gradle, the plugin components of verification-metadata.xml, a gradle-wrapper.properties only if it exists (_check_wrapper early-returns when absent — so the no-wrapper pattern is a first-class supported state), and every gradle-version: input of contracts*.yml workflow steps. One failure code: GRADLE_PIN_MISMATCH.

No committed wrapper jar

settings.gradle comment is explicit: _"Invoked ... with a pinned Gradle distribution; there is no committed wrapper jar (see tools/install_tools.py and tools/pins.json)."_ CI invokes the installed gradle binary directly (on $PATH), not ./gradlew.

verification-metadata.xml

Stored at contracts/gradle/verification-metadata.xml, 1032 lines. Schema dependency-verification-1.3.xsd; <configuration><verify-metadata>true</verify-metadata><verify-signatures>false</verify-signatures></configuration>; one <component>/<artifact> per dependency with <sha256 ... origin="Generated by Gradle"/>.
Generated, never hand-edited (build.gradle comment: _"generated, never hand-edited"_). The canonical generation command is documented in the supply-chain toolguide (see §4): gradle --write-verification-metadata sha256 <tasks> --no-daemon (plus --write-locks only when dependencyLocking is enabled). README line 243 references contracts/gradle/verification-metadata.xml and the gradle-version pin together.

JDK

ADR + spec require Java 25 for acceptance-jvm (per the 2026-10-03 amendment of ADR 2026-10-01-2). Note divergence: contracts.yml currently pins actions/[email protected], distribution: temurin, java-version: '21'. So acceptance-jvm cannot copy the JDK version verbatim — it bumps to 25.

Is `contracts/` run in CI today?

Yes — per-PR, via .github/workflows/contracts.yml (on: pull_request + push: main, path-scoped to contracts/, tests/contract/, the two contract workflows, uv.lock, pyproject.toml, pytest.ini, tests/conftest.py, src/specify_cli/status/lifecycle_events.py). It validates/bundles/cross-checks contracts; gradle_pin_check.py runs at L72; Gradle installed at L130; JDK set up at L124/L327. This is the anti-pattern for acceptance-jvm: the new runner must be nightly-only (NFR-004), so do NOT create an acceptance-jvm.yml per-PR workflow.

3. Invariants that must survive (with citations)

Packaging (sdist/wheel)

``python with tarfile.open(sdist_path, "r:gz") as tar: all_files = tar.getnames() bad_kittify_files = [f for f in all_files if ".kittify/" in f and "/src/" not in f] assert not bad_kittify_files, ... ``

Wheel ships only src/ packages + packs/built-in — ADR §Packaging ("The wheel ships only the src/ packages and is unaffected"); CLAUDE.md "Pack Tiers"; enforced by test_packaging_safety.py::test_wheel_contains_only_known_packages (known_prefixes = specify_cli/, doctrine/, charter/, kernel/, glossary/, mission_runtime/, runtime/, packs/built-in/) and test_wheel_config_only_includes_built_in_pack_skills (config-only twin). The runner/scenarios never touch the wheel — they are top-level dirs outside src/, so wheel tests already exclude them; no wheel assertion change needed.
sdist exclusion is the FR-013 deliverable. Current sdist exclude (pyproject L221–225) does NOT list acceptance-jvm/tests/scenarios. Must add both. The existing packaging test has no generic sdist-exclude assertion — only test_sdist_contains_no_kittify_paths (L~155). FR-013 adds a NEW @pytest.mark.slow test asserting both paths are absent from the built sdist AND a positive control (e.g. a src/ package IS present — mirror the test_wheel_contains_templates positive-control style).
Exact assertion shape to mirror (quote, test_sdist_contains_no_kittify_paths):
Positive-control style to mirror (test_wheel_contains_templates): assert template_files, "Wheel missing template files".

ci-router group→gate model (the "gates no job" mechanic)

Two authorities, one parser. ci-router.yml header (L1–30): authority #1 = dorny filter block (group→globs); authority #2 = job if: needs.changes.outputs.<group> (group→job). gate_selection.load_router parses both; _job_gates (L108–115) maps each job→frozenset of the groups its if: references.
"Gates no per-PR job" = no job's if: references the group. select_gates (L134–163): gated_selected = {job for job,groups in router.job_gates if groups & selected_groups}. A group that appears in no job if: contributes to nothing. This is exactly how ci and any_src work today. Reference comment in ci-router.yml for the ci group (L~262): _"No job gates on this group BY DESIGN."_
Workflow (never push main): CLAUDE.md "Git Workflow" — main is integration-only; open a topic branch + PR into main; spec-kitty consolidate is local-only. Branch is already cli-acceptance-bdd-nightly-01M4NZRJ.

Supply-chain pinning (NFR-003)

100% JVM deps in generated verification-metadata.xml; Gradle distribution checksum-pinned; 0 wrapper jars committed. Same model as §2.

4. Architectural / AST gates that reference or would be tripped by these files

Ranked by likelihood of tripping:

1. tests/architectural/test_workflow_coherence.py — FR-003b (HIGH, WILL TRIP).

_DELIBERATELY_UNGATED_FILTER_GROUPS = frozenset({"any_src", "ci"}) (L310). unconsumed_filter_groups (L330) = (filter_groups - this_set) - consumed_by_any_job_if. test_every_restored_filter_group_is_consumed_live (L128) asserts it is empty.
To keep green: add "scenarios" and "acceptance_jvm" to this frozenset. (Honest: they genuinely gate no job, so the exemption is truthful, not green-washing.) The fault-injection twin test_faultinjection_deliberately_ungated_group_is_exempt_but_orphans_still_red (L426) proves the set still catches real orphans.

2. tests/architectural/test_workflow_coherence.py — FR-003c (HIGH, ORDERING CONSTRAINT).

test_every_restored_filter_glob_is_live (L157) + glob_is_live (L338): every filter glob must match ≥1 git-tracked path (_tracked_paths = git ls-files, L109).
To keep green: tests/scenarios/ and acceptance-jvm/ must be tracked when the router groups land. Commit seed .features + the subproject with/before the router edit. VESTIGIAL_FILTER_GLOBS ledger (L56) is empty {} and is for DEAD globs only — do NOT add rows for live globs.

3. tests/architectural/test_ci_router_transcription_guards.py (MEDIUM).

test_router_src_filters_derive_from_scrub_verbatim (L104): requires scrub groups ⊆ router filters, and that src-backed router groups == src-backed scrub groups. New groups are non-src, and the scrub (tests/release/ci_retirement_scrub.json, groups list confirmed: consolidation, missions, post_merge, release, status, review, next, lanes, upgrade, cli, charter, agent, kernel, glossary, execution_context, core_misc, unit, specify_cli_runtime, ci, auth) requires NO entry for them (precedent: mission_state, corpus, ci_config, contract_tools, docs, e2e, architectural are all non-src router groups absent from scrub and pass). No scrub edit needed.
test_changes_outputs_cover_every_routing_group (L148, WILL TRIP if forgotten): changes.outputs (minus unmatched) must EQUAL the filter routing groups. Must add scenarios + acceptance_jvm to the changes.outputs block (mirror the existing foo: ${{ (inputs.mode == 'full' || ...unmatched...) && 'true' || steps.filter.outputs.foo }} line shape).
test_unmatched_union_covers_every_src_backed_group (L172): new groups are non-src → must NOT be added to the unmatched union loop. (Match docs/corpus/ci treatment.)

4. tests/architectural/test_gate_selection_authority.py (LOW). Parses real on-disk router; proves docs/corpus are non-src & excluded from unmatched loop. New non-src groups follow the same path; non-vacuity floor unaffected.

5. tests/architectural/test_ci_corpus_trigger_completeness.py (LOW). Corpus-specific (Gate-0 on.paths completeness for @corpus tests). Not triggered by scenarios unless you add @pytest.mark.corpus readers — don't.

6. tests/architectural/test_ci_quality_path_filters.py (LOW). Parses the router filter block shape; two-authority model guard. Adding well-formed groups is within its model.

7. tests/charter_offering/test_supply_chain_single_owner.py::test_covers_maven_strict_checksums_and_gradle_write_verification (L220, NOT TRIPPED — reference only). Asserts the Java toolguide MD contains --write-verification-metadata, verification-metadata.xml, --write-locks, dependencyLocking/lockAllConfigurations. This is doctrine-prose coverage, not a gate on acceptance-jvm/. It is the canonical source for the exact Gradle verification-metadata command to put in the acceptance-jvm README (gradle --write-verification-metadata sha256 ... --no-daemon).

8. tests/release/test_release_ci_ownership.py / tests/ci/test_contracts_workflows.py (CHECK). Pin per-workflow job sets and "every test has exactly one per-PR home." Adding a nightly-only job to ci-nightly.yml should be in scope of ci-nightly, but verify these don't enumerate a frozen nightly job set that a new acceptance-bdd job would violate. (Not read in depth — flag for implementer.)

5. Test seams and what they cost

How packaging tests build artifacts: session fixture build_artifacts (tests/conftest.py:1242) → tests/_support/shared_build_artifacts.py::default_wheel_sdist_builder runs python -m build --no-isolation (NOT uv build; build + hatchling are pinned test extras). ~1 minute per run, built once per pytest run and shared across xdist workers via a kernel.locks machine lock (_LOCK_TIMEOUT_S = 1200). All packaging assertions are @pytest.mark.slow + @pytest.mark.integration → they run in the slow/nightly/full tier, not make test-fast.
Note vs spec: the packaging TEST uses python -m build; the spec/ADR's uv build requirement (C-002) is for the CLI-under-test wheel built by the nightly JVM runner, a separate context. No conflict.
Existing JVM/Gradle test invocation: only contracts.yml (per-PR, path-scoped). There is NO pytest that drives Gradle, and NO JVM test in tests/. acceptance-jvm introduces the first nightly JVM run; its own suite runs inside Gradle/JUnit Platform, not pytest. gradle_pin_check.py is the only Python that reasons about a Gradle build and it has unit tests under tests/contract/ (contract-tools lane).

6. Open-P0 / sibling-run interactions & 90-day churn

File	Commits/90d	Churn verdict
`.github/workflows/ci-router.yml`	54	HOT — high merge risk. Recent adds are the SAME pattern we repeat: `mission_state` group (#5864, `aaccc6b6dc`), contract-cited src routing (#5838/#5776). Rebase likely; keep the router diff minimal and additive.
`.github/workflows/ci-nightly.yml`	40	HOT — high merge risk. Recent: p0-repro lane split (`fcfc7559f0`), interpreter-shard gaps (`f3e4b78856`), test-universe collection (`14bd5dbb35`). Append the `acceptance-bdd` job at the end; decide whether to wire into `nightly-summary.needs` (see below).
`pyproject.toml`	244	Extremely hot overall, but the `[tool.hatch.build.targets.sdist]` `exclude` block is stable. Expect conflicts on unrelated hunks; your 2-line add is low-risk if isolated.
`tests/cross_cutting/packaging/test_packaging_safety.py`	~8	Moderate; last touched by WP05 pack-skill wheel-safety (`7fa2395688`) + landing retirement (`f2b25d23b2`).
`contracts/build.gradle`, `contracts/tools/pins.json`	2	Stable — both from #5558 (Sept 2026). Safe to copy from.

Sibling-run interaction — nightly-summary decision point: ci-nightly.yml:1422 nightly-summary.needs is a fixed 16-job list and the job only echos each result (always(), never fails). The new non-gating acceptance-bdd job can be left OUT of needs (it uploads its own artifacts; nothing downstream in core consumes it — US-3 says QA consumes cross-repo). If added for visibility, extend both the needs: list and the echo block. Recommend: leave out of nightly-summary to minimise the hot-file diff, unless a release-readiness gate requires it (verify release-readiness.yml / tests/release/test_release_ci_ownership.py).
No open P0 reproductions reference these files in recent history (the p0-repro lane commits are about lane plumbing, not these paths). The churn is feature/landing work, not firefighting.
Fork context: this is a shadow clone; several "90-day" commits are convergence imports (ec62c35de3 chore(charter): ... CI workflows). The hot-file signal still holds for merge planning.

7. Decision-relevant contradictions / complications for the spec

1. FR-001 amendment must flip "scenarios gates the architectural job" → "gates no job" and record the deletion-before-birth of promise-index.json, test_scenario_promise_tags.py, and PROMISE_INDEX_SYNC_TOKEN (ADR Open Question 1). These never existed in core, so FR-014's "drop" is really "never add." 2. Atomicity: FR-012 (router groups) cannot land before FR-002/FR-005 (tracked tests/scenarios/ + acceptance-jvm/ files) or FR-003c reds. Sequence the WPs so the directories are tracked first, or land them together. 3. test_workflow_coherence.py:310 frozenset edit is mandatory and is a test-file edit in tests/architectural/ — the mission's implement phase owns it; grounding did not touch it. 4. JDK bump: acceptance-jvm uses Java 25; the contracts/ precedent uses Java 21 — do not copy the java-version verbatim. 5. Gradle version mirror: spec/assumption says "pinned Gradle mirrors contracts/" → 8.14.5, sha256 6f74b601...e854. A separate acceptance-jvm/tools/pins.json (or reuse) + its own verification-metadata.xml is needed; gradle_pin_check.py currently only globs contracts*.yml workflows and contracts/ paths, so it will NOT automatically cover acceptance-jvm — either generalise it or accept that acceptance-jvm pin-drift is unguarded (flag for the implementer; NFR-003 wants it guarded). 6. verification-metadata.xml generation command (copy into the acceptance-jvm README): gradle --write-verification-metadata sha256 <tasks> --no-daemon, run once, commit the result, never hand-edit (origin="Generated by Gradle").

8. Plan-phase decisions (Phase 0 resolution)

DM-01M4P2B3S88TYZAK2WH5HM12PZ — Gradle pin-drift guard scope.
Decision: Generalize the shared contracts/tools/gradle_pin_check.py to also scan acceptance-jvm/ and the acceptance-bdd nightly job's gradle-version, keeping one canonical GRADLE_PIN_MISMATCH authority (IC-04).
Rationale: single canonical authority (charter Quality Standing Orders); fully satisfies NFR-003; the guard already tolerates an absent wrapper file, so the no-wrapper acceptance-jvm shape is first-class.
Alternatives considered: a separate acceptance-jvm pin-check (rejected: duplicated logic, two authorities); accept unguarded drift in v1 (rejected by operator: NFR-003 wants it guarded).
DM-01M4NZV51YAVWX0WHNCZGTB0CS — seed scenario tag posture. Seed init scenarios land bound + untagged (gating run); @manual reserved for heavy/on-demand scenarios. (Resolved in specify; drives FR-003/FR-004 and the Tag entity.)
nightly-summary.needs wiring (no DM — in-scope engineering call): leave the non-gating acceptance-bdd job OUT of the fixed 16-job summary list to minimise the hot-file diff; it uploads its own artifacts and nothing in core consumes it. IC-06 verifies release-readiness.yml / tests/release/test_release_ci_ownership.py do not require it; if one does, wire it in.
Installer reuse (no DM — single-canonical-source call): reuse contracts/tools/install_tools.py for acceptance-jvm's Gradle install (own pins.json), rather than copying the installer.

Supply-chain adversarial-evidence disposition (plan readiness)

Per the adversarial-squad-deployment findings-disposition contract: the dependency decisions (Cucumber-JVM 7, allure-cucumber7-jvm, JUnit Platform, pinned Gradle 8.14.5) replicate the already -reviewed contracts/ supply-chain model (sha256-pinned distribution via a checksum-refusing installer; Gradle-generated per-artifact verification-metadata.xml; no committed wrapper jar; pin-drift guarded). Novelty is low and no new PyPI advisory surface is introduced. The adversarial challenge pass is explicitly deferred to the closeout pre-PR review squad (semi-auto delivery step 8), not run as a standalone pass now. No contested finding is dropped; any raised at closeout will be dispositioned here before the PR flips to ready-for-review.