---
type: "Concept"
title: "CLI Acceptance Scenarios (Cucumber-JVM nightly)"
description: "How to write, run and read the Gherkin acceptance scenarios that check the built spec-kitty CLI every night on Linux, macOS and Windows."
status: "stable"
generated:
  by: "process:dockitty-export"
  at: "2026-11-02T00:00:00Z"
---

# CLI Acceptance Scenarios (Cucumber-JVM nightly)

The acceptance scenarios check what the `spec-kitty` CLI promises its users, from the
outside. They run the CLI built from the commit as a separate process and look only at
what a user would see: files, exit codes and git state. A broken promise turns the
nightly red on every supported operating system.

| Part | Location |
|------|----------|
| Scenarios (Gherkin) | `tests/scenarios/<area>/*.feature` |
| Runner, step definitions, driver | `acceptance-jvm/` (Gradle, Cucumber-JVM 7, Java 25) |
| Nightly job | `acceptance-bdd` in `.github/workflows/ci-nightly.yml` |
| Decision records | ADR `2026-10-11-3` (acceptance testing and BDD support), ADR `2026-10-13-1` (promise links, per-PR check, evidence), ADR `2026-10-15-1` (ownership and absorption) |
| Absorbed engineering tests | `tests/scenarios/absorption-ledger.json` |

## Who owns what

| Part | Owners | Review |
|------|--------|--------|
| Scenarios, the absorption ledger, this guide | QA (spec-kitty-qa) and core | a scenario change asks a QA owner and a core owner (`.github/CODEOWNERS`) |
| Runner, step definitions, driver, fixture builders (`acceptance-jvm/`) | core | core |
| Promise pages, promise index, trace lint, Allure report | spec-kitty-qa | spec-kitty-qa |

QA drafts a scenario from a promise and lands it as `@wip`; an engineer binds its steps and adds
a positive control; `@wip` comes off in a pull request a QA owner and a core owner reviewed.
`tests/architectural/test_scenario_contract.py` checks the CODEOWNERS rules on every pull
request. Requiring code-owner review on GitHub is a repository setting, not set today.

## Write a scenario

1. Pick a promise the CLI makes to its users, such as a sentence from a command's
   `--help` text. Name it in the scenario file's maintainer notes.
2. Write the scenario from the user's point of view: what they have, what they run,
   what they get. Leave test mechanics out of the `Feature` and `Scenario` text.
3. Order the steps Arrange, Assume, Act, Assert (see below), and reuse existing steps
   before adding new ones.
4. Give the scenario a same-fixture positive control, so a command that did nothing
   cannot pass it. For example, "the folder `.claude/skills` exists" proves that `init`
   actually ran before "the file `.kittify/config.yaml` is unchanged" can mean anything.

`tests/scenarios/README.md` lists the available steps.

## Step structure: Arrange, Assume, Act, Assert

Each lifecycle group has its own step class in the `kitty.acceptance.steps`
package of the `acceptance-jvm` test sources. They share one
`ScenarioContext` per scenario, injected by Cucumber's PicoContainer integration.

| Group | Gherkin | Class | Does |
|-------|---------|-------|------|
| Arrange | `Given` | `ArrangeSteps` | Builds the starting conditions and fails fast when that did not work. |
| Assume | `Given` / `And` | `AssumeSteps` | Checks that the starting state is what the scenario expects, and records it for later comparison. |
| Act | `When` | `ActSteps` | Runs the command under test. |
| Assert | `Then` / `And` | `AssertSteps` | Checks the post-conditions. |

Steps talk to the CLI only through the `SpecKittyDriver` port and never import Python
code. Assertions use AssertJ.

### The one Arrange write that is not the CLI's

ADR 2026-10-15-1 decision 6 lists what an Arrange builder may write: the planning files
an agent authors, the code a work package's implementer writes, a reviewer's feedback file
outside the repository, and plain files a user edits in git. It never writes CLI-managed
state.

There is one narrow, documented exception, for a promise whose subject *is* the state the
CLI reduces: a scenario that checks how the status log's ordering is honoured may plant a
row as a teammate's `git pull` delivers it. `MissionLifecycle.plantFutureStampedApproval`
restamps a work package's `approved` row to a fixed ten minutes into the future and commits
it, so the reducer sees a clock that ran ahead of the rejecting reviewer's. The row keeps
exactly the shape the CLI writes (no `lamport_clock` on lane transitions), and everything
after it goes through the CLI. `tests/scenarios/status/event-traceability.feature` is the
only user, added for [#6260](https://github.com/spec-kitty/spec-kitty/issues/6260); do not
copy the pattern for a behaviour a real CLI call can arrange.

### Builders name the topology they mean

Since [#7282](https://github.com/spec-kitty/spec-kitty/pull/7282) (ADR 2026-11-06-3),
`agent mission create` without `--topology` records an unsettled `single_branch` Mission: it
switches the checkout onto a Mission topic branch and makes no coordination branch and no lane
worktree. A builder that means a coordination Mission therefore asks for one:
`MissionLifecycle.planMission` and `createMission`, and `MissionFixture.createMission`, pass
`--topology lanes_with_coord` (`MissionLifecycle.COORDINATION_TOPOLOGY`), and
`planSingleBranchMission` passes `--topology single_branch`. A scenario about the size default
itself must call `mission create` with no `--topology` explicitly, and must commit or discard each
Mission's `spec.md` scaffold before the next create: a default create refuses a dirty checkout
with `WRITE_CHECKOUT_DIRTY`, by design, so one Mission's untracked scaffold never follows the
checkout onto another Mission's branch. A Python acceptance helper
that pins `SPEC_KITTY_HOME` puts the home beside the project checkout, never inside it: cache
files written there would dirty the write checkout, and the Mission-branch switch refuses a
dirty one ([#7342](https://github.com/spec-kitty/spec-kitty/issues/7342)).

## Tags

| Tag | Rule |
|-----|------|
| `@promise:<id>` | Names the promise the scenario checks. `<id>` matches `[A-Za-z0-9][A-Za-z0-9._-]*`, for example `PR-CLI-INIT-001`. Not repeated on one scenario. |
| `@wip` | Draft or under repair. Needs `@wip-owner:<handle>` and `@wip-since:<YYYY-MM-DD>`. Runs in the separate non-gating `acceptance-bdd-wip` nightly job, which never fails a leg and nothing waits on (#7492). |
| `@wip-owner:<handle>` | Owner of the draft. `<handle>` matches `[A-Za-z0-9][A-Za-z0-9._-]*`. |
| `@wip-since:<YYYY-MM-DD>` | Date the scenario was marked `@wip`. |
| `@manual` | Never runs automatically. |
| `@defect:<issue>` | Links a known defect. `<issue>` is a positive integer. |
| `@qa:<ref>` | Reference for the QA repository. `<ref>` uses the identifier character set. |
| `@<bare>` | Bare tag matching `[A-Za-z0-9][A-Za-z0-9_-]*`, with no `:`. |
| none | Runs in the gating nightly. A failure fails that OS leg. |

Any other tag containing `:` is an error. Tags on a `Feature` or `Rule` apply to every
scenario under it, and tags on an `Examples:` block apply to the rows of that table. Everything except `@wip` and `@manual` gates the nightly.

### Linking a scenario to a promise

A scenario links to a promise by identifier only, with `@promise:<id>`. The scenario
carries no promise wording, rationale or status. This repository checks the shape of the
identifier and nothing more; resolving it to a promise happens in the QA repository.

### Drafted scenarios

A scenario drafted from a promise page lands as `@wip`, with its `@promise:<id>` tag,
`@wip-owner` and `@wip-since`. It may use steps that have no binding yet. An engineer
then binds the steps, adds a positive control and removes `@wip`. Tooling never invents
an identifier.

Removing `@wip` and adding `@promise:<id>` are separate steps. The engineer's pull
request binds the steps, adds the positive control and drops `@wip`; the identifier comes
from the QA promise registry, so if it is not assigned yet the scenario gates without it
(the contract check validates the identifier's shape only when the tag is present) and a
QA owner adds the tag in review. Never hold a bound scenario in `@wip` until an identifier
arrives: an unbound promise that nobody executes is the worse outcome.

### Per-PR check

`tests/architectural/test_scenario_contract.py` validates the feature files, the tags and
the step bindings on every pull request. A feature file with no scenario fails it with
`NO_SCENARIOS`, so a misspelt `Scenario:` keyword cannot hide its steps as description
text. Run it locally with:

```bash
uv run --frozen pytest -q tests/architectural/test_scenario_contract.py
```

The gating run has no retries. A flaky scenario is a defect to fix, not to retry
(see [the flakiness policy](/development/testing/testing-flakiness.md)).

## Absorb an engineering test

A test in `tests/integration` or `tests/e2e` that checks something a user can see from the
built CLI can move into a scenario. The rules are in the
[test-pyramid rebalance plan](/development/testing/test-pyramid-rebalance.md#moving-a-test-into-a-scenario); in short:

1. Write an untagged scenario with at least the test's assertions, on an equal or stronger arrange.
2. Run it, then put the defect the test guards back in the source and watch the scenario go red.
   Restore the source.
3. Delete the test in the same pull request, and add a row to
   `tests/scenarios/absorption-ledger.json` naming the retired test, the scenario that replaces it,
   the issue and a note on any difference.

The per-PR check fails when a retired test still exists, a replacement scenario is missing or is
`@wip`/`@manual`, or a file the ledger's baseline lists was deleted without a row. A red gating
night on `main` files one P0 issue through the `acceptance-bdd-escalation` job, as a red
`integration-next` night does.

## Run the scenarios locally

You need JDK 25 and the pinned Gradle. Install Gradle with the shared installer, which
checks the download against its pinned sha256:

```bash
python contracts/tools/install_tools.py --pins acceptance-jvm/tools/pins.json --dest "$HOME/.cache/spec-kitty-tools" --only gradle
export PATH="$HOME/.cache/spec-kitty-tools/gradle-8.14.5/bin:$PATH"
gradle -p acceptance-jvm test --no-daemon -Pacceptance.pass=gating
```

`-Pacceptance.pass=gating` runs `not @wip and not @manual`; `-Pacceptance.pass=wip` runs the
`@wip` scenarios. The property selects the tag filter and the output folder together.
`-Pacceptance.shard=K/N` runs shard `K` of `N` of the pass: the scenarios the tag filter would run,
ordered by feature path and line and dealt out round-robin, so the shards are disjoint and together
run every scenario. The nightly runs the gating pass in three shards on Windows, two on macOS and one
on Linux.

The runner builds a wheel from your checkout with `uv build` and installs it into a
throwaway virtual environment, so it tests your working tree as a user would install
it. Each scenario gets its own temporary directory, home directory and git repository. Scenarios run four at a time
(the runner's `junit-platform.properties`), so a step must never share
mutable state across scenarios. The test task never resolves UP-TO-DATE: the wheel is rebuilt on
every run, so a change under `src/` is always tested.

Each pass writes only under `acceptance-jvm/build/acceptance/<pass>/`, so running `wip`
after `gating` leaves the gating results untouched:

- JUnit XML: `acceptance-jvm/build/acceptance/<pass>/junit/`
- Cucumber JSON: `acceptance-jvm/build/acceptance/<pass>/cucumber.json`
- Allure results: `acceptance-jvm/build/acceptance/<pass>/allure-results/`

The nightly's evidence script also writes `acceptance-jvm/build/acceptance/evidence.json`,
counted from the JUnit XML of the passes. The nightly runs the two passes in separate jobs
(`acceptance-bdd` for gating, `acceptance-bdd-wip` for `@wip`, #7492), so each job's bundle
lists the pass it ran in `passes_run` and reports the other pass as `missing`. It is also given
the outcome of the step that ran each pass, so a run that stopped after writing some passing
JUnit XML is recorded as failed. Only a gating pass that did not pass fails the evidence step.

A gating run fails if no scenario ran, so a mistyped tag filter or a wrong feature path
cannot pass as green. Only the `@wip` run may find none.

Each scenario also checks its own isolation. The CLI must write something under the
redirected home directory, which shows the environment redirection took effect. The real
`~/.spec-kitty` must be left untouched. Lock files are ignored in that comparison,
because another `spec-kitty` process on your machine may touch them.

On a host that runs several Missions at once, export `SPEC_KITTY_ACCEPTANCE_SHARED_HOST=1`
(`1`, `true`, `yes` or `on`) before the run. The comparison then also skips the two
runtime-state subtrees that any concurrent `spec-kitty` process writes: `spec-kitty-prompts/`
(prompt temp files, one directory per repository) and `gate-locks/` (machine-wide
cross-process gate locks). Every other part of the real root stays compared, including `credentials` and any new top-level
file a redirection bug would leave behind. The switch is **ignored on a GitHub Actions runner**,
so the nightly always keeps the whole-tree comparison, and each scenario runs a built-in control
that proves a write outside the two skipped subtrees still fails it — the narrowing can never
quietly become a check that cannot fail. `kitty/acceptance/environment/RealHomeFingerprint.java`
is the single owner of that list; this page only points at it.

## View the Allure report

`acceptance-jvm/build/acceptance/gating/allure-results/` holds raw result JSON, not a report you can open in a
browser. Turn it into the HTML report with the Allure command-line tool. If you have it
installed, `serve` builds the report into a temporary directory and opens it:

```bash
allure serve acceptance-jvm/build/acceptance/gating/allure-results
```

Without a local install, run the same tool through `npx` (it needs a JDK on `PATH`):

```bash
npx allure-commandline serve acceptance-jvm/build/acceptance/gating/allure-results
```

To keep the report rather than serve it from a temporary directory, generate it into a
folder and open that folder:

```bash
allure generate acceptance-jvm/build/acceptance/gating/allure-results --clean -o acceptance-jvm/allure-report
allure open acceptance-jvm/allure-report
```

The nightly does not build this report; it uploads the raw Allure results inside the
`acceptance-bdd-results-<os>-shard-<k>` artifacts, one per shard (the `@wip` pass in
`acceptance-bdd-wip-results-<os>`).
To view a nightly run as a report, download that artifact and run one of these commands on its
`gating/allure-results/` (or `wip/allure-results/`) directory.

## Change a dependency

`acceptance-jvm/gradle/verification-metadata.xml` pins every dependency by sha256. It is
generated, never edited by hand. After changing `acceptance-jvm/build.gradle`, regenerate
it and commit the result:

```bash
gradle -p acceptance-jvm --write-verification-metadata sha256 test --no-daemon
```

`acceptance-jvm` declares no Gradle version of its own and commits no wrapper; the
nightly installs Gradle from `acceptance-jvm/tools/pins.json`. The nightly's pin-guard
step runs `contracts/tools/gradle_pin_check.py`, which for `acceptance-jvm` checks only
that the `url` in `pins.json` carries the pinned `version` (and a `gradle-version:` input
of the `acceptance-bdd` job, if one is ever added). It does not check the plugin or
dependency pins; the sha256 `verification-metadata.xml` covers those.

## Read a nightly failure

The `acceptance-bdd` job runs once per OS and shard. Open the failed leg, then download its
`acceptance-bdd-results-<os>-shard-<k>` artifact. The Cucumber JSON and JUnit XML name the failed
step and carry its assertion message, including the command's output. The spec-kitty-qa
nightly also collects these results.

## Related

- [ADR: acceptance testing and BDD support with Gherkin scenarios and a black-box Cucumber-JVM runner](/adr/4.x/2026-10-11-3-acceptance-testing-and-bdd-support.md)
- [ADR: QA and core share ownership of the acceptance scenarios, and engineering tests move into them only through a checked ledger](/adr/4.x/2026-10-15-1-qa-and-core-share-the-acceptance-scenarios-and-absorb-engineering-tests-through-a-ledger.md)
- [Testing the Spec Kitty codebase](/development/testing/index.md)
- [Test-pyramid rebalance plan](/development/testing/test-pyramid-rebalance.md)
- [Test-flakiness handling policy](/development/testing/testing-flakiness.md)

## Audience

- [Stakeholder Persona: Lead Developer](/context/audience/internal/lead-developer.md) - Written for the Lead Developer reader.
