<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xml:base="https://docs.spec-kitty.ai/development/testing/acceptance-scenarios/">
  <id>https://docs.spec-kitty.ai/development/testing/acceptance-scenarios/</id>
  <title>CLI Acceptance Scenarios (Cucumber-JVM nightly)</title>
  <updated>2026-11-02T00:00:00.000Z</updated>
  <author><name>Spec Kitty</name></author>
  <link rel="alternate" type="text/html" href="https://docs.spec-kitty.ai/development/testing/acceptance-scenarios/"/>
  <summary>How to write, run and read the Gherkin acceptance scenarios that check the built spec-kitty CLI every night on Linux, macOS and Windows.</summary>
  <content type="html">&lt;span id=&quot;cli-acceptance-scenarios-cucumber-jvm-nightly&quot;&gt;&lt;/span&gt;
&lt;p&gt;The acceptance scenarios check what the &lt;code dir=&quot;auto&quot;&gt;spec-kitty&lt;/code&gt; CLI promises its users, from the
outside. They run the CLI built from the commit as a separate process and look only at
what a user would see: files, exit codes and git state. A broken promise turns the
nightly red on every supported operating system.&lt;/p&gt;





























&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Part&lt;/th&gt;&lt;th&gt;Location&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;Scenarios (Gherkin)&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;tests/scenarios/&amp;#x3C;area&gt;/*.feature&lt;/code&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Runner, step definitions, driver&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/&lt;/code&gt; (Gradle, Cucumber-JVM 7, Java 25)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Nightly job&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;acceptance-bdd&lt;/code&gt; in &lt;code dir=&quot;auto&quot;&gt;.github/workflows/ci-nightly.yml&lt;/code&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Decision records&lt;/td&gt;&lt;td&gt;ADR &lt;code dir=&quot;auto&quot;&gt;2026-10-11-3&lt;/code&gt; (acceptance testing and BDD support), ADR &lt;code dir=&quot;auto&quot;&gt;2026-10-13-1&lt;/code&gt; (promise links, per-PR check, evidence), ADR &lt;code dir=&quot;auto&quot;&gt;2026-10-15-1&lt;/code&gt; (ownership and absorption)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Absorbed engineering tests&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;tests/scenarios/absorption-ledger.json&lt;/code&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;h2 id=&quot;who-owns-what&quot;&gt;Who owns what&lt;/h2&gt;

























&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Part&lt;/th&gt;&lt;th&gt;Owners&lt;/th&gt;&lt;th&gt;Review&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;Scenarios, the absorption ledger, this guide&lt;/td&gt;&lt;td&gt;QA (spec-kitty-qa) and core&lt;/td&gt;&lt;td&gt;a scenario change asks a QA owner and a core owner (&lt;code dir=&quot;auto&quot;&gt;.github/CODEOWNERS&lt;/code&gt;)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Runner, step definitions, driver, fixture builders (&lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/&lt;/code&gt;)&lt;/td&gt;&lt;td&gt;core&lt;/td&gt;&lt;td&gt;core&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Promise pages, promise index, trace lint, Allure report&lt;/td&gt;&lt;td&gt;spec-kitty-qa&lt;/td&gt;&lt;td&gt;spec-kitty-qa&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;QA drafts a scenario from a promise and lands it as &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt;; an engineer binds its steps and adds
a positive control; &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt; comes off in a pull request a QA owner and a core owner reviewed.
&lt;code dir=&quot;auto&quot;&gt;tests/architectural/test_scenario_contract.py&lt;/code&gt; checks the CODEOWNERS rules on every pull
request. Requiring code-owner review on GitHub is a repository setting, not set today.&lt;/p&gt;
&lt;h2 id=&quot;write-a-scenario&quot;&gt;Write a scenario&lt;/h2&gt;
&lt;ol&gt;
&lt;li&gt;Pick a promise the CLI makes to its users, such as a sentence from a command’s
&lt;code dir=&quot;auto&quot;&gt;--help&lt;/code&gt; text. Name it in the scenario file’s maintainer notes.&lt;/li&gt;
&lt;li&gt;Write the scenario from the user’s point of view: what they have, what they run,
what they get. Leave test mechanics out of the &lt;code dir=&quot;auto&quot;&gt;Feature&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;Scenario&lt;/code&gt; text.&lt;/li&gt;
&lt;li&gt;Order the steps Arrange, Assume, Act, Assert (see below), and reuse existing steps
before adding new ones.&lt;/li&gt;
&lt;li&gt;Give the scenario a same-fixture positive control, so a command that did nothing
cannot pass it. For example, “the folder &lt;code dir=&quot;auto&quot;&gt;.claude/skills&lt;/code&gt; exists” proves that &lt;code dir=&quot;auto&quot;&gt;init&lt;/code&gt;
actually ran before “the file &lt;code dir=&quot;auto&quot;&gt;.kittify/config.yaml&lt;/code&gt; is unchanged” can mean anything.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;code dir=&quot;auto&quot;&gt;tests/scenarios/README.md&lt;/code&gt; lists the available steps.&lt;/p&gt;
&lt;h2 id=&quot;step-structure-arrange-assume-act-assert&quot;&gt;Step structure: Arrange, Assume, Act, Assert&lt;/h2&gt;
&lt;p&gt;Each lifecycle group has its own step class in the &lt;code dir=&quot;auto&quot;&gt;kitty.acceptance.steps&lt;/code&gt;
package of the &lt;code dir=&quot;auto&quot;&gt;acceptance-jvm&lt;/code&gt; test sources. They share one
&lt;code dir=&quot;auto&quot;&gt;ScenarioContext&lt;/code&gt; per scenario, injected by Cucumber’s PicoContainer integration.&lt;/p&gt;



































&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Group&lt;/th&gt;&lt;th&gt;Gherkin&lt;/th&gt;&lt;th&gt;Class&lt;/th&gt;&lt;th&gt;Does&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;Arrange&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;Given&lt;/code&gt;&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;ArrangeSteps&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Builds the starting conditions and fails fast when that did not work.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Assume&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;Given&lt;/code&gt; / &lt;code dir=&quot;auto&quot;&gt;And&lt;/code&gt;&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;AssumeSteps&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Checks that the starting state is what the scenario expects, and records it for later comparison.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Act&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;When&lt;/code&gt;&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;ActSteps&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Runs the command under test.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Assert&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;Then&lt;/code&gt; / &lt;code dir=&quot;auto&quot;&gt;And&lt;/code&gt;&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;AssertSteps&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Checks the post-conditions.&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Steps talk to the CLI only through the &lt;code dir=&quot;auto&quot;&gt;SpecKittyDriver&lt;/code&gt; port and never import Python
code. Assertions use AssertJ.&lt;/p&gt;
&lt;h3 id=&quot;the-one-arrange-write-that-is-not-the-clis&quot;&gt;The one Arrange write that is not the CLI’s&lt;/h3&gt;
&lt;p&gt;ADR 2026-10-15-1 decision 6 lists what an Arrange builder may write: the planning files
an agent authors, the code a work package’s implementer writes, a reviewer’s feedback file
outside the repository, and plain files a user edits in git. It never writes CLI-managed
state.&lt;/p&gt;
&lt;p&gt;There is one narrow, documented exception, for a promise whose subject &lt;em&gt;is&lt;/em&gt; the state the
CLI reduces: a scenario that checks how the status log’s ordering is honoured may plant a
row as a teammate’s &lt;code dir=&quot;auto&quot;&gt;git pull&lt;/code&gt; delivers it. &lt;code dir=&quot;auto&quot;&gt;MissionLifecycle.plantFutureStampedApproval&lt;/code&gt;
restamps a work package’s &lt;code dir=&quot;auto&quot;&gt;approved&lt;/code&gt; row to a fixed ten minutes into the future and commits
it, so the reducer sees a clock that ran ahead of the rejecting reviewer’s. The row keeps
exactly the shape the CLI writes (no &lt;code dir=&quot;auto&quot;&gt;lamport_clock&lt;/code&gt; on lane transitions), and everything
after it goes through the CLI. &lt;code dir=&quot;auto&quot;&gt;tests/scenarios/status/event-traceability.feature&lt;/code&gt; is the
only user, added for &lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/issues/6260&quot;&gt;#6260&lt;/a&gt;; do not
copy the pattern for a behaviour a real CLI call can arrange.&lt;/p&gt;
&lt;h3 id=&quot;builders-name-the-topology-they-mean&quot;&gt;Builders name the topology they mean&lt;/h3&gt;
&lt;p&gt;Since &lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/pull/7282&quot;&gt;#7282&lt;/a&gt; (ADR 2026-11-06-3),
&lt;code dir=&quot;auto&quot;&gt;agent mission create&lt;/code&gt; without &lt;code dir=&quot;auto&quot;&gt;--topology&lt;/code&gt; records an unsettled &lt;code dir=&quot;auto&quot;&gt;single_branch&lt;/code&gt; Mission: it
switches the checkout onto a Mission topic branch and makes no coordination branch and no lane
worktree. A builder that means a coordination Mission therefore asks for one:
&lt;code dir=&quot;auto&quot;&gt;MissionLifecycle.planMission&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;createMission&lt;/code&gt;, and &lt;code dir=&quot;auto&quot;&gt;MissionFixture.createMission&lt;/code&gt;, pass
&lt;code dir=&quot;auto&quot;&gt;--topology lanes_with_coord&lt;/code&gt; (&lt;code dir=&quot;auto&quot;&gt;MissionLifecycle.COORDINATION_TOPOLOGY&lt;/code&gt;), and
&lt;code dir=&quot;auto&quot;&gt;planSingleBranchMission&lt;/code&gt; passes &lt;code dir=&quot;auto&quot;&gt;--topology single_branch&lt;/code&gt;. A scenario about the size default
itself must call &lt;code dir=&quot;auto&quot;&gt;mission create&lt;/code&gt; with no &lt;code dir=&quot;auto&quot;&gt;--topology&lt;/code&gt; explicitly, and must commit or discard each
Mission’s &lt;code dir=&quot;auto&quot;&gt;spec.md&lt;/code&gt; scaffold before the next create: a default create refuses a dirty checkout
with &lt;code dir=&quot;auto&quot;&gt;WRITE_CHECKOUT_DIRTY&lt;/code&gt;, by design, so one Mission’s untracked scaffold never follows the
checkout onto another Mission’s branch. A Python acceptance helper
that pins &lt;code dir=&quot;auto&quot;&gt;SPEC_KITTY_HOME&lt;/code&gt; puts the home beside the project checkout, never inside it: cache
files written there would dirty the write checkout, and the Mission-branch switch refuses a
dirty one (&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/issues/7342&quot;&gt;#7342&lt;/a&gt;).&lt;/p&gt;
&lt;h2 id=&quot;tags&quot;&gt;Tags&lt;/h2&gt;













































&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Tag&lt;/th&gt;&lt;th&gt;Rule&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;@promise:&amp;#x3C;id&gt;&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Names the promise the scenario checks. &lt;code dir=&quot;auto&quot;&gt;&amp;#x3C;id&gt;&lt;/code&gt; matches &lt;code dir=&quot;auto&quot;&gt;[A-Za-z0-9][A-Za-z0-9._-]*&lt;/code&gt;, for example &lt;code dir=&quot;auto&quot;&gt;PR-CLI-INIT-001&lt;/code&gt;. Not repeated on one scenario.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Draft or under repair. Needs &lt;code dir=&quot;auto&quot;&gt;@wip-owner:&amp;#x3C;handle&gt;&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;@wip-since:&amp;#x3C;YYYY-MM-DD&gt;&lt;/code&gt;. Runs in the separate non-gating &lt;code dir=&quot;auto&quot;&gt;acceptance-bdd-wip&lt;/code&gt; nightly job, which never fails a leg and nothing waits on (#7492).&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;@wip-owner:&amp;#x3C;handle&gt;&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Owner of the draft. &lt;code dir=&quot;auto&quot;&gt;&amp;#x3C;handle&gt;&lt;/code&gt; matches &lt;code dir=&quot;auto&quot;&gt;[A-Za-z0-9][A-Za-z0-9._-]*&lt;/code&gt;.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;@wip-since:&amp;#x3C;YYYY-MM-DD&gt;&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Date the scenario was marked &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt;.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;@manual&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Never runs automatically.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;@defect:&amp;#x3C;issue&gt;&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Links a known defect. &lt;code dir=&quot;auto&quot;&gt;&amp;#x3C;issue&gt;&lt;/code&gt; is a positive integer.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;@qa:&amp;#x3C;ref&gt;&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Reference for the QA repository. &lt;code dir=&quot;auto&quot;&gt;&amp;#x3C;ref&gt;&lt;/code&gt; uses the identifier character set.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;@&amp;#x3C;bare&gt;&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Bare tag matching &lt;code dir=&quot;auto&quot;&gt;[A-Za-z0-9][A-Za-z0-9_-]*&lt;/code&gt;, with no &lt;code dir=&quot;auto&quot;&gt;:&lt;/code&gt;.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;none&lt;/td&gt;&lt;td&gt;Runs in the gating nightly. A failure fails that OS leg.&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Any other tag containing &lt;code dir=&quot;auto&quot;&gt;:&lt;/code&gt; is an error. Tags on a &lt;code dir=&quot;auto&quot;&gt;Feature&lt;/code&gt; or &lt;code dir=&quot;auto&quot;&gt;Rule&lt;/code&gt; apply to every
scenario under it, and tags on an &lt;code dir=&quot;auto&quot;&gt;Examples:&lt;/code&gt; block apply to the rows of that table. Everything except &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;@manual&lt;/code&gt; gates the nightly.&lt;/p&gt;
&lt;h3 id=&quot;linking-a-scenario-to-a-promise&quot;&gt;Linking a scenario to a promise&lt;/h3&gt;
&lt;p&gt;A scenario links to a promise by identifier only, with &lt;code dir=&quot;auto&quot;&gt;@promise:&amp;#x3C;id&gt;&lt;/code&gt;. The scenario
carries no promise wording, rationale or status. This repository checks the shape of the
identifier and nothing more; resolving it to a promise happens in the QA repository.&lt;/p&gt;
&lt;h3 id=&quot;drafted-scenarios&quot;&gt;Drafted scenarios&lt;/h3&gt;
&lt;p&gt;A scenario drafted from a promise page lands as &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt;, with its &lt;code dir=&quot;auto&quot;&gt;@promise:&amp;#x3C;id&gt;&lt;/code&gt; tag,
&lt;code dir=&quot;auto&quot;&gt;@wip-owner&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;@wip-since&lt;/code&gt;. It may use steps that have no binding yet. An engineer
then binds the steps, adds a positive control and removes &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt;. Tooling never invents
an identifier.&lt;/p&gt;
&lt;p&gt;Removing &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt; and adding &lt;code dir=&quot;auto&quot;&gt;@promise:&amp;#x3C;id&gt;&lt;/code&gt; are separate steps. The engineer’s pull
request binds the steps, adds the positive control and drops &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt;; the identifier comes
from the QA promise registry, so if it is not assigned yet the scenario gates without it
(the contract check validates the identifier’s shape only when the tag is present) and a
QA owner adds the tag in review. Never hold a bound scenario in &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt; until an identifier
arrives: an unbound promise that nobody executes is the worse outcome.&lt;/p&gt;
&lt;h3 id=&quot;per-pr-check&quot;&gt;Per-PR check&lt;/h3&gt;
&lt;p&gt;&lt;code dir=&quot;auto&quot;&gt;tests/architectural/test_scenario_contract.py&lt;/code&gt; validates the feature files, the tags and
the step bindings on every pull request. A feature file with no scenario fails it with
&lt;code dir=&quot;auto&quot;&gt;NO_SCENARIOS&lt;/code&gt;, so a misspelt &lt;code dir=&quot;auto&quot;&gt;Scenario:&lt;/code&gt; keyword cannot hide its steps as description
text. Run it locally with:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--frozen&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;pytest&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;-q&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;tests/architectural/test_scenario_contract.py&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;The gating run has no retries. A flaky scenario is a defect to fix, not to retry
(see &lt;a href=&quot;/development/testing/testing-flakiness/&quot;&gt;the flakiness policy&lt;/a&gt;).&lt;/p&gt;
&lt;h2 id=&quot;absorb-an-engineering-test&quot;&gt;Absorb an engineering test&lt;/h2&gt;
&lt;p&gt;A test in &lt;code dir=&quot;auto&quot;&gt;tests/integration&lt;/code&gt; or &lt;code dir=&quot;auto&quot;&gt;tests/e2e&lt;/code&gt; that checks something a user can see from the
built CLI can move into a scenario. The rules are in the
&lt;a href=&quot;/development/testing/test-pyramid-rebalance/#moving-a-test-into-a-scenario&quot;&gt;test-pyramid rebalance plan&lt;/a&gt;; in short:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Write an untagged scenario with at least the test’s assertions, on an equal or stronger arrange.&lt;/li&gt;
&lt;li&gt;Run it, then put the defect the test guards back in the source and watch the scenario go red.
Restore the source.&lt;/li&gt;
&lt;li&gt;Delete the test in the same pull request, and add a row to
&lt;code dir=&quot;auto&quot;&gt;tests/scenarios/absorption-ledger.json&lt;/code&gt; naming the retired test, the scenario that replaces it,
the issue and a note on any difference.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The per-PR check fails when a retired test still exists, a replacement scenario is missing or is
&lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt;/&lt;code dir=&quot;auto&quot;&gt;@manual&lt;/code&gt;, or a file the ledger’s baseline lists was deleted without a row. A red gating
night on &lt;code dir=&quot;auto&quot;&gt;main&lt;/code&gt; files one P0 issue through the &lt;code dir=&quot;auto&quot;&gt;acceptance-bdd-escalation&lt;/code&gt; job, as a red
&lt;code dir=&quot;auto&quot;&gt;integration-next&lt;/code&gt; night does.&lt;/p&gt;
&lt;h2 id=&quot;run-the-scenarios-locally&quot;&gt;Run the scenarios locally&lt;/h2&gt;
&lt;p&gt;You need JDK 25 and the pinned Gradle. Install Gradle with the shared installer, which
checks the download against its pinned sha256:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;python&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;contracts/tools/install_tools.py&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--pins&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;acceptance-jvm/tools/pins.json&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--dest&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;$HOME&lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#984E4D&quot;&gt;/.cache/spec-kitty-tools&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--only&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;gradle&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;export&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;PATH&lt;/span&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;$HOME&lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#984E4D&quot;&gt;/.cache/spec-kitty-tools/gradle-8.14.5/bin:&lt;/span&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;$PATH&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;gradle&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;-p&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;acceptance-jvm&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;test&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--no-daemon&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;-Pacceptance.pass=gating&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;&lt;code dir=&quot;auto&quot;&gt;-Pacceptance.pass=gating&lt;/code&gt; runs &lt;code dir=&quot;auto&quot;&gt;not @wip and not @manual&lt;/code&gt;; &lt;code dir=&quot;auto&quot;&gt;-Pacceptance.pass=wip&lt;/code&gt; runs the
&lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt; scenarios. The property selects the tag filter and the output folder together.
&lt;code dir=&quot;auto&quot;&gt;-Pacceptance.shard=K/N&lt;/code&gt; runs shard &lt;code dir=&quot;auto&quot;&gt;K&lt;/code&gt; of &lt;code dir=&quot;auto&quot;&gt;N&lt;/code&gt; of the pass: the scenarios the tag filter would run,
ordered by feature path and line and dealt out round-robin, so the shards are disjoint and together
run every scenario. The nightly runs the gating pass in three shards on Windows, two on macOS and one
on Linux.&lt;/p&gt;
&lt;p&gt;The runner builds a wheel from your checkout with &lt;code dir=&quot;auto&quot;&gt;uv build&lt;/code&gt; and installs it into a
throwaway virtual environment, so it tests your working tree as a user would install
it. Each scenario gets its own temporary directory, home directory and git repository. Scenarios run four at a time
(the runner’s &lt;code dir=&quot;auto&quot;&gt;junit-platform.properties&lt;/code&gt;), so a step must never share
mutable state across scenarios. The test task never resolves UP-TO-DATE: the wheel is rebuilt on
every run, so a change under &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt; is always tested.&lt;/p&gt;
&lt;p&gt;Each pass writes only under &lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/build/acceptance/&amp;#x3C;pass&gt;/&lt;/code&gt;, so running &lt;code dir=&quot;auto&quot;&gt;wip&lt;/code&gt;
after &lt;code dir=&quot;auto&quot;&gt;gating&lt;/code&gt; leaves the gating results untouched:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;JUnit XML: &lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/build/acceptance/&amp;#x3C;pass&gt;/junit/&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Cucumber JSON: &lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/build/acceptance/&amp;#x3C;pass&gt;/cucumber.json&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Allure results: &lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/build/acceptance/&amp;#x3C;pass&gt;/allure-results/&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The nightly’s evidence script also writes &lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/build/acceptance/evidence.json&lt;/code&gt;,
counted from the JUnit XML of the passes. The nightly runs the two passes in separate jobs
(&lt;code dir=&quot;auto&quot;&gt;acceptance-bdd&lt;/code&gt; for gating, &lt;code dir=&quot;auto&quot;&gt;acceptance-bdd-wip&lt;/code&gt; for &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt;, #7492), so each job’s bundle
lists the pass it ran in &lt;code dir=&quot;auto&quot;&gt;passes_run&lt;/code&gt; and reports the other pass as &lt;code dir=&quot;auto&quot;&gt;missing&lt;/code&gt;. It is also given
the outcome of the step that ran each pass, so a run that stopped after writing some passing
JUnit XML is recorded as failed. Only a gating pass that did not pass fails the evidence step.&lt;/p&gt;
&lt;p&gt;A gating run fails if no scenario ran, so a mistyped tag filter or a wrong feature path
cannot pass as green. Only the &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt; run may find none.&lt;/p&gt;
&lt;p&gt;Each scenario also checks its own isolation. The CLI must write something under the
redirected home directory, which shows the environment redirection took effect. The real
&lt;code dir=&quot;auto&quot;&gt;~/.spec-kitty&lt;/code&gt; must be left untouched. Lock files are ignored in that comparison,
because another &lt;code dir=&quot;auto&quot;&gt;spec-kitty&lt;/code&gt; process on your machine may touch them.&lt;/p&gt;
&lt;p&gt;On a host that runs several Missions at once, export &lt;code dir=&quot;auto&quot;&gt;SPEC_KITTY_ACCEPTANCE_SHARED_HOST=1&lt;/code&gt;
(&lt;code dir=&quot;auto&quot;&gt;1&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;true&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;yes&lt;/code&gt; or &lt;code dir=&quot;auto&quot;&gt;on&lt;/code&gt;) before the run. The comparison then also skips the two
runtime-state subtrees that any concurrent &lt;code dir=&quot;auto&quot;&gt;spec-kitty&lt;/code&gt; process writes: &lt;code dir=&quot;auto&quot;&gt;spec-kitty-prompts/&lt;/code&gt;
(prompt temp files, one directory per repository) and &lt;code dir=&quot;auto&quot;&gt;gate-locks/&lt;/code&gt; (machine-wide
cross-process gate locks). Every other part of the real root stays compared, including &lt;code dir=&quot;auto&quot;&gt;credentials&lt;/code&gt; and any new top-level
file a redirection bug would leave behind. The switch is &lt;strong&gt;ignored on a GitHub Actions runner&lt;/strong&gt;,
so the nightly always keeps the whole-tree comparison, and each scenario runs a built-in control
that proves a write outside the two skipped subtrees still fails it — the narrowing can never
quietly become a check that cannot fail. &lt;code dir=&quot;auto&quot;&gt;kitty/acceptance/environment/RealHomeFingerprint.java&lt;/code&gt;
is the single owner of that list; this page only points at it.&lt;/p&gt;
&lt;h2 id=&quot;view-the-allure-report&quot;&gt;View the Allure report&lt;/h2&gt;
&lt;p&gt;&lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/build/acceptance/gating/allure-results/&lt;/code&gt; holds raw result JSON, not a report you can open in a
browser. Turn it into the HTML report with the Allure command-line tool. If you have it
installed, &lt;code dir=&quot;auto&quot;&gt;serve&lt;/code&gt; builds the report into a temporary directory and opens it:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;allure&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;serve&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;acceptance-jvm/build/acceptance/gating/allure-results&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;Without a local install, run the same tool through &lt;code dir=&quot;auto&quot;&gt;npx&lt;/code&gt; (it needs a JDK on &lt;code dir=&quot;auto&quot;&gt;PATH&lt;/code&gt;):&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;npx&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;allure-commandline&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;serve&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;acceptance-jvm/build/acceptance/gating/allure-results&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;To keep the report rather than serve it from a temporary directory, generate it into a
folder and open that folder:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;allure&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;generate&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;acceptance-jvm/build/acceptance/gating/allure-results&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--clean&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;-o&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;acceptance-jvm/allure-report&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;allure&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;open&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;acceptance-jvm/allure-report&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;The nightly does not build this report; it uploads the raw Allure results inside the
&lt;code dir=&quot;auto&quot;&gt;acceptance-bdd-results-&amp;#x3C;os&gt;-shard-&amp;#x3C;k&gt;&lt;/code&gt; artifacts, one per shard (the &lt;code dir=&quot;auto&quot;&gt;@wip&lt;/code&gt; pass in
&lt;code dir=&quot;auto&quot;&gt;acceptance-bdd-wip-results-&amp;#x3C;os&gt;&lt;/code&gt;).
To view a nightly run as a report, download that artifact and run one of these commands on its
&lt;code dir=&quot;auto&quot;&gt;gating/allure-results/&lt;/code&gt; (or &lt;code dir=&quot;auto&quot;&gt;wip/allure-results/&lt;/code&gt;) directory.&lt;/p&gt;
&lt;h2 id=&quot;change-a-dependency&quot;&gt;Change a dependency&lt;/h2&gt;
&lt;p&gt;&lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/gradle/verification-metadata.xml&lt;/code&gt; pins every dependency by sha256. It is
generated, never edited by hand. After changing &lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/build.gradle&lt;/code&gt;, regenerate
it and commit the result:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;gradle&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;-p&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;acceptance-jvm&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--write-verification-metadata&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;sha256&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;test&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--no-daemon&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;&lt;code dir=&quot;auto&quot;&gt;acceptance-jvm&lt;/code&gt; declares no Gradle version of its own and commits no wrapper; the
nightly installs Gradle from &lt;code dir=&quot;auto&quot;&gt;acceptance-jvm/tools/pins.json&lt;/code&gt;. The nightly’s pin-guard
step runs &lt;code dir=&quot;auto&quot;&gt;contracts/tools/gradle_pin_check.py&lt;/code&gt;, which for &lt;code dir=&quot;auto&quot;&gt;acceptance-jvm&lt;/code&gt; checks only
that the &lt;code dir=&quot;auto&quot;&gt;url&lt;/code&gt; in &lt;code dir=&quot;auto&quot;&gt;pins.json&lt;/code&gt; carries the pinned &lt;code dir=&quot;auto&quot;&gt;version&lt;/code&gt; (and a &lt;code dir=&quot;auto&quot;&gt;gradle-version:&lt;/code&gt; input
of the &lt;code dir=&quot;auto&quot;&gt;acceptance-bdd&lt;/code&gt; job, if one is ever added). It does not check the plugin or
dependency pins; the sha256 &lt;code dir=&quot;auto&quot;&gt;verification-metadata.xml&lt;/code&gt; covers those.&lt;/p&gt;
&lt;h2 id=&quot;read-a-nightly-failure&quot;&gt;Read a nightly failure&lt;/h2&gt;
&lt;p&gt;The &lt;code dir=&quot;auto&quot;&gt;acceptance-bdd&lt;/code&gt; job runs once per OS and shard. Open the failed leg, then download its
&lt;code dir=&quot;auto&quot;&gt;acceptance-bdd-results-&amp;#x3C;os&gt;-shard-&amp;#x3C;k&gt;&lt;/code&gt; artifact. The Cucumber JSON and JUnit XML name the failed
step and carry its assertion message, including the command’s output. The spec-kitty-qa
nightly also collects these results.&lt;/p&gt;</content>
</entry>
