<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xml:base="https://docs.spec-kitty.ai/adr/4.x/2026-10-08-1-serialise-run-dir-advance/">
  <id>https://docs.spec-kitty.ai/adr/4.x/2026-10-08-1-serialise-run-dir-advance/</id>
  <title>ADR: advancing a mission run is serialised by a per-run-dir lock and a caller-step compare-and-swap</title>
  <updated>2026-10-08T00:00:00.000Z</updated>
  <author><name>Spec Kitty</name></author>
  <link rel="alternate" type="text/html" href="https://docs.spec-kitty.ai/adr/4.x/2026-10-08-1-serialise-run-dir-advance/"/>
  <summary>Concurrent `spec-kitty next` on one run is serialised: a stale advance is refused with a named `blocked` reason, and at most one writer mutates a run cursor at a time.</summary>
  <content type="html">&lt;p&gt;&lt;strong&gt;Status:&lt;/strong&gt; Accepted&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Date:&lt;/strong&gt; 2026-10-08&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Deciders:&lt;/strong&gt; Stijn Dejongh (owner), via the governed mission &lt;code dir=&quot;auto&quot;&gt;serialise-next-advance-01M4EF16&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Technical Story:&lt;/strong&gt; &lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/issues/5854&quot;&gt;#5854&lt;/a&gt; (P2, epic &lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/issues/3897&quot;&gt;#3897&lt;/a&gt;) and &lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/issues/5682&quot;&gt;#5682&lt;/a&gt; (P1). Reuses the &lt;code dir=&quot;auto&quot;&gt;next&lt;/code&gt; contract fixed by ADR &lt;a href=&quot;/adr/2.x/2026-02-17-1-canonical-next-command-runtime-loop/&quot;&gt;2026-02-17-1&lt;/a&gt; and follows the lock pattern the run-INDEX lock established (&lt;code dir=&quot;auto&quot;&gt;src/runtime/next/run_index.py&lt;/code&gt;, #5389/#5390, which shipped without an ADR of its own).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Reader:&lt;/strong&gt; a maintainer who needs to know why a per-run-directory lock and a caller-evaluated-step compare-and-swap guard the &lt;code dir=&quot;auto&quot;&gt;next&lt;/code&gt; advance, which writers they cover, and why a contended or stale advance surfaces as &lt;code dir=&quot;auto&quot;&gt;blocked&lt;/code&gt;.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;context-and-problem-statement&quot;&gt;Context and Problem Statement&lt;/h2&gt;
&lt;p&gt;&lt;code dir=&quot;auto&quot;&gt;spec-kitty next&lt;/code&gt; advances a mission run by reading the run cursor (&lt;code dir=&quot;auto&quot;&gt;state.json&lt;/code&gt;’s &lt;code dir=&quot;auto&quot;&gt;issued_step_id&lt;/code&gt;), evaluating gates against it, then committing the advance. The documented multi-agent loop runs several agents on one mission in parallel, and one agent may retry a slow call — so two advances overlap. Two defects followed, one cause in two layers:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;The advance never recorded which step its caller evaluated.&lt;/strong&gt; The bridge read &lt;code dir=&quot;auto&quot;&gt;issued_step_id&lt;/code&gt; once at bootstrap, but &lt;code dir=&quot;auto&quot;&gt;plan_advance&lt;/code&gt; re-read &lt;code dir=&quot;auto&quot;&gt;state.json&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;commit_advance&lt;/code&gt;’s stale check compared only that &lt;em&gt;second&lt;/em&gt; read against &lt;code dir=&quot;auto&quot;&gt;plan.source&lt;/code&gt;. A peer advance that landed between the bootstrap read and &lt;code dir=&quot;auto&quot;&gt;plan_advance&lt;/code&gt;’s read was invisible, so the caller completed a step it never evaluated — a review step marked done while the WP sat in &lt;code dir=&quot;auto&quot;&gt;for_review&lt;/code&gt;, the loop reporting &lt;code dir=&quot;auto&quot;&gt;terminal&lt;/code&gt; (#5682). The same window hit both the composition and engine advance paths.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;No mutual exclusion covered the read→modify→write of a run cursor.&lt;/strong&gt; &lt;code dir=&quot;auto&quot;&gt;commit_advance&lt;/code&gt; was a compare-then-write TOCTOU; &lt;code dir=&quot;auto&quot;&gt;_write_snapshot&lt;/code&gt; staged through a fixed &lt;code dir=&quot;auto&quot;&gt;state.json.tmp&lt;/code&gt; that two writers collided on; &lt;code dir=&quot;auto&quot;&gt;provide_decision_answer&lt;/code&gt; was a third unguarded writer. On a stale plan the engine path re-planned through &lt;code dir=&quot;auto&quot;&gt;next_step&lt;/code&gt; and re-applied &lt;code dir=&quot;auto&quot;&gt;success&lt;/code&gt;, double-completing a step (#5854), while the composition path already refused.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The single-writer assumption was stated only in a code comment. Nothing enforced it.&lt;/p&gt;
&lt;h2 id=&quot;considered-options&quot;&gt;Considered Options&lt;/h2&gt;
&lt;p&gt;Recorded in the mission’s &lt;code dir=&quot;auto&quot;&gt;research.md&lt;/code&gt; (pre-spec and post-spec adversarial squads). The chosen shape is the synthesis.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Hold one lock across the whole advance (bootstrap read → commit).&lt;/strong&gt; Rejected: the composition advance runs step-contract executors of unbounded duration, so the hold would breach the primitive’s 10 s max-hold default and risk the 60 s stale-reclaim. The architect lens rejected it.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;A compare-and-swap alone, no lock.&lt;/strong&gt; Rejected: the CAS closes the bootstrap→plan &lt;em&gt;staleness&lt;/em&gt; window, but two genuinely concurrent writers can both re-read the same cursor, both pass the CAS, and both append — a true-concurrency TOCTOU only a lock closes. The reviewer lens proved this (reverting the lock while keeping the CAS leaves the race live).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;A new &lt;code dir=&quot;auto&quot;&gt;specify_cli&lt;/code&gt; mission-write lock.&lt;/strong&gt; Rejected: the runtime layer must not import &lt;code dir=&quot;auto&quot;&gt;specify_cli&lt;/code&gt; (layer rules), so PR #5890’s lock is not reusable here.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;A new transient-contention Decision kind / exit code.&lt;/strong&gt; Rejected: it is a &lt;code dir=&quot;auto&quot;&gt;next&lt;/code&gt; contract change with no canonical answer; the existing &lt;code dir=&quot;auto&quot;&gt;blocked&lt;/code&gt; + &lt;code dir=&quot;auto&quot;&gt;reason&lt;/code&gt; (EDGE-003) already carries a refused advance, so contention and staleness reuse it.&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 id=&quot;decision&quot;&gt;Decision&lt;/h2&gt;
&lt;p&gt;Advancing a run is serialised by two complementary mechanisms, both load-bearing:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Caller-step compare-and-swap.&lt;/strong&gt; &lt;code dir=&quot;auto&quot;&gt;AdvancePlan&lt;/code&gt; carries the &lt;code dir=&quot;auto&quot;&gt;issued_step_id&lt;/code&gt; the caller evaluated at bootstrap (&lt;code dir=&quot;auto&quot;&gt;expected_issued_step&lt;/code&gt;), threaded from &lt;code dir=&quot;auto&quot;&gt;plan_advance&lt;/code&gt; through both the composition and engine paths. The one staleness authority — &lt;code dir=&quot;auto&quot;&gt;commit_advance&lt;/code&gt;’s &lt;code dir=&quot;auto&quot;&gt;StaleAdvancePlan&lt;/code&gt;, via &lt;code dir=&quot;auto&quot;&gt;_refuse_stale_plan&lt;/code&gt; — is &lt;strong&gt;widened&lt;/strong&gt; (not duplicated) to also refuse when the live on-disk &lt;code dir=&quot;auto&quot;&gt;issued_step_id&lt;/code&gt; differs from that caller-evaluated step. This closes the bootstrap→plan window (#5682).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Per-run-directory lock.&lt;/strong&gt; A dedicated &lt;code dir=&quot;auto&quot;&gt;&amp;#x3C;run_dir&gt;/state.json.lock&lt;/code&gt; sidecar, constructed through the one canonical primitive &lt;code dir=&quot;auto&quot;&gt;kernel.locks.machine_file_lock&lt;/code&gt; (blocking, 10 s timeout), mirroring &lt;code dir=&quot;auto&quot;&gt;run_index.py&lt;/code&gt;. It is held &lt;strong&gt;only&lt;/strong&gt; around the commit read→validate→append→write in &lt;code dir=&quot;auto&quot;&gt;commit_advance&lt;/code&gt;, around &lt;code dir=&quot;auto&quot;&gt;next_step&lt;/code&gt;’s plan &lt;strong&gt;and&lt;/strong&gt; commit together (one hold, so the plan is fresh by construction and needs no separate staleness check), around the whole of &lt;code dir=&quot;auto&quot;&gt;provide_decision_answer&lt;/code&gt;, and around the retrospective capture + rollback writer. The rollback is conditional: after the speculative commit the bridge records the post-commit &lt;code dir=&quot;auto&quot;&gt;state.json&lt;/code&gt; bytes and event-log size, and under the lock it restores the pre-state only while the live run still equals them; if a peer committed in the gap it logs a warning and leaves the peer’s state alone. It is &lt;strong&gt;never&lt;/strong&gt; held across executor dispatch. Run start (&lt;code dir=&quot;auto&quot;&gt;start_mission_run&lt;/code&gt;) stays lock-free because its run directory is private until the run index publishes it.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Supporting changes: &lt;code dir=&quot;auto&quot;&gt;_write_snapshot&lt;/code&gt; stages through a unique &lt;code dir=&quot;auto&quot;&gt;tempfile.mkstemp&lt;/code&gt; temp (no fixed-name collision); the engine path (&lt;code dir=&quot;auto&quot;&gt;_dn_advance_engine&lt;/code&gt;) refuses a &lt;code dir=&quot;auto&quot;&gt;StaleAdvancePlan&lt;/code&gt; or a &lt;code dir=&quot;auto&quot;&gt;LockAcquireTimeout&lt;/code&gt; with a &lt;code dir=&quot;auto&quot;&gt;blocked&lt;/code&gt; Decision and a named reason — reusing ADR 2026-02-17-1’s contract, with &lt;strong&gt;no&lt;/strong&gt; new Decision kind and &lt;strong&gt;no&lt;/strong&gt; new exit code — and no longer falls back to &lt;code dir=&quot;auto&quot;&gt;next_step&lt;/code&gt; (#5854).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Invariants:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The run-INDEX lock (taken in &lt;code dir=&quot;auto&quot;&gt;get_or_start_run&lt;/code&gt;) is released before any run-cursor lock is acquired; the two are never held nested, so the two locks cannot deadlock.&lt;/li&gt;
&lt;li&gt;The run-cursor lock span never encloses executor dispatch. The hold is bounded by the commit work itself, not by an executor, but it is not always sub-second (see the residual on emitter calls below).&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;consequences&quot;&gt;Consequences&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;A stale or contended advance is refused as &lt;code dir=&quot;auto&quot;&gt;blocked&lt;/code&gt; with a named reason and writes nothing; the loop does not complete an unevaluated step.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Residual (out of scope):&lt;/strong&gt; a refusing stale path can make an outer retry loop spin on &lt;code dir=&quot;auto&quot;&gt;blocked&lt;/code&gt; — the livelock guard tracked by &lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/issues/5112&quot;&gt;#5112&lt;/a&gt;. The CAS guarantees run-state integrity, not executor exactly-once: a peer that landed before the composition executor ran still fired that executor’s side effects.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Residual:&lt;/strong&gt; &lt;code dir=&quot;auto&quot;&gt;next_step&lt;/code&gt; plans and commits under one lock hold, so two concurrent callers serialise and each plans from the state the other left. It carries no caller-evaluated-step CAS: it is reached in production only on the narrow plan-unavailable edge (no previewed step to complete), where there is no evaluated step to compare, so it is not a #5682 exposure.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Residual:&lt;/strong&gt; the lock is held across emitter calls made inside &lt;code dir=&quot;auto&quot;&gt;_commit_advance&lt;/code&gt; (the &lt;code dir=&quot;auto&quot;&gt;DecisionGitLog&lt;/code&gt; emitter makes git commits) and across the &lt;code dir=&quot;auto&quot;&gt;before_run_completed&lt;/code&gt; retrospective capture, so the hold is not always sub-second and a slow git or capture can push a waiting peer toward the 10 s bounded wait, which then surfaces as &lt;code dir=&quot;auto&quot;&gt;blocked&lt;/code&gt; (retryable).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Residual:&lt;/strong&gt; the post-commit state the rollback compares against is read in a separate locked read after the commit’s lock hold ends, so a peer that lands in that narrow gap is recorded as this caller’s own state and would still be rolled back. If that read fails, the rollback is skipped rather than risk clobbering a peer.&lt;/li&gt;
&lt;li&gt;The acceptance set pins each part independently (deterministic, no-sleep): the CAS, the engine refusal, the &lt;code dir=&quot;auto&quot;&gt;provide_decision_answer&lt;/code&gt; guard, the lock (a deterministic held-lock-blocks test plus a barrier true-concurrency proof), and the unique temp — reverting any one part turns at least one test red.&lt;/li&gt;
&lt;/ul&gt;</content>
</entry>
