Contracts
built-output-verifier.md
Contract: Built-Output Verifier
Modules: scripts/docs/seo_verify.py (new), scripts/docs/seo_postprocess.py (modified) Concern: IC-03 Requirements: FR-001, FR-005, FR-008, FR-010, FR-011, FR-012
Asserts against the rendered docs/_site. This is the only layer that can observe a render-path defect — frontmatter can be perfectly correct while the emitted HTML omits the tag entirely, which is exactly today's state for 147 pages.
Part 1 — seo_postprocess.py change
C-B1 — Emit a description tag (FR-005)
Today seo_postprocess.py reads a description (extract_description()) and uses it for Open Graph, Twitter, and structured data — but never writes a <meta name="description">. When DocFX emits none (no frontmatter description), the published page ships with none.
The SEO block gains:
<meta name="description" content="{escaped_desc}">emitted only when the page has no description tag already, so DocFX's own output stays authoritative where present (single canonical authority — the frontmatter is the author's intent; this is a backstop, not an override).
C-B2 — Idempotence
The existing SEO_BLOCK_RE strip-then-reinsert cycle must remain idempotent. Running the post-processor twice produces identical output. A test asserts this directly — the block is delimited by <!-- spec-kitty-seo:start --> / :end, and duplicate injection would silently double every tag.
C-B3 — Fallback stays visible
The boilerplate fallback remains as a last resort but must be detectable so the source gate can flag it (C-S3). Do not make the fallback indistinguishable from an authored description — that would let the backstop mask the defect it exists to reveal.
Part 2 — seo_verify.py (new)
Public surface
python3 scripts/docs/seo_verify.py --site-dir docs/_site [--strict] [--json REPORT]| Flag | Behaviour |
|---|---|
| (none) | Report-only, exit 0 |
--strict | Exit non-zero on any violation |
--json PATH | Write the AuditRecord (FR-001) |
Mirrors the exit contract of description_length_check.py and related_validator.py.
C-B4 — Reuse the indexability predicate (I-08)
Classification imports seo_postprocess.should_index(). It does not reimplement the rule. A second definition of "indexable" would recreate the two-authorities bug this mission exists to fix, one module over.
C-B5 — Rules applied to indexable pages only
| Rule | Assertion |
|---|---|
| V-06 | <meta name="description"> present |
| V-07 | Not the boilerplate fallback |
| V-08 | <link rel="canonical"> equals the page's own canonical address |
| V-09 | og:title matches <title>; og:description matches the description |
| V-10 | Description unique across all indexable pages |
Titles: non-empty and not equal to the bare site default (NFR-001).
C-B6 — Stub and sitemap invariants (FR-012, I-09)
- Every
REDIRECT_STUBcarriesnoindex. - No stub address appears in
sitemap.xml. - Sitemap entries and indexable pages are the same set.
- The verifier never mutates
_site. It is read-only. A tool that can fix what it checks can pass itself.
C-B7 — Deterministic output (I-06)
Violations sorted by path. Two runs over identical input produce byte-identical reports. Follows the inventory lockfile's established convention.
C-B8 — Records the stale-URL finding (FR-011)
The audit report includes a section noting that the two addresses named in issue #1652 are pre-move addresses now served as stubs, with their current addresses verified. This is what lets the issue be closed on evidence rather than assertion.
Part 3 — Workflow integration
C-B9 — Step position is load-bearing (R-009)
docs-pages.yml order becomes:
docfx build
→ seo_postprocess.py (injects SEO; stubs do not exist yet)
→ glossary_linker.py
→ redirect_stub_generator.py generate
→ redirect_stub_generator.py coverage
→ seo_verify.py --strict ← NEW, last
→ upload artifactThe verifier runs last so it observes the final artifact including stubs, and can assert stubs are correctly excluded (C-B6). Placing it before stub generation would leave stub regressions unobserved. The existing ordering comments in the workflow explain why SEO and glossary injection precede stub generation — that ordering is not to be disturbed.
C-B10 — Blocking
Runs with --strict, so a metadata regression fails the build before upload-pages-artifact. A defect must not reach the deployed site.
Test contract (tests/docs/test_seo_verify.py)
| Test | Asserts |
|---|---|
test_missing_description_is_red | Indexable page without the tag → violation (V-06) |
test_boilerplate_description_is_red | Fallback string → violation (V-07) |
test_wrong_canonical_is_red | Canonical pointing elsewhere → violation (V-08) |
test_og_mismatch_is_red | og:description diverging from description → violation (V-09) |
test_duplicate_description_is_red | Two indexable pages sharing a description → both flagged (V-10) |
test_stub_is_not_indexable | Refresh-stub markup classifies as REDIRECT_STUB, rules skipped |
test_stub_absent_from_sitemap | No stub address in the sitemap (C-B6) |
test_verifier_does_not_mutate_site | Input tree byte-identical after a run (C-B6) |
test_clean_site_is_green | Fully compliant fixture → zero violations, exit 0 |
test_strict_exits_nonzero | Exit contract (C-B10) |
test_report_is_deterministic | Two runs byte-identical (C-B7) |
test_postprocess_emits_description | Page with no description tag gains one (C-B1) |
test_postprocess_preserves_existing_description | Existing DocFX description not overwritten (C-B1) |
test_postprocess_is_idempotent | Two passes produce identical output (C-B2) |
All operate on synthetic _site fixtures under tmp_path — no DocFX build required, so these stay in the fast tier despite testing build output.
published-page-set-resolver.md
Contract: Published-Page-Set Resolver
Module: scripts/docs/_published_pages.py (new) Concern: IC-01 Requirements: FR-002, FR-003, FR-013, NFR-005
This module is the single authority for "which source pages are published". It exists because that question currently has two answers — docs/docfx.json and a hardcoded list in tests/docs/test_docs_seo.py — which silently diverged.
Public surface
def resolve_published_pages(
*,
docs_root: Path,
docfx_config: Path | None = None,
) -> PublishedPageSet: ...| Parameter | Meaning |
|---|---|
docs_root | Directory containing the documentation tree |
docfx_config | Path to docfx.json; defaults to docs_root / "docfx.json" |
Returns a PublishedPageSet (see data-model.md).
Raises
| Condition | Behaviour |
|---|---|
docfx.json missing | FileNotFoundError — fail loud. A missing authority must never degrade to "assume everything" or "assume nothing". |
docfx.json unparseable | ValueError naming the parse failure |
| Resolved set is empty | ValueError — violates I-01 |
| Resolved set below floor | ValueError naming observed and expected counts — violates I-02 |
Fail-closed is mandatory. Every error path raises. There is no path on which this function returns a degraded or partial set, because a silently-partial set is the defect under repair.
Behavioural contract
C-R1 — Reads the build's own globs
The build.content[].files patterns are read from docfx.json at call time. They are not duplicated into a module constant. A test asserts that adding a glob to docfx.json changes the resolved set, proving the read is live rather than shadowed.
C-R2 — Honours exclude
docfx.json declares "exclude": ["*/_.md"]. The resolver applies declared excludes; underscore-prefixed pages are not published and must not be gated.
C-R3 — Explicit, reasoned exclusions
Additional exclusions beyond docfx.json's own are enumerated with a reason each (I-04, I-05). At minimum:
| Pattern | Reason |
|---|---|
archive/** | Immutable legacy snapshot; not rewritten for search (C-005) |
kitty-specs/** | Generated mission-run pages; no human author for a description |
Any further exclusion requires a written reason in the same table. An exclusion without a reason is indistinguishable from an oversight.
C-R4 — Non-vacuity floor
A committed floor constant guards against silent under-collection:
MINIMUM_EXPECTED_PAGES: Final[int] = 500Chosen below the measured 674 so ordinary page churn does not cause false failures, and far above the 16 the broken gate resolves, so the current defect would trip it immediately. Raising this constant is a deliberate act; lowering it requires justification.
> Why a floor rather than an exact count: the repository already retired a hardcoded exact ADR census count (_EXPECTED_CENSUS) on the grounds that it "guards little and merely fails on every legitimate add/remove — pure future friction." A floor captures the real invariant (the set must not collapse) without that friction.
C-R5 — Glob-semantics fidelity
DocFX glob semantics are not Python pathlib semantics. DocFX's context/.md matches recursively including the immediate directory; the naive pathlib translation context//*.md does not match context/foo.md.
This is the single highest-risk detail in the mission. Getting it wrong silently under-collects, which is the exact bug being fixed, wearing a new hat.
Mitigation is empirical, not analytical: a test asserts the resolved count is within a tolerance of the observed 674 and that specific known pages — docs/api/slash-commands.md, docs/guides/install-spec-kitty.md, docs/adr/3.x/2026-07-08-1-mission-resolver-port.md — are members. Reasoning about glob semantics is not accepted as proof; membership assertions are.
C-R6 — Performance
O(n) in tree size, one filesystem walk, one read per file. Must leave headroom inside the 30-second gate budget (NFR-007) alongside the consuming checks.
Test contract
| Test | Asserts |
|---|---|
test_resolves_from_docfx_not_a_constant | Adding a glob to a temp docfx.json changes the result (C-R1) |
test_underscore_prefixed_pages_excluded | _draft.md is absent (C-R2) |
test_every_exclusion_carries_a_reason | All Exclusion.reason non-empty (I-05) |
test_empty_resolution_raises | Empty set raises rather than returning (I-01) |
test_below_floor_raises | Under-collection raises, naming both counts (I-02) |
test_missing_docfx_raises | Absent config raises FileNotFoundError |
test_live_tree_membership | The three known pages above are members (C-R5) |
test_live_tree_count_is_realistic | Live count ≥ floor and within tolerance of 674 (C-R5) |
test_would_have_caught_the_original_regression | A page set built from the retired pre-move globs fails the floor — the regression proof |
The last test is the one that matters. It encodes this specific bug so a future reorganisation cannot reproduce it silently.
source-metadata-gate.md
Contract: Source-Level Metadata Gate
Modules: scripts/docs/description_length_check.py (modified), tests/docs/test_docs_seo.py (modified) Concern: IC-02 Requirements: FR-002, FR-003, FR-006, FR-007, NFR-002–NFR-006
Blocks at PR time, runs without .NET. Catches authoring defects before merge; cannot observe the render (that is built-output-verifier.md).
Changes to description_length_check.py
C-S1 — Consume the resolver
Replace the docs_root.rglob("*.md") walk with resolve_published_pages(...). The gate stops guessing which pages are published and asks the authority.
Consequence: pages under docs/plans/, docs/templates/, and other unpublished trees leave the gate's scope. This is correct — publication status is docfx.json's decision. It may reduce the checked count; that reduction is legitimate and must not be confused with the under-collection I-02 guards against.
C-S2 — Retire the ADR exclusion
_EXCLUDE_PREFIXES: Final[tuple[str, ...]] = ("docs/adr/",)Removed. The accompanying comment is corrected, not deleted (DIRECTIVE_037): it currently cites byte-invariance "enforced by test_adr_content_invariance", which that module's own docstring records as retired on 2026-06-29 (ccd278061). Replace with a note stating the rationale expired and descriptions were backfilled by this mission, so the next reader learns the history rather than finding an unexplained deletion.
Hard ordering constraint: this change must not land before IC-04 completes. Removing the exclusion against un-backfilled ADRs turns CI red for 147 files.
C-S3 — Boilerplate detection (net-new, FR-006)
BOILERPLATE_DESCRIPTIONS: Final[frozenset[str]] = frozenset({
"Spec Kitty documentation for CLI workflows, governed missions, "
"AI harnesses, and 3.2 upgrades.",
})A description matching a known fallback is reported as boilerplate, a distinct reason from missing. Distinct reasons matter: "you wrote nothing" and "you inherited the default" call for different author actions.
Single canonical authority: this set must be imported from, or asserted equal to, seo_postprocess.DEFAULT_DESCRIPTION — not retyped. A test pins the two together so changing the fallback string cannot silently disarm the check.
C-S4 — Uniqueness (net-new, FR-007)
After collecting all descriptions, group by exact value; any group of size > 1 yields one violation per member, each naming its peers (I-07).
Comparison is exact-match on the raw string. Normalisation (case, whitespace) is deliberately not applied — two descriptions differing only in case are still duplicates for search purposes, and exact matching keeps the rule explainable.
C-S5 — Coverage assertion (net-new, FR-003, I-01/I-02)
Before validating, assert the resolved page set is non-empty and above floor. A gate that validates zero pages must fail, not pass.
This single assertion is what makes the class of bug under repair unrepresentable.
C-S6 — Preserve the existing exit contract
--strict exits non-zero on violations; report-only exits 0. Matches related_validator.py. docs-freshness.yml already invokes with --strict; that invocation is unchanged.
C-S7 — Preserve the band
MIN_DESCRIPTION_LENGTH = 50, MAX_DESCRIPTION_LENGTH = 180, inclusive (C-003). Untouched.
Changes to tests/docs/test_docs_seo.py
C-S8 — Delete the hardcoded globs
_published_markdown_files()'s ten-pattern list is removed and replaced by a call to the resolver. This is the direct fix for the 2.4%-coverage defect.
C-S9 — Keep parametrisation
The per-file parametrised shape is retained so a failure names the offending page. Scaling from 16 to ~674 parametrised cases must stay inside the 30-second budget (NFR-007); if it does not, collapse to a single test emitting all violations at once rather than relaxing the budget.
Test contract (NFR-006 — the gate must be provably able to fail)
Extends the existing boundary-proof precedent in test_description_length_gate.py, whose docstring already states the principle: "A length gate that cannot go RED is fake, so the Definition of Done is the boundary proof."
| Test | Asserts |
|---|---|
test_missing_description_is_red | Absent description → violation, reason missing |
test_49_and_181_are_red | Existing boundary proof preserved |
test_50_and_180_are_green | Existing boundary proof preserved |
test_boilerplate_description_is_red | Exact fallback string → reason boilerplate (C-S3) |
test_boilerplate_set_matches_seo_postprocess | Constant pinned to the render-side fallback (C-S3) |
test_duplicate_descriptions_are_red | Two pages, same description → both flagged |
test_duplicate_violation_names_the_peer | Violation carries the colliding path (I-07) |
test_empty_page_set_is_red | Zero resolved pages → failure, not pass (C-S5) |
test_adr_pages_are_now_in_scope | An ADR without a description is flagged (C-S2) |
test_strict_exits_nonzero / test_report_only_exits_zero | Exit contract preserved (C-S6) |
test_live_tree_is_clean | Post-backfill, the real tree yields zero violations |
test_live_tree_is_clean is the acceptance test for IC-04 and will be red until the backfill completes. That is intended and is the red-first signal.