<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xml:base="https://docs.spec-kitty.ai/development/how-to/consumer-census/">
  <id>https://docs.spec-kitty.ai/development/how-to/consumer-census/</id>
  <title>Consumer census: keep CI fan-out honest</title>
  <updated>2026-10-19T00:00:00.000Z</updated>
  <author><name>Spec Kitty</name></author>
  <link rel="alternate" type="text/html" href="https://docs.spec-kitty.ai/development/how-to/consumer-census/"/>
  <summary>What the committed consumer census is, when its CI gates go red, how to regenerate it with --write or --check, and how to fix an unrouted consumer.</summary>
  <content type="html">&lt;span id=&quot;consumer-census-keep-ci-fan-out-honest&quot;&gt;&lt;/span&gt;
&lt;p&gt;The consumer census, &lt;code dir=&quot;auto&quot;&gt;.github/ci-consumer-census.json&lt;/code&gt;, records which test module
consumes which &lt;code dir=&quot;auto&quot;&gt;src/**&lt;/code&gt; or &lt;code dir=&quot;auto&quot;&gt;packs/**&lt;/code&gt; path. It is built by
&lt;code dir=&quot;auto&quot;&gt;scripts/ci/consumer_census.py&lt;/code&gt; from two sources: a static scan of test imports and
path literals, and a replay of a fixed set of resolver surfaces under an audit hook
(so reads a grep cannot see, such as a mission-step prompt resolved through the CLI,
are attributed too). A reader that resolves pack content outside those surfaces is
not seen (tracked in #6214).&lt;/p&gt;
&lt;p&gt;The census checks that a change to shipped pack content selects the modules whose
tests read it (#6166, #6173). &lt;code dir=&quot;auto&quot;&gt;packs/built-in/&lt;/code&gt; is fetched from a pinned commit and is
not tracked (#6451), so pack content changes in a pull request only through the pin
lockfile &lt;code dir=&quot;auto&quot;&gt;packs/built-in.source.yaml&lt;/code&gt;. Each pack fan-out group in
&lt;code dir=&quot;auto&quot;&gt;.github/workflows/ci-router.yml&lt;/code&gt; is triggered by that pin alone. The pack sub-trees each
group answers for are listed in &lt;code dir=&quot;auto&quot;&gt;PACK_CONTENT_SCOPES&lt;/code&gt; (&lt;code dir=&quot;auto&quot;&gt;scripts/ci/gate_selection.py&lt;/code&gt;),
which the census uses to assign a consumed path to its group (#6480).&lt;/p&gt;
&lt;p&gt;The census build and its tests read the synced tree, so run &lt;code dir=&quot;auto&quot;&gt;make sync-packs&lt;/code&gt; first.&lt;/p&gt;
&lt;h2 id=&quot;when-it-goes-red&quot;&gt;When it goes red&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Stale census.&lt;/strong&gt; The &lt;code dir=&quot;auto&quot;&gt;architectural-fast&lt;/code&gt; job runs &lt;code dir=&quot;auto&quot;&gt;consumer_census.py --check&lt;/code&gt; on
every PR. It fails when a new or changed consumer is not reflected in the
committed file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Unrouted consumer.&lt;/strong&gt; &lt;code dir=&quot;auto&quot;&gt;tests/architectural/test_consumer_census_routing.py&lt;/code&gt; fails
when the census records a pack path that no fan-out group glob or &lt;code dir=&quot;auto&quot;&gt;PACK_CONTENT_SCOPES&lt;/code&gt;
entry matches. The same file fails when a pack group’s router filter is anything but
the pin, or when a &lt;code dir=&quot;auto&quot;&gt;PACK_CONTENT_SCOPES&lt;/code&gt; glob matches no file of the synced tree.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Module missing from its fan-out group.&lt;/strong&gt; &lt;code dir=&quot;auto&quot;&gt;tests/ci/test_fanout_derivation.py&lt;/code&gt;
fails when the census derives a module for a group but the module is in neither
&lt;code dir=&quot;auto&quot;&gt;GROUP_MODULE_FANOUT&lt;/code&gt; nor &lt;code dir=&quot;auto&quot;&gt;FANOUT_ALLOWLIST&lt;/code&gt; (&lt;code dir=&quot;auto&quot;&gt;scripts/ci/gate_selection.py&lt;/code&gt;).&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;regenerate&quot;&gt;Regenerate&lt;/h2&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;python&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;scripts/ci/consumer_census.py&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--write&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;   &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# rebuild and write the artifact (about 25 s)&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;python&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;scripts/ci/consumer_census.py&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--check&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;   &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# exit 1 if the committed file is stale&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;Commit the regenerated file with the change that caused it.&lt;/p&gt;
&lt;h2 id=&quot;fix-an-unrouted-consumer-red&quot;&gt;Fix an unrouted-consumer red&lt;/h2&gt;
&lt;ol&gt;
&lt;li&gt;Read the failure: it names the pack path and the consuming module.&lt;/li&gt;
&lt;li&gt;Add a glob for the path to the matching group in &lt;code dir=&quot;auto&quot;&gt;PACK_CONTENT_SCOPES&lt;/code&gt;
(&lt;code dir=&quot;auto&quot;&gt;scripts/ci/gate_selection.py&lt;/code&gt;), not to &lt;code dir=&quot;auto&quot;&gt;ci-router.yml&lt;/code&gt;: a &lt;code dir=&quot;auto&quot;&gt;packs/built-in/**&lt;/code&gt; glob
in the router matches no tracked file. For a new group, list it in
&lt;code dir=&quot;auto&quot;&gt;GROUP_MODULE_FANOUT&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;PACK_CONTENT_SCOPES&lt;/code&gt;, and give its router filter the pin
&lt;code dir=&quot;auto&quot;&gt;packs/built-in.source.yaml&lt;/code&gt; as its only glob.&lt;/li&gt;
&lt;li&gt;For a module-missing red, open the census &lt;code dir=&quot;auto&quot;&gt;via&lt;/code&gt; evidence for that module. If its
tests assert on the shipped content, add the module to &lt;code dir=&quot;auto&quot;&gt;GROUP_MODULE_FANOUT&lt;/code&gt;.
If the read is incidental (a fixture or side effect), add it to &lt;code dir=&quot;auto&quot;&gt;FANOUT_ALLOWLIST&lt;/code&gt;
with a one-line reason.&lt;/li&gt;
&lt;li&gt;Re-run &lt;code dir=&quot;auto&quot;&gt;--write&lt;/code&gt;, then the two tests above.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The census covers pack and source paths only. The &lt;code dir=&quot;auto&quot;&gt;uv.lock&lt;/code&gt; route to the &lt;code dir=&quot;auto&quot;&gt;upgrade&lt;/code&gt;
and &lt;code dir=&quot;auto&quot;&gt;status&lt;/code&gt; shards lives in &lt;code dir=&quot;auto&quot;&gt;DEPENDENCY_MODULE_FANOUT&lt;/code&gt; (&lt;code dir=&quot;auto&quot;&gt;scripts/ci/gate_selection.py&lt;/code&gt;),
outside the census, because a lock file has no test consumer to derive (#6238).&lt;/p&gt;</content>
</entry>
