<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xml:base="https://docs.spec-kitty.ai/development/reference/coverage-signals/">
  <id>https://docs.spec-kitty.ai/development/reference/coverage-signals/</id>
  <title>Coverage signals — reconciling the three &quot;coverage&quot; numbers</title>
  <updated>2026-09-30T00:00:00.000Z</updated>
  <author><name>Spec Kitty</name></author>
  <link rel="alternate" type="text/html" href="https://docs.spec-kitty.ai/development/reference/coverage-signals/"/>
  <summary>Why SonarCloud coverage, new_coverage, and the internal diff-coverage CI gate disagree — and how to tell an expected scope difference from a real coverage regression.</summary>
  <content type="html">&lt;span id=&quot;coverage-signals--reconciling-the-three-coverage-numbers&quot;&gt;&lt;/span&gt;
&lt;p&gt;Three different measurements are all colloquially called “coverage” on a Spec
Kitty pull request, and they routinely disagree by tens of percentage points.
That disagreement is &lt;strong&gt;expected and by design&lt;/strong&gt; — it is not, on its own, a bug
or a regression. This guide explains what each number measures, why they differ,
and gives you a decision aid for the moment they seem to contradict each other:
&lt;em&gt;is this a real regression, or an expected scope difference?&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;If you only remember one thing: &lt;strong&gt;the internal &lt;code dir=&quot;auto&quot;&gt;diff-coverage&lt;/code&gt; gate and the
SonarCloud numbers measure different files, different lines, and against
different baselines. A passing internal gate next to a low SonarCloud number is
the normal, healthy state — not a contradiction.&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-three-signals-at-a-glance&quot;&gt;The three signals at a glance&lt;/h2&gt;

































&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Signal&lt;/th&gt;&lt;th&gt;Where it runs&lt;/th&gt;&lt;th&gt;Which files it scores&lt;/th&gt;&lt;th&gt;Which lines it counts&lt;/th&gt;&lt;th&gt;Threshold&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;Internal &lt;code dir=&quot;auto&quot;&gt;diff-cover&lt;/code&gt; gate&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;The &lt;code dir=&quot;auto&quot;&gt;diff-cover&lt;/code&gt; job in &lt;code dir=&quot;auto&quot;&gt;.github/workflows/ci-aggregate.yml&lt;/code&gt;, which runs on &lt;code dir=&quot;auto&quot;&gt;workflow_run&lt;/code&gt; after &lt;strong&gt;CI Modules&lt;/strong&gt; completes&lt;/td&gt;&lt;td&gt;A deliberate &lt;strong&gt;critical-path subset&lt;/strong&gt; of &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt; (see list below)&lt;/td&gt;&lt;td&gt;&lt;strong&gt;Only the lines your change touched&lt;/strong&gt;, as the statement-level diff &lt;code dir=&quot;auto&quot;&gt;scripts/ci/validate_diff_coverage.py&lt;/code&gt; derives&lt;/td&gt;&lt;td&gt;&lt;strong&gt;90%&lt;/strong&gt;, blocking&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;SonarCloud &lt;code dir=&quot;auto&quot;&gt;coverage&lt;/code&gt;&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;Nightly Sonar analysis (&lt;code dir=&quot;auto&quot;&gt;.github/workflows/sonar.yml&lt;/code&gt;, cron &lt;code dir=&quot;auto&quot;&gt;0 3 * * *&lt;/code&gt;) / manual dispatch&lt;/td&gt;&lt;td&gt;The &lt;strong&gt;whole &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt; tree&lt;/strong&gt; (&lt;code dir=&quot;auto&quot;&gt;sonar.sources=src&lt;/code&gt;)&lt;/td&gt;&lt;td&gt;&lt;strong&gt;Every&lt;/strong&gt; executable line in the tree, cumulative&lt;/td&gt;&lt;td&gt;No blocking floor on the overall number (the gate uses the &lt;code dir=&quot;auto&quot;&gt;new_*&lt;/code&gt; metrics)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;SonarCloud &lt;code dir=&quot;auto&quot;&gt;new_coverage&lt;/code&gt;&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;Same nightly analysis&lt;/td&gt;&lt;td&gt;The whole &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt; tree&lt;/td&gt;&lt;td&gt;Lines in the &lt;strong&gt;New Code Period&lt;/strong&gt; (since the &lt;code dir=&quot;auto&quot;&gt;projectVersion&lt;/code&gt; baseline)&lt;/td&gt;&lt;td&gt;&lt;strong&gt;80%&lt;/strong&gt;, blocking Sonar’s own gate&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;One measurement, several readers.&lt;/strong&gt; Every number above is computed from the
&lt;em&gt;same&lt;/em&gt; per-module coverage reports: &lt;code dir=&quot;auto&quot;&gt;module-tests.yml&lt;/code&gt;’s shards each run pytest
with dotted &lt;code dir=&quot;auto&quot;&gt;--cov=&lt;/code&gt; targets and upload a &lt;code dir=&quot;auto&quot;&gt;coverage-&amp;#x3C;tier&gt;-&amp;#x3C;module&gt;-shard&amp;#x3C;n&gt;.xml&lt;/code&gt;;
&lt;code dir=&quot;auto&quot;&gt;ci-aggregate.yml&lt;/code&gt;’s &lt;code dir=&quot;auto&quot;&gt;collect&lt;/code&gt; job reconciles that set for one change, and both
the &lt;code dir=&quot;auto&quot;&gt;diff-cover&lt;/code&gt; gate and the per-change Sonar report read the reconciliation.
Nothing re-runs the suite to obtain a coverage number.&lt;/p&gt;
&lt;p&gt;There is also a &lt;strong&gt;per-change SonarCloud report&lt;/strong&gt; — the &lt;code dir=&quot;auto&quot;&gt;sonar-pr&lt;/code&gt; job in
&lt;code dir=&quot;auto&quot;&gt;ci-aggregate.yml&lt;/code&gt; (spec-kitty#4334) — which analyses the same reconciled
measurement. It is informational and &lt;strong&gt;can never block a merge&lt;/strong&gt;: it carries
&lt;code dir=&quot;auto&quot;&gt;continue-on-error&lt;/code&gt; and is excluded from the terminal verdict’s &lt;code dir=&quot;auto&quot;&gt;needs:&lt;/code&gt;. Until
spec-kitty#4350 clears, it may refuse to publish; see &lt;a href=&quot;#known-caveats-and-follow-ups&quot;&gt;Known
caveats&lt;/a&gt;. Before that mission, the per-PR Sonar
surface was a &lt;code dir=&quot;auto&quot;&gt;sonarcloud&lt;/code&gt; job in &lt;code dir=&quot;auto&quot;&gt;ci-quality.yml&lt;/code&gt; that re-ran the fast tier
under &lt;code dir=&quot;auto&quot;&gt;pytest --cov&lt;/code&gt; purely to produce a report the shards had already produced
for the same commit — a second measurement of the same tests, discarded.&lt;/p&gt;
&lt;h3 id=&quot;the-internal-gates-critical-path-allowlist&quot;&gt;The internal gate’s critical-path allowlist&lt;/h3&gt;
&lt;p&gt;The enforced internal gate restricts itself (&lt;code dir=&quot;auto&quot;&gt;diff-cover ... --include&lt;/code&gt;) to these
paths — the kernel, charter, status, merge, and mission-runtime
surfaces where a coverage miss is highest-risk:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;plaintext&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;src/kernel/*&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;src/charter/*&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;src/specify_cli/status/*&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;src/specify_cli/lanes/branch_naming.py&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;src/specify_cli/consolidation/*&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;src/runtime/next/*&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;src/mission_runtime/*&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;That is roughly &lt;strong&gt;247 Python files&lt;/strong&gt; — a strict subset of the &lt;strong&gt;~969 tracked
&lt;code dir=&quot;auto&quot;&gt;.py&lt;/code&gt; files&lt;/strong&gt; SonarCloud scores across the whole &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt; tree (SonarCloud indexes
&lt;strong&gt;1305 files&lt;/strong&gt; in total under &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt;, spanning the top-level packages of that
measurement: &lt;code dir=&quot;auto&quot;&gt;specify_cli&lt;/code&gt;, the former &lt;code dir=&quot;auto&quot;&gt;doctrine&lt;/code&gt; (since absorbed into
&lt;code dir=&quot;auto&quot;&gt;charter.offering&lt;/code&gt;), &lt;code dir=&quot;auto&quot;&gt;charter&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;runtime&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;glossary&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;kernel&lt;/code&gt;,
&lt;code dir=&quot;auto&quot;&gt;mission_runtime&lt;/code&gt;).&lt;/p&gt;
&lt;h3 id=&quot;remedy-git-mv-into-a-critical-path-dir-needs-fast-marked-coverage&quot;&gt;Remedy: &lt;code dir=&quot;auto&quot;&gt;git mv&lt;/code&gt; into a critical-path dir needs &lt;code dir=&quot;auto&quot;&gt;fast&lt;/code&gt;-marked coverage&lt;/h3&gt;
&lt;p&gt;Moving (&lt;code dir=&quot;auto&quot;&gt;git mv&lt;/code&gt;) a module &lt;em&gt;into&lt;/em&gt; one of the critical-path directories above
subjects its relocated lines to the enforced 90% diff-coverage floor — but the
coverage numerator that floor is judged against does not come from “however
the module is tested somewhere in the suite.” The numerator comes from the
per-module shards &lt;code dir=&quot;auto&quot;&gt;module-tests.yml&lt;/code&gt; runs, each scoped by the module registry’s
tier marker filter and measuring the full top-level package set derived from &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt; (#4334)
(&lt;code dir=&quot;auto&quot;&gt;.github/ci-module-registry.yml&lt;/code&gt;). &lt;code dir=&quot;auto&quot;&gt;ci-aggregate.yml&lt;/code&gt;’s &lt;code dir=&quot;auto&quot;&gt;collect&lt;/code&gt; job
reconciles every shard’s &lt;code dir=&quot;auto&quot;&gt;coverage-*.xml&lt;/code&gt; for the change, and &lt;code dir=&quot;auto&quot;&gt;diff-cover&lt;/code&gt;
scores the critical-path statement diff over that combined set.&lt;/p&gt;
&lt;p&gt;If the moved module’s real tests carry a marker its shard’s filter excludes,
they never execute inside that shard, so the module’s lines are effectively
absent from — or
reported near-0% in — the aggregated XML, and &lt;code dir=&quot;auto&quot;&gt;diff-cover&lt;/code&gt; fails the
critical-path move even though the module is well-tested elsewhere in the
suite by its non-fast parity tests.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; add a test module in &lt;code dir=&quot;auto&quot;&gt;tests/&amp;#x3C;critical-path-dir&gt;/&lt;/code&gt; carrying a marker
the covering shard &lt;em&gt;does&lt;/em&gt; select, exercising the moved module with
mocked/stubbed dependencies (no real subprocess/filesystem/git calls), so its
lines reach the reconciled set. Keep the original real-integration tests
(&lt;code dir=&quot;auto&quot;&gt;git_repo&lt;/code&gt;/&lt;code dir=&quot;auto&quot;&gt;non_sandbox&lt;/code&gt;/&lt;code dir=&quot;auto&quot;&gt;integration&lt;/code&gt;-marked) where they already are — the
new fast module is additive, not a replacement.&lt;/p&gt;
&lt;p&gt;This is a fresh trap (PR #3437, mission &lt;code dir=&quot;auto&quot;&gt;write-path-integrity&lt;/code&gt;): moving
&lt;code dir=&quot;auto&quot;&gt;git_topology&lt;/code&gt; into &lt;code dir=&quot;auto&quot;&gt;src/kernel/&lt;/code&gt; to fix a layer inversion left it at 34%
diff-coverage because &lt;code dir=&quot;auto&quot;&gt;kernel-tests&lt;/code&gt; (&lt;code dir=&quot;auto&quot;&gt;-m fast&lt;/code&gt;) could not run the module’s
real-git &lt;code dir=&quot;auto&quot;&gt;git_repo&lt;/code&gt;-marked parity tests in &lt;code dir=&quot;auto&quot;&gt;tests/git/test_git_topology.py&lt;/code&gt;.
Adding &lt;code dir=&quot;auto&quot;&gt;tests/kernel/test_git_topology_fast.py&lt;/code&gt; — &lt;code dir=&quot;auto&quot;&gt;pytest.mark.fast&lt;/code&gt;, with
&lt;code dir=&quot;auto&quot;&gt;subprocess.run&lt;/code&gt; mocked — brought the module to 100% line coverage in
&lt;code dir=&quot;auto&quot;&gt;coverage-kernel.xml&lt;/code&gt;, which satisfied both the &lt;code dir=&quot;auto&quot;&gt;diff-coverage&lt;/code&gt; gate and
&lt;code dir=&quot;auto&quot;&gt;kernel-tests&lt;/code&gt;’ own 90% floor over &lt;code dir=&quot;auto&quot;&gt;--cov=src/kernel&lt;/code&gt;.&lt;/p&gt;
&lt;h2 id=&quot;why-the-numbers-differ-the-evidence&quot;&gt;Why the numbers differ (the evidence)&lt;/h2&gt;
&lt;p&gt;The reconciliation below was produced against SonarCloud’s &lt;strong&gt;public read API&lt;/strong&gt;
(no token) using the read-only tool &lt;code dir=&quot;auto&quot;&gt;scripts/ci/sonarcloud_branch_review.sh&lt;/code&gt;.
You can reproduce every figure yourself:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;scripts/ci/sonarcloud_branch_review.sh&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;coverage&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;       &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# overall + new_coverage&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;scripts/ci/sonarcloud_branch_review.sh&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;quality-gate&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;   &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# gate conditions + thresholds&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;scripts/ci/sonarcloud_branch_review.sh&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;version&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;        &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# projectVersion / baseline history&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;Three independent axes make the numbers diverge:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;File set (scope of files).&lt;/strong&gt; SonarCloud scores &lt;em&gt;all&lt;/em&gt; of &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt;; the enforced
internal gate scores only the critical-path subset above. So SonarCloud will
surface uncovered lines in packages (for example &lt;code dir=&quot;auto&quot;&gt;src/specify_cli/cli/&lt;/code&gt;,
&lt;code dir=&quot;auto&quot;&gt;.../missions/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;.../doctor/&lt;/code&gt;) that the &lt;em&gt;enforced&lt;/em&gt; internal gate never looks
at. This is intentional — coverage effort is kept proportional to risk — and
the full-diff advisory step exists precisely so the wider diff is still
visible without blocking.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Line basis (which lines, over what history).&lt;/strong&gt; This is the dominant driver.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;SonarCloud &lt;code dir=&quot;auto&quot;&gt;coverage&lt;/code&gt; (&lt;strong&gt;47.3%&lt;/strong&gt;) is a &lt;strong&gt;cumulative average over the entire
codebase&lt;/strong&gt;: &lt;code dir=&quot;auto&quot;&gt;lines_to_cover = 95,111&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;uncovered_lines = 50,090&lt;/code&gt;. It counts
every executable line ever written, covered or not.&lt;/li&gt;
&lt;li&gt;The internal gate (&lt;strong&gt;90%&lt;/strong&gt;) counts &lt;strong&gt;only the handful of lines the current
PR changed&lt;/strong&gt;. A ten-line PR is judged on ten lines.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These two can never be close: one is a whole-history denominator of ~95k
lines, the other is your diff.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Baseline (what “new” means).&lt;/strong&gt; SonarCloud &lt;code dir=&quot;auto&quot;&gt;new_coverage&lt;/code&gt; (&lt;strong&gt;50.06%&lt;/strong&gt;)
&lt;em&gt;sounds&lt;/em&gt; like it should match the internal per-PR gate — both mention “new”
code — but it does not, for a concrete reason: &lt;code dir=&quot;auto&quot;&gt;new_lines_to_cover = 80,038&lt;/code&gt;.
About &lt;strong&gt;84% of the whole tree&lt;/strong&gt; is currently counted as “new code,” because
every recent nightly analysis reports &lt;code dir=&quot;auto&quot;&gt;projectVersion = &quot;not provided&quot;&lt;/code&gt; and
the New Code Period baseline is therefore frozen. So today &lt;code dir=&quot;auto&quot;&gt;new_coverage&lt;/code&gt;
(~50%) is effectively &lt;em&gt;another whole-repo number&lt;/em&gt;, not a per-PR one — which is
why it sits right next to the overall &lt;code dir=&quot;auto&quot;&gt;coverage&lt;/code&gt;, nowhere near 90%. (See
&lt;a href=&quot;#known-caveats-and-follow-ups&quot;&gt;Known caveats&lt;/a&gt; — resetting this baseline
per release cycle is part of the pending Sonar decision. Even after such a
fix, &lt;code dir=&quot;auto&quot;&gt;new_coverage&lt;/code&gt; becomes a &lt;em&gt;per-release-cycle&lt;/em&gt; number, still not a
&lt;em&gt;per-PR&lt;/em&gt; one.)&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 id=&quot;the-verdict-file-set-and-philosophy-differ--but-nothing-is-misconfigured&quot;&gt;The verdict: file-set &lt;strong&gt;and&lt;/strong&gt; philosophy differ — but nothing is misconfigured&lt;/h2&gt;
&lt;p&gt;Investigation finding, stated plainly:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;The file sets do differ.&lt;/strong&gt; SonarCloud scores the entire &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt; tree; the
enforced internal gate scores only a critical-path subset of the PR diff.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;The measurement philosophy also differs&lt;/strong&gt;, and more decisively: whole-repo
cumulative average (and a baseline-anchored “new code” number) versus per-PR
changed-lines-only.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Neither difference is a &lt;code dir=&quot;auto&quot;&gt;sources&lt;/code&gt;/&lt;code dir=&quot;auto&quot;&gt;exclusions&lt;/code&gt; misconfiguration.&lt;/strong&gt;
&lt;code dir=&quot;auto&quot;&gt;sonar.sources=src&lt;/code&gt; with &lt;code dir=&quot;auto&quot;&gt;sonar.exclusions=**/__pycache__/**,**/*.pyc,**/migrations/**&lt;/code&gt;
is correct and standard: tests are registered separately as &lt;code dir=&quot;auto&quot;&gt;sonar.tests&lt;/code&gt;
(so they are not counted as production lines to cover), and no generated or
vendored code is wrongly swept in (the agent directories live at the repo
root, outside &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt;). The internal allowlist is a deliberate,
risk-proportional choice, not an accident.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Because the divergence is a philosophy-and-baseline difference rather than a
genuine file-set misconfiguration, it is &lt;strong&gt;discharged by this document, not by a
config change&lt;/strong&gt; (research-first, per the mission constraint C-002). In
particular, this guide deliberately does &lt;strong&gt;not&lt;/strong&gt; recommend narrowing Sonar’s
scope to flatter the number — doing so would mask genuinely untested code, which
the project’s no-ratchet standing order forbids.&lt;/p&gt;
&lt;h2 id=&quot;decision-aid-real-regression-or-expected-scope-difference&quot;&gt;Decision aid: real regression, or expected scope difference?&lt;/h2&gt;
&lt;p&gt;When a SonarCloud number looks alarming next to the internal gate, walk these
questions in order:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Did the internal &lt;code dir=&quot;auto&quot;&gt;diff-coverage&lt;/code&gt; (critical-path, enforced) gate pass on
your PR?&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Yes&lt;/strong&gt; → the lines you changed in critical-path modules are ≥90% covered.
A low SonarCloud &lt;em&gt;overall&lt;/em&gt; &lt;code dir=&quot;auto&quot;&gt;coverage&lt;/code&gt; or &lt;em&gt;current&lt;/em&gt; &lt;code dir=&quot;auto&quot;&gt;new_coverage&lt;/code&gt; is &lt;strong&gt;not a
regression you introduced&lt;/strong&gt; — it is the whole-repo / frozen-baseline scope.
Stop here for those two numbers.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;No&lt;/strong&gt; → you have a real gap in changed critical-path lines. Add tests
before merge. This is a real signal, not a scope artifact.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Is the number you are worried about SonarCloud’s whole-repo &lt;code dir=&quot;auto&quot;&gt;coverage&lt;/code&gt;
(~47%)?&lt;/strong&gt; Judge it against the &lt;strong&gt;previous nightly analysis’s &lt;code dir=&quot;auto&quot;&gt;coverage&lt;/code&gt;&lt;/strong&gt;, not
against the 90% per-PR gate. A &lt;em&gt;drop&lt;/em&gt; versus the previous analysis is worth
investigating; a low &lt;em&gt;absolute&lt;/em&gt; value is expected and structural.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Is it SonarCloud &lt;code dir=&quot;auto&quot;&gt;new_coverage&lt;/code&gt;?&lt;/strong&gt; Until the &lt;code dir=&quot;auto&quot;&gt;projectVersion&lt;/code&gt; baseline is
wired (see caveats), “new code” ≈ the whole repo, so read it like the overall
number. After that fix it becomes a per-release-cycle figure — still not the
per-PR diff the internal gate reports.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Did your PR change files &lt;em&gt;outside&lt;/em&gt; the critical-path allowlist?&lt;/strong&gt; The
&lt;em&gt;enforced&lt;/em&gt; internal gate does not measure them (only the advisory full-diff
step does); SonarCloud does. So SonarCloud can legitimately show uncovered
new lines that the enforced gate stayed silent on — an &lt;strong&gt;expected scope
difference, not a gate bug&lt;/strong&gt;. If those lines carry real logic, add focused
tests anyway (the charter’s rule: every new branch/helper ships with tests).&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;If after these steps a real coverage gap remains in code you changed, treat it
as a regression and add tests. If every “gap” resolves to whole-repo scope or a
frozen baseline, it is an expected scope difference — record that reasoning in
the PR so the next reader does not re-litigate it.&lt;/p&gt;
&lt;h2 id=&quot;where-each-signal-is-configured&quot;&gt;Where each signal is configured&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Internal &lt;code dir=&quot;auto&quot;&gt;diff-cover&lt;/code&gt; gate&lt;/strong&gt; — the &lt;code dir=&quot;auto&quot;&gt;diff-cover&lt;/code&gt; job in
&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/.github/workflows/ci-aggregate.yml&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;.github/workflows/ci-aggregate.yml&lt;/code&gt;&lt;/a&gt;
(&lt;code dir=&quot;auto&quot;&gt;--fail-under=90&lt;/code&gt; over the statement diff produced by
&lt;code dir=&quot;auto&quot;&gt;scripts/ci/validate_diff_coverage.py&lt;/code&gt;). The critical-path list itself is
single-sourced as &lt;code dir=&quot;auto&quot;&gt;CRITICAL_PATHS&lt;/code&gt; in
&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/scripts/ci/aggregate_source.py&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;scripts/ci/aggregate_source.py&lt;/code&gt;&lt;/a&gt;,
which materialises the diff the gate scores.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Per-change SonarCloud report&lt;/strong&gt; — the &lt;code dir=&quot;auto&quot;&gt;sonar-pr&lt;/code&gt; job in the same file,
driven by &lt;code dir=&quot;auto&quot;&gt;scripts/ci/sonar_pr_analysis.py&lt;/code&gt;. Informational only.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Nightly SonarCloud analysis&lt;/strong&gt; —
&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/.github/workflows/sonar.yml&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;.github/workflows/sonar.yml&lt;/code&gt;&lt;/a&gt;
(cron &lt;code dir=&quot;auto&quot;&gt;0 3 * * *&lt;/code&gt;), which aggregates the same shard coverage artefacts.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;The measurement itself&lt;/strong&gt; —
&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/.github/workflows/module-tests.yml&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;.github/workflows/module-tests.yml&lt;/code&gt;&lt;/a&gt;
and the module registry &lt;code dir=&quot;auto&quot;&gt;.github/ci-module-registry.yml&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;SonarCloud scope and exclusions&lt;/strong&gt; —
&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/sonar-project.properties&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;sonar-project.properties&lt;/code&gt;&lt;/a&gt;
(&lt;code dir=&quot;auto&quot;&gt;sonar.sources=src&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;sonar.tests=tests&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;sonar.exclusions=...&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Read-only query tool&lt;/strong&gt; —
&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/scripts/ci/sonarcloud_branch_review.sh&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;scripts/ci/sonarcloud_branch_review.sh&lt;/code&gt;&lt;/a&gt;,
which reproduces every number above against the public API with no
&lt;code dir=&quot;auto&quot;&gt;SONAR_TOKEN&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Related testing guides&lt;/strong&gt; — &lt;a href=&quot;/development/testing/testing-flakiness/&quot;&gt;Test-flakiness handling policy&lt;/a&gt;,
&lt;a href=&quot;/development/testing/testing-parallel/&quot;&gt;Running the test suite in parallel&lt;/a&gt;,
&lt;a href=&quot;/development/how-to/review-gates/&quot;&gt;Review gates&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;known-caveats-and-follow-ups&quot;&gt;Known caveats and follow-ups&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code dir=&quot;auto&quot;&gt;projectVersion&lt;/code&gt; baseline is frozen, and the cause is upstream.&lt;/strong&gt; Recent
analyses report &lt;code dir=&quot;auto&quot;&gt;projectVersion = &quot;not provided&quot;&lt;/code&gt;, so SonarCloud’s New Code
Period never resets and &lt;code dir=&quot;auto&quot;&gt;new_coverage&lt;/code&gt; behaves like a whole-repo metric. The
&lt;code dir=&quot;auto&quot;&gt;projectVersion&lt;/code&gt; wiring is &lt;strong&gt;not&lt;/strong&gt; missing — &lt;code dir=&quot;auto&quot;&gt;scripts/ci/sonar_project_version.py&lt;/code&gt;
derives it from &lt;code dir=&quot;auto&quot;&gt;pyproject.toml&lt;/code&gt; and both &lt;code dir=&quot;auto&quot;&gt;sonar.yml&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;ci-aggregate.yml&lt;/code&gt;’s
&lt;code dir=&quot;auto&quot;&gt;sonar-pr&lt;/code&gt; job stamp it. What blocks it is &lt;strong&gt;spec-kitty#4350&lt;/strong&gt;: SonarCloud’s
server-side &lt;em&gt;Automatic Analysis&lt;/em&gt; is enabled on the &lt;code dir=&quot;auto&quot;&gt;spec-kitty_spec-kitty&lt;/code&gt;
project, and an Automatic Analysis result cannot be replaced by a pipeline
upload — so a correctly-wired upload is refused and is indistinguishable, from
the pipeline side, from a broken one. Disabling Automatic Analysis is an
action inside the vendor’s interface; until it is taken, &lt;strong&gt;do not read a green
CI run as evidence that a report was published&lt;/strong&gt;. Any fix also takes effect on
the &lt;strong&gt;next analysis after it merges&lt;/strong&gt;, not on merge itself.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Internal allowlist entry repointed.&lt;/strong&gt; The critical-path &lt;code dir=&quot;auto&quot;&gt;--include&lt;/code&gt; list
references &lt;code dir=&quot;auto&quot;&gt;src/specify_cli/lanes/branch_naming.py&lt;/code&gt;
(&lt;code dir=&quot;auto&quot;&gt;parse_mission_slug_from_branch&lt;/code&gt;) — the real defining home of the
branch-based mission-slug detection. An earlier draft pointed this entry at a
path that had since been removed/superseded; #2443 repointed it to this
defining home in &lt;strong&gt;both&lt;/strong&gt; authorities (the workflow &lt;code dir=&quot;auto&quot;&gt;--include&lt;/code&gt; array and
&lt;code dir=&quot;auto&quot;&gt;tests/release/test_diff_coverage_policy.py&lt;/code&gt;). This was always an incidental
staleness nit in the internal gate’s config, &lt;strong&gt;not&lt;/strong&gt; a SonarCloud
misconfiguration.&lt;/li&gt;
&lt;/ul&gt;</content>
</entry>
