<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xml:base="https://docs.spec-kitty.ai/development/testing/run-mutation-tests/">
  <id>https://docs.spec-kitty.ai/development/testing/run-mutation-tests/</id>
  <title>Run Mutation Tests Locally</title>
  <updated>2026-09-30T00:00:00.000Z</updated>
  <author><name>Spec Kitty</name></author>
  <link rel="alternate" type="text/html" href="https://docs.spec-kitty.ai/development/testing/run-mutation-tests/"/>
  <summary>How to run mutation tests locally with mutmut: mutation testing answers &quot;if I introduced a bug here, would my tests fail?&quot; where line coverage cannot.</summary>
  <content type="html">&lt;span id=&quot;run-mutation-tests-locally&quot;&gt;&lt;/span&gt;
&lt;p&gt;Mutation testing answers the question &lt;strong&gt;“if I introduced a bug here, would my tests
fail?”&lt;/strong&gt;. Line and branch coverage only answer “did my tests touch this code?” — a
suite with 95% coverage can still accept silent regressions when its assertions are
weak. Mutation testing makes those weaknesses visible.&lt;/p&gt;
&lt;p&gt;Spec Kitty uses &lt;code dir=&quot;auto&quot;&gt;mutmut&lt;/code&gt; 3.5.0 as a &lt;strong&gt;local-only developer tool&lt;/strong&gt;. There is no CI
gate. Run it when:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;You’ve finished a work package and want an objective signal of assertion quality.&lt;/li&gt;
&lt;li&gt;You suspect a module has “happy-path-only” tests (coverage high, asserts flimsy).&lt;/li&gt;
&lt;li&gt;You’re reviewing a contributor PR and want to sanity-check their test discipline.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;See &lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/docs/adr/3.x/2026-04-20-1-mutation-testing-as-local-only-quality-gate.md&quot;&gt;ADR 2026-04-20-1&lt;/a&gt;
for the decision to ship it local-only, and the curated doctrine set in
&lt;code dir=&quot;auto&quot;&gt;packs/built-in/tactics/testing/mutation-testing-workflow.tactic.yaml&lt;/code&gt; and
&lt;code dir=&quot;auto&quot;&gt;packs/built-in/styleguides/mutation-aware-test-design.styleguide.yaml&lt;/code&gt; for
how to write tests that kill mutants.&lt;/p&gt;
&lt;h2 id=&quot;quick-start&quot;&gt;Quick start&lt;/h2&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# Install the test extras (includes mutmut)&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;sync&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--extra&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;test&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# Run the full mutation suite — expect 30–90 min on a modern machine&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# Pipe results somewhere durable (mutmut&apos;s results are in-memory)&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;results&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;&gt;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut-results.txt&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;h2 id=&quot;scope-a-run-to-one-module&quot;&gt;Scope a run to one module&lt;/h2&gt;
&lt;p&gt;Mutmut patterns match &lt;strong&gt;dotted module names&lt;/strong&gt;, not file paths:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# correct&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#984E4D&quot;&gt;specify_cli.compat*&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#984E4D&quot;&gt;charter.activation._drg_helpers&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# wrong (file paths do not match)&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#984E4D&quot;&gt;src/specify_cli/compat/*&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;A focused run finishes in minutes and is the right mode for kill-the-survivor work.&lt;/p&gt;
&lt;h2 id=&quot;triage-surviving-mutants&quot;&gt;Triage surviving mutants&lt;/h2&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# Summary table: Killed / Survived / No Coverage / Timeout / Equivalent&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;results&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# Show every surviving mutant as a unified diff&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;show&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;all&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# Interactive TUI — navigate survivors by file&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;browse&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# One mutant at a time (id from `mutmut results`)&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;show&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;&amp;#x3C;id&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# Apply a mutant as a patch to see what test would catch it&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;apply&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;&amp;#x3C;id&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# ... write the test that kills it ...&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;git&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;checkout&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;&amp;#x3C;file&gt;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;For each surviving mutant, classify it using the four-bucket taxonomy from
&lt;code dir=&quot;auto&quot;&gt;mutation-testing-workflow.tactic.yaml&lt;/code&gt;:&lt;/p&gt;






























&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Category&lt;/th&gt;&lt;th&gt;Meaning&lt;/th&gt;&lt;th&gt;Action&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;Killed&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;A test failed when the mutant was injected&lt;/td&gt;&lt;td&gt;Nothing — the test suite did its job&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;Survived&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;All tests passed with the mutant in place&lt;/td&gt;&lt;td&gt;&lt;strong&gt;Add or strengthen a test&lt;/strong&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;No Coverage&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;No test runs the mutated line&lt;/td&gt;&lt;td&gt;Add a test that reaches the code&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;Equivalent&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;Mutation produces no observable behaviour change&lt;/td&gt;&lt;td&gt;Suppress with &lt;code dir=&quot;auto&quot;&gt;# pragma: no mutate&lt;/code&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;For &lt;strong&gt;Survived&lt;/strong&gt; mutants, the kill strategy depends on the mutation family. The
full operator table lives in
&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/packs/built-in/toolguides/PYTHON_MUTATION_TOOLS.md&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;packs/built-in/toolguides/PYTHON_MUTATION_TOOLS.md&lt;/code&gt;&lt;/a&gt;.
Short version: comparison flips need boundary-value tests, arithmetic swaps need
non-identity inputs (never &lt;code dir=&quot;auto&quot;&gt;0&lt;/code&gt; for &lt;code dir=&quot;auto&quot;&gt;+&lt;/code&gt;, never &lt;code dir=&quot;auto&quot;&gt;1&lt;/code&gt; for &lt;code dir=&quot;auto&quot;&gt;*&lt;/code&gt;), logical operator swaps
need bi-directional cases (exactly one operand true/false).&lt;/p&gt;
&lt;h2 id=&quot;parallel-execution&quot;&gt;Parallel execution&lt;/h2&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame is-terminal not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;span class=&quot;title&quot;&gt;&lt;/span&gt;&lt;span class=&quot;sr-only&quot;&gt;Terminal window&lt;/span&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# CLI override&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;uv&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;mutmut&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#3B61B0&quot;&gt;run&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;--max-children&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#F78C6C;--1:#AA0982&quot;&gt;16&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# Or persist in pyproject.toml [tool.mutmut]&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# max_children = 8&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;Each child forks a pytest run. Above ~16 on most machines, pytest startup overhead
dominates and the speedup plateaus.&lt;/p&gt;
&lt;h2 id=&quot;suppress-equivalent-mutants&quot;&gt;Suppress equivalent mutants&lt;/h2&gt;
&lt;p&gt;If a mutation genuinely cannot change observable behaviour (a log message, a
version string, a formatting-only branch), suppress it inline:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;python&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;LOG_PREFIX&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#984E4D&quot;&gt;cache-evict&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;  &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# pragma: no mutate&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;format_version&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;major&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;, &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;minor&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;, &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;patch&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;:&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;    &lt;/span&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;return&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;f&lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#984E4D&quot;&gt;&quot;v&lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;{&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;major&lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;}&lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#984E4D&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;{&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;minor&lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;}&lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#984E4D&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;{&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;patch&lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;}&lt;/span&gt;&lt;span style=&quot;--0:#ECC48D;--1:#984E4D&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;  &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# pragma: no mutate&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;Do not over-use &lt;code dir=&quot;auto&quot;&gt;# pragma: no mutate&lt;/code&gt;. If equivalent-mutant inflation goes above
~10 % of total mutants, you are probably using it to dodge survivors rather than
to annotate true equivalents.&lt;/p&gt;
&lt;h2 id=&quot;adding-a-test-that-wont-run-in-the-mutmut-sandbox&quot;&gt;Adding a test that won’t run in the mutmut sandbox&lt;/h2&gt;
&lt;p&gt;Some tests are structurally incompatible with mutmut’s forked sandbox:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;They invoke &lt;code dir=&quot;auto&quot;&gt;python -m specify_cli&lt;/code&gt; as a subprocess (hits the
&lt;a href=&quot;https://github.com/boxed/mutmut/issues/&quot;&gt;mutmut 3.5.0 trampoline &lt;code dir=&quot;auto&quot;&gt;NoneType&lt;/code&gt; bug&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;They walk the whole codebase with &lt;code dir=&quot;auto&quot;&gt;ast.parse&lt;/code&gt; (blows the 30 s per-test timeout).&lt;/li&gt;
&lt;li&gt;They build a wheel or spin up a fresh venv (too slow).&lt;/li&gt;
&lt;li&gt;They depend on &lt;code dir=&quot;auto&quot;&gt;kitty-specs/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;scripts/&lt;/code&gt;, or other repo-root paths not in
mutmut’s &lt;code dir=&quot;auto&quot;&gt;also_copy&lt;/code&gt; tree.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For these tests, add the &lt;code dir=&quot;auto&quot;&gt;non_sandbox&lt;/code&gt; marker at &lt;strong&gt;module level&lt;/strong&gt;:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;python&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;import&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; pytest&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# Marked for mutmut sandbox skip — see ADR 2026-04-20-1.&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# Reason: subprocess CLI invocation hits mutmut 3.5.0 trampoline bug.&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;pytestmark &lt;/span&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; pytest.mark.non_sandbox&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;If the test file already has a marker, combine them as a list:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;python&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;pytestmark &lt;/span&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;[&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;pytest.mark.git_repo, pytest.mark.non_sandbox&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;]&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;  &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# non_sandbox: subprocess CLI invocation&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;The &lt;code dir=&quot;auto&quot;&gt;-m &quot;not non_sandbox&quot;&lt;/code&gt; filter in &lt;code dir=&quot;auto&quot;&gt;pyproject.toml[tool.mutmut].pytest_add_cli_args&lt;/code&gt;
will skip these in the mutmut sandbox. They still run in the main test suite.&lt;/p&gt;
&lt;h2 id=&quot;flaky-tests-under-mutmut&quot;&gt;Flaky tests under mutmut&lt;/h2&gt;
&lt;p&gt;If a test passes reliably in the main suite but produces different results under
mutmut (e.g., it reads &lt;code dir=&quot;auto&quot;&gt;target_branch&lt;/code&gt; from git and gets &lt;code dir=&quot;auto&quot;&gt;main&lt;/code&gt; in one phase vs.
&lt;code dir=&quot;auto&quot;&gt;2.x&lt;/code&gt; in another), &lt;strong&gt;that is tech debt&lt;/strong&gt;, not accepted loss. Mark it &lt;code dir=&quot;auto&quot;&gt;flaky&lt;/code&gt; with
a root-cause TODO:&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;python&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# &lt;/span&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;TODO&lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;: target_branch detection races with git state in mutmut&apos;s clean-test phase.&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;#       Root cause is unknown — should be fixed and the marker removed.&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;pytestmark &lt;/span&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; pytest.mark.flaky&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;The goal for &lt;code dir=&quot;auto&quot;&gt;flaky&lt;/code&gt; is to shrink over time. Do not use it as a long-term
suppression mechanism.&lt;/p&gt;
&lt;h2 id=&quot;sandbox-configuration-in-pyprojecttoml&quot;&gt;Sandbox configuration in &lt;code dir=&quot;auto&quot;&gt;pyproject.toml&lt;/code&gt;&lt;/h2&gt;
&lt;p&gt;The &lt;code dir=&quot;auto&quot;&gt;[tool.mutmut]&lt;/code&gt; section controls the sandbox. Relevant knobs:&lt;/p&gt;

































&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Key&lt;/th&gt;&lt;th&gt;Purpose&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;paths_to_mutate&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Which directories under &lt;code dir=&quot;auto&quot;&gt;src/&lt;/code&gt; get mutated&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;do_not_mutate&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Files exempted — currently &lt;code dir=&quot;auto&quot;&gt;version_utils.py&lt;/code&gt; (contains the mutmut trampoline) and &lt;code dir=&quot;auto&quot;&gt;upgrade/migrations/&lt;/code&gt; (idempotent, not meaningfully testable via mutation)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;also_copy&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Extra files/dirs copied into &lt;code dir=&quot;auto&quot;&gt;mutants/&lt;/code&gt; sandbox (contracts, docs, architecture are already included)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;pytest_add_cli_args&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Appended to pytest invocation in sandbox — includes &lt;code dir=&quot;auto&quot;&gt;-m &quot;not non_sandbox and not flaky and ...&quot;&lt;/code&gt; deselection&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;max_children&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Number of parallel mutant workers&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;timeout_constant&lt;/code&gt;/&lt;code dir=&quot;auto&quot;&gt;timeout_multiplier&lt;/code&gt;&lt;/td&gt;&lt;td&gt;How long mutmut waits before killing a slow mutant&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;h2 id=&quot;target-mutation-scores&quot;&gt;Target mutation scores&lt;/h2&gt;

























&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Score&lt;/th&gt;&lt;th&gt;Interpretation&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;&gt; 90 %&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;Strong — but watch for equivalent-mutant inflation&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;80–90 %&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;Good&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;60–80 %&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;Moderate — improvements available&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;strong&gt;&amp;#x3C; 60 %&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;Structurally weak — tests don’t actually assert behaviour&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Focus effort on core business logic (&lt;code dir=&quot;auto&quot;&gt;src/specify_cli/compat/&lt;/code&gt;,
&lt;code dir=&quot;auto&quot;&gt;src/specify_cli/status/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;src/charter/activation/synthesizer/&lt;/code&gt;). Mutation score on
generated code, migrations, or boilerplate is not meaningful.&lt;/p&gt;
&lt;h2 id=&quot;when-the-sandbox-baseline-fails&quot;&gt;When the sandbox baseline fails&lt;/h2&gt;
&lt;p&gt;If &lt;code dir=&quot;auto&quot;&gt;mutmut run&lt;/code&gt; exits with a baseline test failure before reaching any mutants,
the failing test is either structurally incompatible with the sandbox (add
&lt;code dir=&quot;auto&quot;&gt;non_sandbox&lt;/code&gt;) or genuinely flaky under mutmut (add &lt;code dir=&quot;auto&quot;&gt;flaky&lt;/code&gt;). Check the failure
mode:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code dir=&quot;auto&quot;&gt;ModuleNotFoundError&lt;/code&gt;&lt;/strong&gt; at collection → the test imports a bare script-module
that isn’t on &lt;code dir=&quot;auto&quot;&gt;sys.path&lt;/code&gt; in the sandbox. Ignore it at directory level in
&lt;code dir=&quot;auto&quot;&gt;pytest_add_cli_args --ignore=&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code dir=&quot;auto&quot;&gt;AttributeError: &apos;NoneType&apos; object has no attribute &apos;max_stack_depth&apos;&lt;/code&gt;&lt;/strong&gt; →
the mutmut trampoline bug. The test spawned &lt;code dir=&quot;auto&quot;&gt;python -m specify_cli&lt;/code&gt; as a
subprocess. Mark &lt;code dir=&quot;auto&quot;&gt;non_sandbox&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code dir=&quot;auto&quot;&gt;pytest.fail: Timeout (&gt;30.0s)&lt;/code&gt;&lt;/strong&gt; → a whole-codebase walker, a wheel build,
or a slow integration setup. Mark &lt;code dir=&quot;auto&quot;&gt;non_sandbox&lt;/code&gt; or &lt;code dir=&quot;auto&quot;&gt;slow&lt;/code&gt; depending on cause.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Assertion mismatch with no obvious cause&lt;/strong&gt; → likely &lt;code dir=&quot;auto&quot;&gt;flaky&lt;/code&gt;. Investigate;
add the marker with a TODO.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;see-also&quot;&gt;See also&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/docs/adr/3.x/2026-04-20-1-mutation-testing-as-local-only-quality-gate.md&quot;&gt;ADR 2026-04-20-1&lt;/a&gt;
— the decision record covering scope, doctrine, and the &lt;code dir=&quot;auto&quot;&gt;non_sandbox&lt;/code&gt;/&lt;code dir=&quot;auto&quot;&gt;flaky&lt;/code&gt;
marker taxonomy.&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/packs/built-in/tactics/testing/mutation-testing-workflow.tactic.yaml&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;packs/built-in/tactics/testing/mutation-testing-workflow.tactic.yaml&lt;/code&gt;&lt;/a&gt;
— the five-step kill-the-survivor workflow.&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/packs/built-in/styleguides/mutation-aware-test-design.styleguide.yaml&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;packs/built-in/styleguides/mutation-aware-test-design.styleguide.yaml&lt;/code&gt;&lt;/a&gt;
— boundary-pair, non-identity-inputs, bi-directional-logic patterns.&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/packs/built-in/toolguides/PYTHON_MUTATION_TOOLS.md&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;packs/built-in/toolguides/PYTHON_MUTATION_TOOLS.md&lt;/code&gt;&lt;/a&gt;
— full Python operator reference.&lt;/li&gt;
&lt;li&gt;&lt;a href=&quot;https://github.com/spec-kitty/spec-kitty/blob/main/packs/built-in/toolguides/TYPESCRIPT_MUTATION_TOOLS.md&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;packs/built-in/toolguides/TYPESCRIPT_MUTATION_TOOLS.md&lt;/code&gt;&lt;/a&gt;
— parallel guide for TypeScript projects using Stryker.&lt;/li&gt;
&lt;/ul&gt;</content>
</entry>
