<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xml:base="https://docs.spec-kitty.ai/adr/2.x/2026-03-15-1-vertical-slice-test-organisation/">
  <id>https://docs.spec-kitty.ai/adr/2.x/2026-03-15-1-vertical-slice-test-organisation/</id>
  <title>Vertical-Slice Test Organisation</title>
  <updated>2026-03-15T00:00:00.000Z</updated>
  <author><name>Spec Kitty</name></author>
  <link rel="alternate" type="text/html" href="https://docs.spec-kitty.ai/adr/2.x/2026-03-15-1-vertical-slice-test-organisation/"/>
  <summary>Reorganises the suite by system capability instead of by unit-versus-integration, demoting test type to pytest markers and filename suffixes where it belongs.</summary>
  <content type="html">&lt;h2 id=&quot;context-and-problem-statement&quot;&gt;Context and Problem Statement&lt;/h2&gt;
&lt;p&gt;The test suite grew organically alongside the codebase and ended up with two competing
top-level organising axes that are never reconciled:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;unit/&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;integration/&lt;/code&gt; are organised by &lt;strong&gt;test type&lt;/strong&gt; — but their internals are a
large flat bag of files with no relation to what the system actually does. A reader
cannot look at either directory and understand system capabilities.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;specify_cli/&lt;/code&gt; mirrors the &lt;strong&gt;source package tree&lt;/strong&gt; — but mixes unit and integration
tests indiscriminately inside each module folder, so the type axis is invisible.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Neither axis is consistently applied. The result is that navigating tests to understand
system behaviour requires reading filenames and imports rather than following directory
structure. It also makes it hard to answer “what tests cover the merge workflow?” or
“where do I add a test for a new mission capability?” without a mental map built from
experience.&lt;/p&gt;
&lt;p&gt;A secondary issue: several test-type directories (&lt;code dir=&quot;auto&quot;&gt;unit/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;integration/&lt;/code&gt;) have
accumulated flat test files at their root that cover unrelated concerns, functioning as
catch-all dumping grounds. This compounds over time.&lt;/p&gt;
&lt;h2 id=&quot;decision-drivers&quot;&gt;Decision Drivers&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Readability&lt;/strong&gt;: the test tree should be a readable description of system capabilities,
not a reflection of internal module boundaries or test-runner mechanics.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Navigability&lt;/strong&gt;: a new contributor should be able to locate tests for a feature by
finding the feature’s slice directory, without knowing the source package layout.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear placement&lt;/strong&gt;: when adding a test, there should be one obvious place for it.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Testing pyramid&lt;/strong&gt;: the pyramid (many unit, fewer integration, few e2e) should be
visible from markers and filename conventions, not directory depth.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Consistency&lt;/strong&gt;: the organising principle must be applied uniformly — no directories
that mix axes.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;considered-options&quot;&gt;Considered Options&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Option A — Vertical slices as top level, test type via markers + filename suffix&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Option B — Source package mirror as top level, each module gets &lt;code dir=&quot;auto&quot;&gt;unit/&lt;/code&gt; + &lt;code dir=&quot;auto&quot;&gt;integration/&lt;/code&gt; subdirs&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Option C — Keep &lt;code dir=&quot;auto&quot;&gt;unit/&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;integration/&lt;/code&gt; top level, reorganise their internals to mirror the source package tree&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;decision-outcome&quot;&gt;Decision Outcome&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Chosen option: Option A&lt;/strong&gt;, because it expresses the system as a set of capabilities
rather than implementation artefacts. Test type (unit vs integration) is a
runner-mechanics concern that is better expressed through pytest markers and filename
suffixes than through directory structure. Option B optimises for source code proximity
at the cost of capability visibility. Option C only partially fixes the problem — the
type/slice inversion remains at the top level.&lt;/p&gt;
&lt;h3 id=&quot;consequences&quot;&gt;Consequences&lt;/h3&gt;
&lt;h4 id=&quot;positive&quot;&gt;Positive&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;The test tree reads as a capability map of the system.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;pytest tests/&amp;#x3C;slice&gt;/&lt;/code&gt; runs all tests for one vertical of the product.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;pytest -m fast&lt;/code&gt; or &lt;code dir=&quot;auto&quot;&gt;pytest -m git_repo&lt;/code&gt; give fast-feedback and mid-tier runs
orthogonally — test-type selection is via markers, not path.&lt;/li&gt;
&lt;li&gt;New tests have one unambiguous home.&lt;/li&gt;
&lt;li&gt;The migration is incremental: each slice can be reorganised independently without
breaking others.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id=&quot;negative&quot;&gt;Negative&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;A one-time migration effort is required to move files from the current layout.&lt;/li&gt;
&lt;li&gt;Some tests genuinely span multiple slices (e.g. a migration that touches both
&lt;code dir=&quot;auto&quot;&gt;upgrade&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;missions&lt;/code&gt;); these go into the most responsible slice with a comment
noting the cross-cutting concern.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id=&quot;neutral&quot;&gt;Neutral&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;legacy/&lt;/code&gt; is unchanged for now; it will be addressed as a follow-up (audit valuable
knowledge, extract 2.x-relevant tests, then delete).&lt;/li&gt;
&lt;li&gt;Cross-cutting concerns without a natural slice home (encoding validation, packaging,
doctrine schema compliance) keep their existing &lt;code dir=&quot;auto&quot;&gt;cross_cutting/&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;doctrine/&lt;/code&gt;
directories — those are already organised by concern, not by test type.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;confirmation&quot;&gt;Confirmation&lt;/h3&gt;
&lt;p&gt;The restructure is complete when:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;unit/&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;integration/&lt;/code&gt; directories no longer exist (or are empty stubs awaiting
migration).&lt;/li&gt;
&lt;li&gt;Every slice directory contains both &lt;code dir=&quot;auto&quot;&gt;*_unit.py&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;*_integration.py&lt;/code&gt; files (or
&lt;code dir=&quot;auto&quot;&gt;*_unit.py&lt;/code&gt; only, for slices with no integration surface).&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;pytest tests/&amp;#x3C;slice&gt;/&lt;/code&gt; produces a meaningful subset of the suite for each slice.&lt;/li&gt;
&lt;li&gt;The full suite still passes with no regressions.&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 id=&quot;pros-and-cons-of-the-options&quot;&gt;Pros and Cons of the Options&lt;/h2&gt;
&lt;h3 id=&quot;option-a--vertical-slices-as-top-level&quot;&gt;Option A — Vertical slices as top level&lt;/h3&gt;
&lt;p&gt;Top-level directories map to system capabilities:
&lt;code dir=&quot;auto&quot;&gt;missions/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;merge/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;agent/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;tasks/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;git_ops/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;status/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;upgrade/&lt;/code&gt;,
&lt;code dir=&quot;auto&quot;&gt;init/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;sync/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;runtime/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;research/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;next/&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Test type is expressed by:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;pytest marker: &lt;code dir=&quot;auto&quot;&gt;fast&lt;/code&gt; (unit/mock-boundary) or &lt;code dir=&quot;auto&quot;&gt;git_repo&lt;/code&gt;/&lt;code dir=&quot;auto&quot;&gt;slow&lt;/code&gt; (integration)&lt;/li&gt;
&lt;li&gt;filename suffix: &lt;code dir=&quot;auto&quot;&gt;*_unit.py&lt;/code&gt; vs &lt;code dir=&quot;auto&quot;&gt;*_integration.py&lt;/code&gt; (or plain &lt;code dir=&quot;auto&quot;&gt;test_*.py&lt;/code&gt; where the
distinction is not meaningful)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Capability-first — matches how developers and product owners think about the system.&lt;/li&gt;
&lt;li&gt;Markers + suffixes give orthogonal test-type filtering without directory nesting.&lt;/li&gt;
&lt;li&gt;Scales well: adding a new capability means adding a new slice, not editing a shared bucket.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Requires migrating existing &lt;code dir=&quot;auto&quot;&gt;unit/&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;integration/&lt;/code&gt; directories.&lt;/li&gt;
&lt;li&gt;Some tests are harder to classify (cross-cutting or multi-concern).&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;option-b--source-package-mirror&quot;&gt;Option B — Source package mirror&lt;/h3&gt;
&lt;p&gt;Top-level directories mirror &lt;code dir=&quot;auto&quot;&gt;src/specify_cli/&lt;/code&gt;: &lt;code dir=&quot;auto&quot;&gt;charter/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;glossary/&lt;/code&gt;,
&lt;code dir=&quot;auto&quot;&gt;status/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;sync/&lt;/code&gt;, etc. Each module directory gets &lt;code dir=&quot;auto&quot;&gt;unit/&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;integration/&lt;/code&gt;
subdirs.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Strong coupling between source module and its tests — easy to find tests for a
given module.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;specify_cli/&lt;/code&gt; already partially follows this pattern.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Internal module boundaries leak into the test tree. Refactoring the source
package (e.g. splitting or merging modules) forces test directory renames.&lt;/li&gt;
&lt;li&gt;Module boundaries do not align with user-facing capabilities — &lt;code dir=&quot;auto&quot;&gt;glossary/&lt;/code&gt; and
&lt;code dir=&quot;auto&quot;&gt;charter/&lt;/code&gt; are implementation details, not product features.&lt;/li&gt;
&lt;li&gt;The &lt;code dir=&quot;auto&quot;&gt;unit/&lt;/code&gt; + &lt;code dir=&quot;auto&quot;&gt;integration/&lt;/code&gt; nesting restores the type-first problem one level deeper.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;option-c--type-first-top-level-internals-mirror-source&quot;&gt;Option C — Type-first top level, internals mirror source&lt;/h3&gt;
&lt;p&gt;Keep &lt;code dir=&quot;auto&quot;&gt;unit/&lt;/code&gt; and &lt;code dir=&quot;auto&quot;&gt;integration/&lt;/code&gt; as top-level directories but organise their
internals to mirror the source package tree (e.g. &lt;code dir=&quot;auto&quot;&gt;unit/agent/&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;unit/merge/&lt;/code&gt;,
&lt;code dir=&quot;auto&quot;&gt;integration/merge/&lt;/code&gt;).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;No change to the top-level shape that CI scripts and contributor docs reference.&lt;/li&gt;
&lt;li&gt;Internal structure becomes navigable.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The fundamental inversion remains: test type is the primary axis, capability is
secondary. This is the opposite of what communicates system behaviour.&lt;/li&gt;
&lt;li&gt;Related tests for one feature are split across two top-level directories, requiring
navigation between them.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;more-information&quot;&gt;More Information&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Initiative tracker: &lt;code dir=&quot;auto&quot;&gt;architecture/2.x/initiatives/test_improvement/README.md&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Current test suite profile: section 2 of the initiative README&lt;/li&gt;
&lt;li&gt;Filename convention: &lt;code dir=&quot;auto&quot;&gt;*_unit.py&lt;/code&gt; files carry &lt;code dir=&quot;auto&quot;&gt;pytestmark = pytest.mark.fast&lt;/code&gt;;
integration tests carry &lt;code dir=&quot;auto&quot;&gt;pytestmark = pytest.mark.git_repo&lt;/code&gt; (or &lt;code dir=&quot;auto&quot;&gt;slow&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;Marker registry: &lt;code dir=&quot;auto&quot;&gt;pytest.ini&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;</content>
</entry>
