<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xml:base="https://docs.spec-kitty.ai/adr/3.x/2026-04-22-5-glossary-chokepoint-p95-measurement/">
  <id>https://docs.spec-kitty.ai/adr/3.x/2026-04-22-5-glossary-chokepoint-p95-measurement/</id>
  <title>Glossary Chokepoint p95 Latency Measurement</title>
  <updated>2026-04-22T00:00:00.000Z</updated>
  <author><name>Spec Kitty</name></author>
  <link rel="alternate" type="text/html" href="https://docs.spec-kitty.ai/adr/3.x/2026-04-22-5-glossary-chokepoint-p95-measurement/"/>
  <summary>Confirms the 50 ms p95 budget for the glossary chokepoint with measured sub-linear scaling, keeping the per-step semantic scan off the agent loop critical path.</summary>
  <content type="html">&lt;h2 id=&quot;context-and-problem-statement&quot;&gt;Context and Problem Statement&lt;/h2&gt;
&lt;p&gt;&lt;code dir=&quot;auto&quot;&gt;GlossaryChokepoint.run()&lt;/code&gt; is invoked on every agent step to scan request text
for semantic conflicts.  It must complete well within the human-perceptible
threshold so that the scan never becomes a bottleneck in the agent loop.&lt;/p&gt;
&lt;p&gt;The design target was &lt;strong&gt;p95 ≤ 50 ms&lt;/strong&gt; for all realistic input sizes.  This ADR
records the measured performance to confirm or revise that threshold.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;benchmark-setup&quot;&gt;Benchmark Setup&lt;/h2&gt;
&lt;p&gt;Benchmark script: &lt;code dir=&quot;auto&quot;&gt;tests/specify_cli/glossary/bench_chokepoint.py&lt;/code&gt;&lt;/p&gt;





























&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Parameter&lt;/th&gt;&lt;th&gt;Value&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;Synthetic term index&lt;/td&gt;&lt;td&gt;500 active SPEC_KITTY_CORE terms&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Iterations per input size&lt;/td&gt;&lt;td&gt;1 000 (after 10 warm-up)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Input sizes tested&lt;/td&gt;&lt;td&gt;500 words, 2 000 words, 5 000 words&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Term hit rate in texts&lt;/td&gt;&lt;td&gt;~20 % (deliberate over-representation)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Platform&lt;/td&gt;&lt;td&gt;macOS Darwin 24.6.0, Python 3.14.0&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Approximately 20 % of tokens in the generated texts were drawn from the
synthetic index, which is a deliberate over-representation of realistic
inputs.  Real agent requests typically contain far fewer glossary terms.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;measured-results&quot;&gt;Measured Results&lt;/h2&gt;
&lt;p&gt;Benchmark executed on 2026-04-22.&lt;/p&gt;

































&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Input size (words)&lt;/th&gt;&lt;th&gt;mean&lt;/th&gt;&lt;th&gt;p50&lt;/th&gt;&lt;th&gt;p95&lt;/th&gt;&lt;th&gt;p99&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;500&lt;/td&gt;&lt;td&gt;2.28 ms&lt;/td&gt;&lt;td&gt;2.19 ms&lt;/td&gt;&lt;td&gt;&lt;strong&gt;2.44 ms&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;4.03 ms&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;2 000&lt;/td&gt;&lt;td&gt;8.88 ms&lt;/td&gt;&lt;td&gt;8.79 ms&lt;/td&gt;&lt;td&gt;&lt;strong&gt;9.16 ms&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;9.67 ms&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;5 000&lt;/td&gt;&lt;td&gt;22.19 ms&lt;/td&gt;&lt;td&gt;21.91 ms&lt;/td&gt;&lt;td&gt;&lt;strong&gt;22.85 ms&lt;/strong&gt;&lt;/td&gt;&lt;td&gt;26.70 ms&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;All three input sizes pass p95 ≤ 50 ms comfortably.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;decision&quot;&gt;Decision&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Threshold confirmed: p95 ≤ 50 ms.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The measured p95 values fall well below the 50 ms target at every tested
input size.  The design constraint is satisfied without modification.&lt;/p&gt;
&lt;p&gt;Key observations:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Latency scales sub-linearly with input size.&lt;/strong&gt;  A 10× input increase
(500 → 5 000 words) yields only a ~9× p95 increase (2.44 ms → 22.85 ms),
thanks to the per-token deduplication and the O(1) dictionary lookup in
&lt;code dir=&quot;auto&quot;&gt;GlossaryTermIndex.surface_to_senses&lt;/code&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Index load is amortised.&lt;/strong&gt;  &lt;code dir=&quot;auto&quot;&gt;_load_index()&lt;/code&gt; is called at most once per
&lt;code dir=&quot;auto&quot;&gt;GlossaryChokepoint&lt;/code&gt; instance.  The benchmark pre-loads the index before
timed iterations, matching production behaviour where the chokepoint is
constructed once and reused across many calls.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Headroom is substantial.&lt;/strong&gt;  Even at 5 000 words the p95 is 22.85 ms —
less than half the threshold.  This leaves room for future index growth
(e.g. 2 000+ terms) before the threshold becomes a concern.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;h2 id=&quot;consequences&quot;&gt;Consequences&lt;/h2&gt;
&lt;h3 id=&quot;positive&quot;&gt;Positive&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;The 50 ms p95 threshold is confirmed as a realistic operating constraint.&lt;/li&gt;
&lt;li&gt;No performance-driven architecture changes are needed at this time.&lt;/li&gt;
&lt;li&gt;CI tests can run without special hardware; the chokepoint adds negligible
latency to the existing test suite (&amp;#x3C; 1 ms per call in tests).&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;negative&quot;&gt;Negative&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;None identified.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;neutral&quot;&gt;Neutral&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;The threshold should be re-measured if the index grows beyond ~5 000 terms
or if a new text pre-processing step (e.g. NLP tokenisation) is added.&lt;/li&gt;
&lt;li&gt;The benchmark script (&lt;code dir=&quot;auto&quot;&gt;bench_chokepoint.py&lt;/code&gt;) is &lt;strong&gt;not&lt;/strong&gt; part of the CI suite;
it must be run manually when the chokepoint implementation changes.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id=&quot;re-measurement-trigger-conditions&quot;&gt;Re-measurement Trigger Conditions&lt;/h2&gt;
&lt;p&gt;Re-run &lt;code dir=&quot;auto&quot;&gt;tests/specify_cli/glossary/bench_chokepoint.py&lt;/code&gt; and update this ADR if:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;The &lt;code dir=&quot;auto&quot;&gt;GlossaryTermIndex&lt;/code&gt; grows to more than 5 000 active terms.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;_run_inner()&lt;/code&gt; is modified to call additional I/O-bound operations.&lt;/li&gt;
&lt;li&gt;A new text pre-processing step (e.g. sentence segmentation, NLP) is added
before the tokenisation loop.&lt;/li&gt;
&lt;li&gt;The runtime target platform changes significantly (e.g. from macOS to a
resource-constrained container).&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;h2 id=&quot;related-documents&quot;&gt;Related Documents&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/specify_cli/glossary/chokepoint.py&lt;/code&gt; — implementation&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;tests/specify_cli/glossary/bench_chokepoint.py&lt;/code&gt; — benchmark script&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;tests/specify_cli/glossary/test_chokepoint.py&lt;/code&gt; — unit tests (T014)&lt;/li&gt;
&lt;li&gt;WP01 ADR: &lt;code dir=&quot;auto&quot;&gt;architecture/adrs/2026-04-09-1-mission-identity-uses-ulid-not-sequential-prefix.md&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;</content>
</entry>
