Skip to content

Glossary Chokepoint p95 Latency Measurement

Context and Problem Statement

GlossaryChokepoint.run() is invoked on every agent step to scan request text for semantic conflicts. It must complete well within the human-perceptible threshold so that the scan never becomes a bottleneck in the agent loop.

The design target was p95 ≤ 50 ms for all realistic input sizes. This ADR records the measured performance to confirm or revise that threshold.


Benchmark Setup

Benchmark script: tests/specify_cli/glossary/bench_chokepoint.py

ParameterValue
Synthetic term index500 active SPEC_KITTY_CORE terms
Iterations per input size1 000 (after 10 warm-up)
Input sizes tested500 words, 2 000 words, 5 000 words
Term hit rate in texts~20 % (deliberate over-representation)
PlatformmacOS Darwin 24.6.0, Python 3.14.0

Approximately 20 % of tokens in the generated texts were drawn from the synthetic index, which is a deliberate over-representation of realistic inputs. Real agent requests typically contain far fewer glossary terms.


Measured Results

Benchmark executed on 2026-04-22.

Input size (words)meanp50p95p99
5002.28 ms2.19 ms2.44 ms4.03 ms
2 0008.88 ms8.79 ms9.16 ms9.67 ms
5 00022.19 ms21.91 ms22.85 ms26.70 ms

All three input sizes pass p95 ≤ 50 ms comfortably.


Decision

Threshold confirmed: p95 ≤ 50 ms.

The measured p95 values fall well below the 50 ms target at every tested input size. The design constraint is satisfied without modification.

Key observations:

  1. Latency scales sub-linearly with input size. A 10× input increase (500 → 5 000 words) yields only a ~9× p95 increase (2.44 ms → 22.85 ms), thanks to the per-token deduplication and the O(1) dictionary lookup in GlossaryTermIndex.surface_to_senses.

  2. Index load is amortised. _load_index() is called at most once per GlossaryChokepoint instance. The benchmark pre-loads the index before timed iterations, matching production behaviour where the chokepoint is constructed once and reused across many calls.

  3. Headroom is substantial. Even at 5 000 words the p95 is 22.85 ms — less than half the threshold. This leaves room for future index growth (e.g. 2 000+ terms) before the threshold becomes a concern.


Consequences

Positive

  • The 50 ms p95 threshold is confirmed as a realistic operating constraint.
  • No performance-driven architecture changes are needed at this time.
  • CI tests can run without special hardware; the chokepoint adds negligible latency to the existing test suite (< 1 ms per call in tests).

Negative

  • None identified.

Neutral

  • The threshold should be re-measured if the index grows beyond ~5 000 terms or if a new text pre-processing step (e.g. NLP tokenisation) is added.
  • The benchmark script (bench_chokepoint.py) is not part of the CI suite; it must be run manually when the chokepoint implementation changes.

Re-measurement Trigger Conditions

Re-run tests/specify_cli/glossary/bench_chokepoint.py and update this ADR if:

  1. The GlossaryTermIndex grows to more than 5 000 active terms.
  2. _run_inner() is modified to call additional I/O-bound operations.
  3. A new text pre-processing step (e.g. sentence segmentation, NLP) is added before the tokenisation loop.
  4. The runtime target platform changes significantly (e.g. from macOS to a resource-constrained container).

  • src/specify_cli/glossary/chokepoint.py — implementation
  • tests/specify_cli/glossary/bench_chokepoint.py — benchmark script
  • tests/specify_cli/glossary/test_chokepoint.py — unit tests (T014)
  • WP01 ADR: architecture/adrs/2026-04-09-1-mission-identity-uses-ulid-not-sequential-prefix.md