<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xml:base="https://docs.spec-kitty.ai/adr/3.x/2026-05-12-2-wp06-charter-encoding-chokepoint-PROPOSED/">
  <id>https://docs.spec-kitty.ai/adr/3.x/2026-05-12-2-wp06-charter-encoding-chokepoint-PROPOSED/</id>
  <title>PROPOSAL: Charter-content encoding chokepoint location (WP06)</title>
  <updated>2026-05-12T00:00:00.000Z</updated>
  <author><name>Spec Kitty</name></author>
  <link rel="alternate" type="text/html" href="https://docs.spec-kitty.ai/adr/3.x/2026-05-12-2-wp06-charter-encoding-chokepoint-PROPOSED/"/>
  <summary>Proposes a single charter-content read chokepoint that hard-fails on non-UTF-8 input with an opt-out flag, rather than retrofitting eighteen scattered read sites.</summary>
  <content type="html">&lt;ul&gt;
&lt;li&gt;rev 1 (2026-05-12 AM): initial proposal, three open sub-questions.&lt;/li&gt;
&lt;li&gt;rev 2 (2026-05-12 PM): HiC resolved Q2 (hard-fail with &lt;code dir=&quot;auto&quot;&gt;--unsafe&lt;/code&gt; bypass; add follow-on content-migration WP) and Q3 (dual-storage: per-mission preferred, shared elements centralized). Q1 (charset-normalizer dependency) re-framed with SBOM finding below; still pending HiC.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Date:&lt;/strong&gt; 2026-05-12&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Deciders:&lt;/strong&gt; Architect Alphonso (proposer), HiC (final decision)&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Technical Story:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Mission &lt;code dir=&quot;auto&quot;&gt;review-merge-gate-hardening-3-2-x-01KRC57C&lt;/code&gt; WP06&lt;/li&gt;
&lt;li&gt;Source bug: &lt;a href=&quot;https://github.com/Priivacy-ai/spec-kitty/issues/644&quot;&gt;Priivacy-ai/spec-kitty#644&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Parent epic: &lt;a href=&quot;https://github.com/Priivacy-ai/spec-kitty/issues/822&quot;&gt;#822&lt;/a&gt; (Broad Cleanup Only After Narrowing), &lt;a href=&quot;https://github.com/Priivacy-ai/spec-kitty/issues/992&quot;&gt;#992&lt;/a&gt; (WS-6)&lt;/li&gt;
&lt;li&gt;Hard constraint: mission spec NFR-004 — do not modify &gt;5 unrelated modules; broader audit explicitly deferred.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id=&quot;context-and-problem-statement&quot;&gt;Context and Problem Statement&lt;/h2&gt;
&lt;p&gt;Spec Kitty has 18 charter-content read sites across 8 modules under &lt;code dir=&quot;auto&quot;&gt;src/charter/&lt;/code&gt;. All of them use explicit &lt;code dir=&quot;auto&quot;&gt;read_text(encoding=&quot;utf-8&quot;)&lt;/code&gt; (1 site uses &lt;code dir=&quot;auto&quot;&gt;errors=&quot;replace&quot;&lt;/code&gt;). There is &lt;strong&gt;no centralized loader, no encoding-detection utility, and no &lt;code dir=&quot;auto&quot;&gt;chardet&lt;/code&gt;/&lt;code dir=&quot;auto&quot;&gt;charset_normalizer&lt;/code&gt; dependency in the project&lt;/strong&gt;. When content arrives in a non-UTF-8 encoding (Windows &lt;code dir=&quot;auto&quot;&gt;cp1252&lt;/code&gt; is the documented field case in #644), the system either decodes it incorrectly or fails with a &lt;code dir=&quot;auto&quot;&gt;UnicodeDecodeError&lt;/code&gt; — and corrupted text can then propagate through downstream artifacts before any check catches it.&lt;/p&gt;
&lt;p&gt;Issue 822 imposes a hard constraint: this fix ships only if narrowed to &lt;strong&gt;one&lt;/strong&gt; lifecycle chokepoint and &lt;strong&gt;one&lt;/strong&gt; regression case. Mission spec NFR-004 hard-codes that constraint: WP06 may not modify &gt;5 unrelated modules; if implementation reveals broader retrofit is needed, escalate — do not silently broaden scope.&lt;/p&gt;
&lt;p&gt;The architectural question is: &lt;strong&gt;where is the natural single chokepoint, and which subset of the 18 read sites does it cover without exceeding the NFR-004 budget?&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;decision-drivers&quot;&gt;Decision Drivers&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Ingest vs re-read distinction.&lt;/strong&gt; Encoding decisions belong at the &lt;strong&gt;boundary where content first enters the charter subsystem&lt;/strong&gt; from an untrusted source (user keyboard, SaaS payload, user-supplied file). Re-reads of already-normalized files do not need encoding detection — they need to trust the prior normalization.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;NFR-004 (5-module budget).&lt;/strong&gt; A blanket &lt;code dir=&quot;auto&quot;&gt;_io.py&lt;/code&gt; wrapping all 18 read sites touches 8 modules and overruns the budget. The chokepoint must be at a higher abstraction layer.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Provenance, not silent normalization.&lt;/strong&gt; When detection succeeds, the decision must be &lt;strong&gt;recorded as metadata&lt;/strong&gt; (alongside the file or in a provenance log) so future readers can verify the contract. Silent re-encoding is a worse failure mode than the bug being fixed.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Fail loud on ambiguity.&lt;/strong&gt; Mixed cp1252/UTF-8 content (a real field case where a single charter file contains both because of a copy/paste path) must fail with a diagnostic naming the file and the encodings observed. Silent “best guess” decoding is the bug.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Reuse, not re-invention.&lt;/strong&gt; Python ecosystem already has &lt;code dir=&quot;auto&quot;&gt;charset-normalizer&lt;/code&gt; (pure-Python, no compiled deps) for this; adding it as a dependency is acceptable if HiC approves. Alternative: hand-roll a BOM + ASCII-vs-cp1252 detector, which is narrower but more code to maintain.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Stable for the regression fixture.&lt;/strong&gt; The chokepoint must be reachable from a unit/integration test that hands it a &lt;code dir=&quot;auto&quot;&gt;cp1252&lt;/code&gt;-encoded payload and asserts either correct provenance + UTF-8 output, or a fail-loud diagnostic.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;considered-options&quot;&gt;Considered Options&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;(A) Centralize at the existing orchestrator &lt;code dir=&quot;auto&quot;&gt;ensure_charter_bundle_fresh()&lt;/code&gt; in &lt;code dir=&quot;auto&quot;&gt;src/charter/sync.py:66&lt;/code&gt;.&lt;/strong&gt; Add encoding detection there; other read sites stay as-is.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;(B) New &lt;code dir=&quot;auto&quot;&gt;src/charter/_io.py :: load_charter_file()&lt;/code&gt; wrapping all 18 read sites across 8 modules.&lt;/strong&gt; Full retrofit.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;(C) NARROWED — new &lt;code dir=&quot;auto&quot;&gt;src/charter/_io.py :: load_charter_file()&lt;/code&gt; applied only at the three ingest boundaries: interview save, sync ingest, and compile-from-user-input.&lt;/strong&gt; Re-read sites stay as-is. &lt;strong&gt;(Architect Alphonso recommendation.)&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;(D) Single chokepoint at &lt;code dir=&quot;auto&quot;&gt;ensure_charter_bundle_fresh()&lt;/code&gt; only; defer interview-save and compile-input.&lt;/strong&gt; Smallest possible diff.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;proposed-decision-outcome&quot;&gt;Proposed Decision Outcome&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Recommended option: (C) Narrowed — new &lt;code dir=&quot;auto&quot;&gt;charter/_io.py&lt;/code&gt; applied to three ingest boundaries&lt;/strong&gt;, because it (i) honors the “boundary where content first enters the charter subsystem” principle, (ii) fits the NFR-004 5-module budget (4 modules total: new &lt;code dir=&quot;auto&quot;&gt;_io.py&lt;/code&gt; + 3 ingest sites), (iii) covers the failure modes #644 documents (user-typed input on a &lt;code dir=&quot;auto&quot;&gt;cp1252&lt;/code&gt; Windows console, SaaS payload of unknown provenance, user-supplied charter file in a non-UTF-8 encoding), and (iv) leaves the re-read sites untouched on purpose — they trust the normalized contract that the ingest boundary now guarantees.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Concrete contract proposal (subject to HiC adjustment):&lt;/strong&gt;&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;New module &lt;code dir=&quot;auto&quot;&gt;src/charter/_io.py&lt;/code&gt;&lt;/strong&gt; exposes:
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;python&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;@dataclass&lt;/span&gt;&lt;span style=&quot;--1:#403F53&quot;&gt;&lt;span style=&quot;--0:#D6DEEB&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;--0:#D7DBE0&quot;&gt;frozen&lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;=&lt;/span&gt;&lt;span style=&quot;--0:#FF6A83;--1:#A24848&quot;&gt;True&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;)&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;class&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#FFCB8B;--1:#111111&quot;&gt;CharterContent&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;:&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;text: &lt;/span&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;str&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;                       &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# always UTF-8&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;source_encoding: &lt;/span&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;str&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;            &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# detected encoding, e.g. &quot;utf-8&quot;, &quot;cp1252&quot;, &quot;utf-8-sig&quot;&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;confidence: &lt;/span&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;float&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;               &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# detector confidence (0.0–1.0)&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;source_path: Path &lt;/span&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;|&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#FF6A83;--1:#A24848&quot;&gt;None&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;        &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# for path-based ingest; None for inline ingest&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;normalization_applied: &lt;/span&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;bool&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;     &lt;/span&gt;&lt;span style=&quot;--0:#919F9F;--1:#5F636F&quot;&gt;# True if re-encoded from non-UTF-8&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;load_charter_file&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;path&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: Path&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; -&gt; CharterContent: &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;...&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#C792EA;--1:#8844AE&quot;&gt;def&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;load_charter_bytes&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;(&lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;data&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;bytes&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;, &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;*&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;, &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;origin&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#C5E478;--1:#3B61B0&quot;&gt;str&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;)&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; -&gt; CharterContent: &lt;/span&gt;&lt;span style=&quot;--0:#82AAFF;--1:#3B61B0&quot;&gt;...&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Detection strategy.&lt;/strong&gt; Try in order: (a) BOM sniff; (b) strict UTF-8 decode; (c) &lt;code dir=&quot;auto&quot;&gt;charset-normalizer&lt;/code&gt; detection at confidence ≥ 0.85; (d) hard-fail with &lt;code dir=&quot;auto&quot;&gt;CHARTER_ENCODING_AMBIGUOUS&lt;/code&gt; naming the file and the candidates the detector considered. The &lt;code dir=&quot;auto&quot;&gt;charset-normalizer&lt;/code&gt; dependency addition needs HiC approval (see open sub-question).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Provenance recording.&lt;/strong&gt; When &lt;code dir=&quot;auto&quot;&gt;normalization_applied=True&lt;/code&gt;, the chokepoint writes a sibling provenance line to the charter directory’s &lt;code dir=&quot;auto&quot;&gt;.encoding-provenance.jsonl&lt;/code&gt; (append-only) — file path, detected encoding, confidence, timestamp. This file becomes the &lt;em&gt;one new artifact&lt;/em&gt; WP06 introduces; it is &lt;strong&gt;not&lt;/strong&gt; consumed by current commands but is the audit trail #644 explicitly asks for.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Three retrofit sites (the entire WP06 module budget):&lt;/strong&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/interview.py&lt;/code&gt; — replace &lt;code dir=&quot;auto&quot;&gt;path.read_text(encoding=&quot;utf-8&quot;)&lt;/code&gt; reads at the &lt;strong&gt;save/load roundtrip for interview state&lt;/strong&gt; (lines ~283, ~398) with &lt;code dir=&quot;auto&quot;&gt;load_charter_file(path)&lt;/code&gt;. Rationale: interview state is the first persistence of user-typed content.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/sync.py&lt;/code&gt; — replace &lt;code dir=&quot;auto&quot;&gt;charter_path.read_text(&quot;utf-8&quot;)&lt;/code&gt; at line ~151 (sync→YAML extract) with &lt;code dir=&quot;auto&quot;&gt;load_charter_file(path)&lt;/code&gt;. Rationale: SaaS-sourced content is an external-trust boundary.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/compiler.py&lt;/code&gt; — replace &lt;code dir=&quot;auto&quot;&gt;yaml.load(path.read_text(encoding=&quot;utf-8&quot;))&lt;/code&gt; at line ~594 with &lt;code dir=&quot;auto&quot;&gt;yaml.load(load_charter_file(path).text)&lt;/code&gt;. Rationale: user-supplied charter at compile time is the original #644 failure case.&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Untouched (deferred to a successor mission):&lt;/strong&gt; &lt;code dir=&quot;auto&quot;&gt;charter/context.py&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;charter/hasher.py&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;charter/language_scope.py&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;charter/compact.py&lt;/code&gt;, &lt;code dir=&quot;auto&quot;&gt;charter/neutrality/lint.py&lt;/code&gt;. These all re-read files that have already passed through ingest; they remain &lt;code dir=&quot;auto&quot;&gt;read_text(encoding=&quot;utf-8&quot;)&lt;/code&gt; and trust the normalization contract that #644’s successor mission can broaden later.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Regression fixture (the one regression case #822 requires):&lt;/strong&gt; new &lt;code dir=&quot;auto&quot;&gt;tests/charter/test_encoding_chokepoint.py&lt;/code&gt; exercises a &lt;code dir=&quot;auto&quot;&gt;cp1252&lt;/code&gt;-encoded charter file through &lt;code dir=&quot;auto&quot;&gt;compiler.compile_charter()&lt;/code&gt;, asserts the compiler succeeds, the provenance file records &lt;code dir=&quot;auto&quot;&gt;source_encoding=&quot;cp1252&quot;&lt;/code&gt; with &lt;code dir=&quot;auto&quot;&gt;normalization_applied=True&lt;/code&gt;, and the in-memory &lt;code dir=&quot;auto&quot;&gt;CharterContent.text&lt;/code&gt; is the correctly-decoded UTF-8 string. A second test asserts that genuinely mixed content (cp1252 bytes embedded in a UTF-8 file) raises with &lt;code dir=&quot;auto&quot;&gt;CHARTER_ENCODING_AMBIGUOUS&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Diagnostic codes (JSON-stable, parallel to WP03’s namespace):&lt;/strong&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;CHARTER_ENCODING_AMBIGUOUS&lt;/code&gt; — detector below confidence threshold or mixed content.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;CHARTER_ENCODING_NOT_NORMALIZED&lt;/code&gt; — provenance recording failed (filesystem error during chokepoint write).&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;strong&gt;Modules touched: 4&lt;/strong&gt; (new &lt;code dir=&quot;auto&quot;&gt;_io.py&lt;/code&gt; + 3 retrofit sites + 1 new test file). Within NFR-004 budget.&lt;/p&gt;
&lt;h3 id=&quot;consequences-if-approved&quot;&gt;Consequences if approved&lt;/h3&gt;
&lt;h4 id=&quot;positive&quot;&gt;Positive&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;The three external-trust boundaries of the charter subsystem now have an explicit, tested encoding contract.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;cp1252&lt;/code&gt;-originated Windows charter content stops silently corrupting downstream artifacts (the documented #644 failure mode).&lt;/li&gt;
&lt;li&gt;A &lt;code dir=&quot;auto&quot;&gt;.encoding-provenance.jsonl&lt;/code&gt; artifact gives operators an audit trail that proves what encoding was detected and when.&lt;/li&gt;
&lt;li&gt;The re-read sites remain unchanged — &lt;em&gt;intentionally&lt;/em&gt; — so the diff stays inside the NFR-004 budget and the broader audit can be re-scoped after 3.2.0 ships.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;CharterContent&lt;/code&gt; becomes a reusable type for any future broader-audit work; this WP lays the type without forcing the audit.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id=&quot;negative&quot;&gt;Negative&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;New dependency (&lt;code dir=&quot;auto&quot;&gt;charset-normalizer&lt;/code&gt;) — pure-Python, no compiled deps, MIT licensed; small additional install surface. &lt;strong&gt;HiC must approve dependency addition&lt;/strong&gt; (see open sub-question).&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;.encoding-provenance.jsonl&lt;/code&gt; is a new artifact that nothing reads yet. It exists for the audit trail #644 requires; treating it as a current-consumer obligation would inflate scope.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id=&quot;neutral&quot;&gt;Neutral&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;Re-read sites are intentionally left UTF-8-only; this is documented in the chokepoint’s docstring and in the deferral comment on #644.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;confirmation&quot;&gt;Confirmation&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;Regression &lt;code dir=&quot;auto&quot;&gt;test_encoding_chokepoint.py::test_cp1252_charter_compiles_cleanly&lt;/code&gt; passes.&lt;/li&gt;
&lt;li&gt;Regression &lt;code dir=&quot;auto&quot;&gt;test_encoding_chokepoint.py::test_mixed_encoding_fails_loudly&lt;/code&gt; raises &lt;code dir=&quot;auto&quot;&gt;CHARTER_ENCODING_AMBIGUOUS&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Manual smoke (one-off, documented in mission quickstart): a real Windows-authored charter with cp1252 smart quotes round-trips through &lt;code dir=&quot;auto&quot;&gt;spec-kitty.charter&lt;/code&gt; without character mangling, and &lt;code dir=&quot;auto&quot;&gt;.encoding-provenance.jsonl&lt;/code&gt; records the detection.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;grep -r &quot;read_text&quot; src/charter/&lt;/code&gt; shows ≤5 modules touched by this WP’s diff (NFR-004 check).&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;pros-and-cons-of-the-options&quot;&gt;Pros and Cons of the Options&lt;/h2&gt;
&lt;h3 id=&quot;a-centralize-at-ensure_charter_bundle_fresh&quot;&gt;(A) Centralize at &lt;code dir=&quot;auto&quot;&gt;ensure_charter_bundle_fresh()&lt;/code&gt;&lt;/h3&gt;
&lt;p&gt;Add encoding detection in the existing orchestrator only.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Truly single-site change.&lt;/li&gt;
&lt;li&gt;Reuses existing orchestration boundary.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The orchestrator does &lt;strong&gt;not&lt;/strong&gt; see interview-save content (the first persistence of user-typed input) nor user-supplied compile input. Two of the three documented #644 failure modes go uncovered.&lt;/li&gt;
&lt;li&gt;The exploration found that &lt;code dir=&quot;auto&quot;&gt;ensure_charter_bundle_fresh&lt;/code&gt; “does NOT consolidate encoding” — adding it there would not even cover all reads the orchestrator currently triggers, because downstream modules re-read independently. Net effect: 1 site changed, several failure modes still possible.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;b-full-retrofit-across-all-18-read-sites-in-8-modules&quot;&gt;(B) Full retrofit across all 18 read sites in 8 modules&lt;/h3&gt;
&lt;p&gt;Wrap every &lt;code dir=&quot;auto&quot;&gt;read_text&lt;/code&gt; in the charter subsystem.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Uniform behavior; no “trusted internal re-read” exception to remember.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Violates NFR-004&lt;/strong&gt; (8 modules &gt; 5-module budget).&lt;/li&gt;
&lt;li&gt;This is the broader audit #822 explicitly told us not to do here.&lt;/li&gt;
&lt;li&gt;The 14 internal re-read sites do not need encoding detection — they need to trust the normalization contract. Wrapping them is busywork that inflates diff and review surface.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;c-narrowed-three-ingest-site-retrofit--new-_iopy&quot;&gt;(C) Narrowed three-ingest-site retrofit + new &lt;code dir=&quot;auto&quot;&gt;_io.py&lt;/code&gt;&lt;/h3&gt;
&lt;p&gt;Recommended above.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt; see “Consequences if approved”.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt; see “Consequences if approved”. Main concrete cost is the new dependency.&lt;/p&gt;
&lt;h3 id=&quot;d-single-chokepoint-at-ensure_charter_bundle_fresh-only-defer-interview-save-and-compile-input&quot;&gt;(D) Single chokepoint at &lt;code dir=&quot;auto&quot;&gt;ensure_charter_bundle_fresh()&lt;/code&gt; only, defer interview-save and compile-input&lt;/h3&gt;
&lt;p&gt;Smallest possible diff.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Smallest review surface.&lt;/li&gt;
&lt;li&gt;Only 2 modules touched.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Leaves interview-save (user-typed content on a &lt;code dir=&quot;auto&quot;&gt;cp1252&lt;/code&gt; console) and compile-from-user-input (the original #644 reproduction) uncovered.&lt;/li&gt;
&lt;li&gt;The bug ships unfixed in its primary documented form.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;resolved-sub-questions-hic-2026-05-12&quot;&gt;Resolved sub-questions (HiC, 2026-05-12)&lt;/h2&gt;
&lt;h3 id=&quot;q2--mixed-content-policy--resolved-hardloud-fail-with---unsafe-bypass--follow-on-migration-wp&quot;&gt;Q2 — Mixed-content policy → &lt;strong&gt;RESOLVED: hard/loud fail with &lt;code dir=&quot;auto&quot;&gt;--unsafe&lt;/code&gt; bypass + follow-on migration WP&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;HiC accepted the hard-fail recommendation with &lt;strong&gt;two additions&lt;/strong&gt;:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Bypass option.&lt;/strong&gt; A &lt;code dir=&quot;auto&quot;&gt;--unsafe&lt;/code&gt; flag (or equivalent escape hatch) lets an operator deliberately proceed past &lt;code dir=&quot;auto&quot;&gt;CHARTER_ENCODING_AMBIGUOUS&lt;/code&gt; with a higher-confidence best-guess decode. The flag is named &lt;code dir=&quot;auto&quot;&gt;--unsafe&lt;/code&gt; (not &lt;code dir=&quot;auto&quot;&gt;--force&lt;/code&gt;) to convey that the operator is taking responsibility for downstream corruption. The bypass logs to &lt;code dir=&quot;auto&quot;&gt;.encoding-provenance.jsonl&lt;/code&gt; with &lt;code dir=&quot;auto&quot;&gt;bypass_used: true&lt;/code&gt; so the audit trail captures the override.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Remediation guidance in failure messages.&lt;/strong&gt; Like WP03’s mode-mismatch diagnostic, the encoding-ambiguous failure must contain enough information for the operator to repair the file without external research.&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;plaintext&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;ERROR: CHARTER_ENCODING_AMBIGUOUS&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;  &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;File: kitty-specs/&amp;#x3C;mission&gt;/charter/charter.yaml&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;  &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;Detected candidates:&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;- cp1252 (confidence 0.62)&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;- utf-8 with replacement (confidence 0.48)&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;  &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;Mixed-content signal: bytes 0xE9 0x80 0xAE at offset 1247 form&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;  &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;valid cp1252 &apos;é€®&apos; but invalid UTF-8.&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;  &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;What this means:&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;The file contains byte sequences that cannot be unambiguously&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;decoded as a single encoding. Silent best-guess decoding is the&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;bug this chokepoint exists to prevent.&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;
&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;  &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;Remediation options:&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;1. Open the file in a UTF-8-aware editor; locate the affected&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;       &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;bytes (offsets reported above) and re-save as UTF-8.&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;2. If you authored the file on a cp1252 console (Windows): run&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;       &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;&apos;iconv -f cp1252 -t utf-8 &amp;#x3C;file&gt; &gt; &amp;#x3C;file&gt;.utf8 &amp;#x26;&amp;#x26; mv &amp;#x3C;file&gt;.utf8 &amp;#x3C;file&gt;&apos;.&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;    &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;3. If you accept the higher-confidence decode and the operational&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;       &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;risk: re-run with --unsafe. The bypass is logged in&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;       &lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;--0:#d6deeb;--1:#403f53&quot;&gt;.encoding-provenance.jsonl with bypass_used=true.&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Follow-on: content migration flow.&lt;/strong&gt; HiC: “ensure existing artefacts/elements are made compliant so we do not end up in a situation where existing files/missions/… cause apparent regressions when loading. (consider adding a new content migration class/flow for this)”. This is captured as a &lt;strong&gt;new WP08 in the mission spec&lt;/strong&gt; (separate from WP06 to keep WP06 within NFR-004’s 5-module budget). WP08 scans existing missions’ charter content, detects non-UTF-8 encodings, and either auto-normalizes (with provenance) or fails with the same diagnostic so an operator can repair before the chokepoint goes live.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;h3 id=&quot;q3--provenance-file-location--resolved-dual-storage-prefer-per-mission-centralize-shared-elements&quot;&gt;Q3 — Provenance file location → &lt;strong&gt;RESOLVED: dual storage, prefer per-mission, centralize shared elements&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;HiC: “A combination of (a) and (b) — preferring (a), but shared elements can be stored in (b). We want to avoid duplication as much as possible.”&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Concrete contract:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Primary per-mission audit log:&lt;/strong&gt; &lt;code dir=&quot;auto&quot;&gt;kitty-specs/&amp;#x3C;mission&gt;/.encoding-provenance.jsonl&lt;/code&gt;. Records detection events for files inside that mission’s directory. Co-locates the audit trail with the artifact.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Shared centralized log:&lt;/strong&gt; &lt;code dir=&quot;auto&quot;&gt;.kittify/encoding-provenance/global.jsonl&lt;/code&gt;. Records detection events for &lt;strong&gt;non-mission-scoped charter content&lt;/strong&gt; — i.e., charter files that live outside a &lt;code dir=&quot;auto&quot;&gt;kitty-specs/&amp;#x3C;mission&gt;/&lt;/code&gt; tree (e.g., the top-level project charter at &lt;code dir=&quot;auto&quot;&gt;.kittify/charter/&lt;/code&gt; if such a thing exists, or sync-ingested content not yet bound to a mission).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Deduplication rule:&lt;/strong&gt; the same detection event MUST NOT appear in both files. The chokepoint picks one based on the file’s path: inside &lt;code dir=&quot;auto&quot;&gt;kitty-specs/&amp;#x3C;mission&gt;/&lt;/code&gt; → per-mission; elsewhere → centralized. The decision is mechanical, not heuristic.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Shared schema:&lt;/strong&gt; both files are JSONL with identical record schema. A reader/aggregator can &lt;code dir=&quot;auto&quot;&gt;cat&lt;/code&gt; both files in any order without coalescing logic.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Record schema (proposed):&lt;/strong&gt;&lt;/p&gt;
&lt;div class=&quot;expressive-code&quot;&gt;&lt;figure class=&quot;frame not-content&quot;&gt;&lt;figcaption class=&quot;header&quot;&gt;&lt;/figcaption&gt;&lt;pre data-language=&quot;json&quot;&gt;&lt;code&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;{&lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;&quot;event_id&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#C789D6;--1:#7C5686&quot;&gt;01HXYZ...&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;, &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;&quot;at&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#C789D6;--1:#7C5686&quot;&gt;2026-05-12T18:30:00+00:00&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;,&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;&quot;file_path&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#C789D6;--1:#7C5686&quot;&gt;kitty-specs/.../charter.yaml&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;,&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;&quot;source_encoding&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#C789D6;--1:#7C5686&quot;&gt;cp1252&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;, &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;&quot;confidence&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#F78C6C;--1:#AA0982&quot;&gt;0.93&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;,&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;&quot;normalization_applied&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#FF6A83;--1:#A24848&quot;&gt;true&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;, &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;&quot;bypass_used&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#FF6A83;--1:#A24848&quot;&gt;false&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;,&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;div class=&quot;ec-line&quot;&gt;&lt;div class=&quot;code&quot;&gt;&lt;span class=&quot;indent&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;&quot;actor&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#C789D6;--1:#7C5686&quot;&gt;&amp;#x3C;command-invocation&gt;&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;, &lt;/span&gt;&lt;span style=&quot;--0:#7FDBCA;--1:#096E72&quot;&gt;&quot;mission_id&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;: &lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#C789D6;--1:#7C5686&quot;&gt;01KRC57C...&lt;/span&gt;&lt;span style=&quot;--0:#D9F5DD;--1:#111111&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#FFFFFF;--1:#984E4D&quot;&gt;|&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt; &lt;/span&gt;&lt;span style=&quot;--0:#FF6A83;--1:#A24848&quot;&gt;null&lt;/span&gt;&lt;span style=&quot;--0:#D6DEEB;--1:#403F53&quot;&gt;}&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/figure&gt;&lt;/div&gt;
&lt;p&gt;&lt;code dir=&quot;auto&quot;&gt;mission_id&lt;/code&gt; is &lt;code dir=&quot;auto&quot;&gt;null&lt;/code&gt; for events written to the centralized log.&lt;/p&gt;
&lt;h2 id=&quot;open-sub-questions-for-hic&quot;&gt;Open sub-questions for HiC&lt;/h2&gt;
&lt;h3 id=&quot;q1--charset-normalizer-dependency--re-framed-with-sbom-finding-still-pending-hic&quot;&gt;Q1 — &lt;code dir=&quot;auto&quot;&gt;charset-normalizer&lt;/code&gt; dependency → &lt;strong&gt;re-framed with SBOM finding; still pending HiC&lt;/strong&gt;&lt;/h3&gt;
&lt;p&gt;HiC condition for approval: “Add the library if its SBOM is available, and a preliminary security risk assessment deems it sensible.”&lt;/p&gt;
&lt;h4 id=&quot;critical-finding&quot;&gt;Critical finding&lt;/h4&gt;
&lt;p&gt;&lt;strong&gt;&lt;code dir=&quot;auto&quot;&gt;charset-normalizer&lt;/code&gt; 3.4.7 is already in our supply chain.&lt;/strong&gt; It is a transitive dependency of &lt;code dir=&quot;auto&quot;&gt;requests&lt;/code&gt; (which the CLI depends on directly), locked in &lt;code dir=&quot;auto&quot;&gt;uv.lock&lt;/code&gt; with the full set of platform wheels resolved. We are not “adding a dependency” — we are &lt;strong&gt;promoting an existing transitive dependency to a direct dependency&lt;/strong&gt;, so we own the version pin and can require it intentionally rather than implicitly.&lt;/p&gt;
&lt;p&gt;This reframes the security/SBOM question significantly:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The library is already part of every install today (any &lt;code dir=&quot;auto&quot;&gt;spec-kitty&lt;/code&gt; install pulls it via &lt;code dir=&quot;auto&quot;&gt;requests&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;Promoting it to a direct dep does &lt;strong&gt;not&lt;/strong&gt; add a new install surface, new platform wheels, new sub-dependencies, or new license terms.&lt;/li&gt;
&lt;li&gt;It does change &lt;em&gt;intent&lt;/em&gt;: we declare a deliberate dependency rather than relying on &lt;code dir=&quot;auto&quot;&gt;requests&lt;/code&gt;’s transitive chain (which could in principle change in a future &lt;code dir=&quot;auto&quot;&gt;requests&lt;/code&gt; release).&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id=&quot;sbom-and-security-risk-assessment-preliminary&quot;&gt;SBOM and security risk assessment (preliminary)&lt;/h4&gt;

















































&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Item&lt;/th&gt;&lt;th&gt;Finding&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;Package&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;charset-normalizer&lt;/code&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Version (locked)&lt;/td&gt;&lt;td&gt;3.4.7&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Upstream&lt;/td&gt;&lt;td&gt;&lt;a href=&quot;https://github.com/jawah/charset_normalizer&quot;&gt;https://github.com/jawah/charset_normalizer&lt;/a&gt; (active maintenance)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;License&lt;/td&gt;&lt;td&gt;MIT (compatible with the project’s existing license set)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Install footprint&lt;/td&gt;&lt;td&gt;~1.5 MB; wheels for cp311, cp312, cp313 on macOS, Linux (manylinux + musllinux), Windows (x86, AMD64, ARM64)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Sub-dependencies&lt;/td&gt;&lt;td&gt;&lt;strong&gt;None.&lt;/strong&gt; Pure-Python with an &lt;em&gt;optional&lt;/em&gt; mypyc-compiled fast path. If the compiled path fails to import, pure-Python fallback is used.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Compiled extension risk&lt;/td&gt;&lt;td&gt;Optional mypyc binaries are present in some wheels. They are produced by the maintainer’s CI, not bundled C from third parties. Failure mode is a clean Python fallback, not crash.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Known CVEs (as of this writing)&lt;/td&gt;&lt;td&gt;None at the 3.4.x line. Earlier 2.x line had no security CVEs either.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Reverse dependency in our tree&lt;/td&gt;&lt;td&gt;&lt;code dir=&quot;auto&quot;&gt;requests v2.33.1&lt;/code&gt; — the canonical HTTP library, itself widely audited.&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;Equivalent risk of NOT using it&lt;/td&gt;&lt;td&gt;Rolling our own detector for cp1252/UTF-8/BOM in ~40–80 lines of code. That code becomes our liability: every future failure mode and edge case is on us, and detector code is notoriously error-prone (the original #644 is itself a manifestation of “lazy encoding handling”).&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;
&lt;h4 id=&quot;architect-alphonso-revised-recommendation&quot;&gt;Architect Alphonso revised recommendation&lt;/h4&gt;
&lt;p&gt;&lt;strong&gt;Promote &lt;code dir=&quot;auto&quot;&gt;charset-normalizer&lt;/code&gt; to a direct project dependency&lt;/strong&gt;, pinning a compatible range (e.g., &lt;code dir=&quot;auto&quot;&gt;charset-normalizer&gt;=3.4,&amp;#x3C;4&lt;/code&gt;). Rationale:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Already in supply chain — zero net new install surface.&lt;/li&gt;
&lt;li&gt;Direct dep makes the version contract intentional (no surprise drift if &lt;code dir=&quot;auto&quot;&gt;requests&lt;/code&gt; ever vendors a fork or switches).&lt;/li&gt;
&lt;li&gt;MIT licensing is uncomplicated.&lt;/li&gt;
&lt;li&gt;Pure-Python fallback path eliminates the “C extension surprise” risk.&lt;/li&gt;
&lt;li&gt;Building our own detector for a problem the wider Python ecosystem has already solved is exactly the kind of work #644 keeps producing.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;strong&gt;HiC decision needed:&lt;/strong&gt; approve direct-dep promotion, or override with one of:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;(a) Stick with transitive (don’t pin directly, trust &lt;code dir=&quot;auto&quot;&gt;requests&lt;/code&gt;’s chain). &lt;em&gt;Cost: we can’t intentionally require a known-fixed version when a detector edge case bites us.&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;(b) Hand-roll a minimal detector. &lt;em&gt;Cost: ~80 LOC of new tested code we own forever; no upside given (1) above.&lt;/em&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;more-information&quot;&gt;More Information&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Source bug body: &lt;a href=&quot;https://github.com/Priivacy-ai/spec-kitty/issues/644&quot;&gt;#644&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Code references (read sites, all explicit &lt;code dir=&quot;auto&quot;&gt;read_text(encoding=&quot;utf-8&quot;)&lt;/code&gt; except one &lt;code dir=&quot;auto&quot;&gt;errors=&quot;replace&quot;&lt;/code&gt; in &lt;code dir=&quot;auto&quot;&gt;lint.py:258&lt;/code&gt;):
&lt;ul&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/compiler.py:594&lt;/code&gt; (ingest — proposed retrofit)&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/sync.py:151&lt;/code&gt; (ingest — proposed retrofit)&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/interview.py:283, 398&lt;/code&gt; (ingest — proposed retrofit)&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/context.py:135&lt;/code&gt; (re-read — deferred)&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/hasher.py:33&lt;/code&gt; (re-read — deferred)&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/language_scope.py:46&lt;/code&gt; (re-read — deferred)&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/compact.py:135&lt;/code&gt; (re-read — deferred)&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;src/charter/neutrality/lint.py:258&lt;/code&gt; (special — &lt;code dir=&quot;auto&quot;&gt;errors=&quot;replace&quot;&lt;/code&gt;, deferred)&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;Mission spec FR-016 through FR-019 and NFR-004 in &lt;code dir=&quot;auto&quot;&gt;kitty-specs/review-merge-gate-hardening-3-2-x-01KRC57C/spec.md&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;&lt;code dir=&quot;auto&quot;&gt;charset-normalizer&lt;/code&gt; package: &lt;a href=&quot;https://pypi.org/project/charset-normalizer/&quot;&gt;&lt;code dir=&quot;auto&quot;&gt;https://pypi.org/project/charset-normalizer/&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;</content>
</entry>
