Spec Kitty

└─ kitty-specs
   └─ Frontmatter History to Canonical JSONL

Mission Run:

📚 Docs ↗

Work Packages: Frontmatter History to Canonical JSONL

Inputs: Design documents from kitty-specs/035-frontmatter-history-to-canonical-jsonl/ Prerequisites: plan.md (required), spec.md (user stories), research.md, data-model.md, quickstart.md

Tests: Tests are included as separate work packages to validate each layer independently before integration.

Organization: Fine-grained subtasks (Txxx) roll up into work packages (WPxx). Each work package is independently deliverable and testable.

Prompt Files: Each work package references a matching prompt file in tasks/ generated by /spec-kitty.tasks.

Work Package WP01: History Parser Module (Priority: P0)

Goal: Create src/specify_cli/status/history_parser.py with the full transition reconstruction algorithm: normalize history entries, collapse duplicates, pair transitions, gap-fill, and extract DoneEvidence. Independent Test: Module imports cleanly. Functions can be called with sample frontmatter dicts and return correct TransitionChain objects. Prompt: tasks/WP01-history-parser-module.md Estimated Size: ~450 lines

Included Subtasks

✅ T001 Create NormalizedHistoryEntry and Transition dataclasses
✅ T002 Implement normalize_entries() for Format A history arrays
✅ T003 Implement collapse_duplicates() to remove consecutive same-lane entries
✅ T004 Implement pair_transitions() for adjacent entry pairing
✅ T005 Implement gap_fill() to bridge last history lane to current frontmatter lane
✅ T006 Implement extract_done_evidence() from frontmatter fields
✅ T007 Implement build_transition_chain() orchestrator function

Implementation Notes

All functions are pure (no I/O, no side effects) — easy to test in isolation
Use resolve_lane_alias() from transitions.py for alias normalization
Actor resolution: use entries[i+1].agent for transition to entries[i+1].lane, fallback to "migration"
DoneEvidence only when review_status == "approved" AND reviewed_by present

Parallel Opportunities

T001 (dataclasses) must come first; T002-T006 can be developed in any order
T007 integrates all, must come last

Dependencies

None (foundation work package)

Risks & Mitigations

Risk: Edge cases in history parsing (empty, single-entry, malformed). Mitigation: Thorough fallback paths defined in plan algorithm section.

Work Package WP02: History Parser Tests (Priority: P0)

Goal: Create tests/specify_cli/status/test_history_parser.py with comprehensive unit tests for the parser module. Validates the reconstruction algorithm before it's wired into the migration engine. Independent Test: pytest tests/specify_cli/status/test_history_parser.py -v passes with all scenarios covered. Prompt: tasks/WP02-history-parser-tests.md Estimated Size: ~400 lines

Included Subtasks

✅ T008 Tests for normalize_entries (Format A, alias resolution, missing fields, empty history)
✅ T009 Tests for collapse_duplicates (consecutive dupes, no dupes, all same)
✅ T010 Tests for pair_transitions (multi-entry, single entry)
✅ T011 Tests for gap_fill (gap needed, no gap, planned-only)
✅ T012 Tests for extract_done_evidence (approved+reviewed_by, has_feedback, missing fields)
✅ T013 [P] Tests for build_transition_chain integration (multi-step, single-entry, empty, planned-only)

Implementation Notes

Each test function corresponds to a plan test case (T080-T091)
Use inline frontmatter dicts as test data (no file I/O needed for parser tests)
Validate both the transition list and metadata (history_entries count, has_evidence flag)

Parallel Opportunities

T008-T012 are independent test groups that can be written in any order
T013 integration tests should come after individual function tests

Dependencies

Depends on WP01 (parser module must exist for tests to import)

Risks & Mitigations

Risk: Test data doesn't match real WP files. Mitigation: Use examples from actual kitty-specs WP files discovered during research.

Work Package WP03: Migration Engine Rewrite (Priority: P0)

Goal: Rewrite src/specify_cli/status/migrate.py to replace bootstrap-only logic with full history reconstruction, 3-layer idempotency, atomic writes, backup safety, and post-migration materialization. Independent Test: migrate_feature() on a test fixture with multi-step history produces the correct number of events with force=true, and re-running produces no changes. Prompt: tasks/WP03-migration-engine-rewrite.md Estimated Size: ~500 lines

Included Subtasks

✅ T014 Update WPMigrationDetail dataclass (events_created, event_ids, history_entries, has_evidence)
✅ T015 Implement _write_events_atomic() helper (temp file + os.replace)
✅ T016 Implement _check_idempotency() with 3-layer logic (marker, live-events, migration-actor-only)
✅ T017 Implement _backup_events_file() for replace-once path
✅ T018 Rewrite migrate_feature() core loop using history_parser.build_transition_chain()
✅ T019 Add post-migration materialize() call
✅ T020 Update FeatureMigrationResult to track replace/backup actions

Implementation Notes

Atomic write mirrors reducer.py's materialize() pattern: write to .tmp, then os.replace()
3-layer check order: (1) marker → skip, (2) live events → skip, (3) migration-only → backup+replace
All StatusEvent objects built directly (NOT via emit_status_transition)
All events: force=true, reason="historical migration" or "historical_frontmatter_to_jsonl:v1" (first event per WP)
Call materialize(feature_dir) once after all events written

Parallel Opportunities

T014, T015, T016, T017 are independent helper implementations
T018 integrates all helpers, T019-T020 extend the result

Dependencies

Depends on WP01 (imports build_transition_chain from history_parser)

Risks & Mitigations

Risk: Breaking existing tests (force=False → force=True). Mitigation: WP04 updates all existing test assertions.
Risk: Atomic write failure leaves .tmp file. Mitigation: Clean up .tmp in finally block.

Work Package WP04: Migration Test Updates (Priority: P1)

Goal: Update tests/specify_cli/status/test_migrate.py — fix all 29 existing tests for the new force=true/multi-event behavior, and add new tests for the 3-layer idempotency, backup, materialization, and atomic write. Independent Test: pytest tests/specify_cli/status/test_migrate.py -v passes with all existing + new tests. Prompt: tasks/WP04-migration-test-updates.md Estimated Size: ~500 lines

Included Subtasks

✅ T021 Update existing TestMigrateFeature tests for force=True and multi-event output
✅ T022 Update TestAliasResolution and TestIdempotency for new behavior
✅ T023 Add test_marker_idempotency (events with marker reason → skip)
✅ T024 Add test_live_events_skip (non-migration actors → skip)
✅ T025 Add test_migration_only_replace (all migration actors → backup + replace)
✅ T026 Add test_materialization_after_migrate (status.json exists and valid)
✅ T027 Update TestMigrateCLI and TestMigrationResultJSON for new output schema

Implementation Notes

Key change: all assertions checking force is False must become force is True
New events have reason field set (was None before)
WPMigrationDetail has new fields: events_created, event_ids, history_entries, has_evidence
Old event_id field replaced by event_ids list
CLI JSON output schema adds new fields

Parallel Opportunities

T021-T022 (update existing) and T023-T026 (new tests) can be worked independently
T027 (CLI tests) can be done last

Dependencies

Depends on WP03 (tests exercise the rewritten migrate_feature)

Risks & Mitigations

Risk: Many test updates needed (29 existing tests). Mitigation: Most changes are mechanical (force=False→True, event_id→event_ids).

Work Package WP05: Upgrade Migration Wrapper (Priority: P1)

Goal: Create src/specify_cli/upgrade/migrations/m_2_0_0_historical_status_migration.py — a BaseMigration subclass that wires migrate_feature() into the spec-kitty upgrade framework. Independent Test: Migration can be imported, registered, and its detect/apply methods work against test fixtures. Prompt: tasks/WP05-upgrade-migration-wrapper.md Estimated Size: ~300 lines

Included Subtasks

✅ T028 Create BaseMigration subclass with migration_id = "2.0.0_historical_status_migration"
✅ T029 Implement detect() — scan kitty-specs for features with WPs but no full-history events
✅ T030 Implement can_apply() — verify status module importable, kitty-specs exists
✅ T031 Implement apply() — iterate features, call migrate_feature, aggregate MigrationResult

Implementation Notes

Import pattern: from specify_cli.status.migrate import migrate_feature, MigrationResult
detect() returns True if ANY feature has WP files but no events (or only migration-actor events)
apply() processes features independently; one failure doesn't abort
apply() returns MigrationResult(success=True/False, changes_made=[...], errors=[...])
dry_run parameter passed through to migrate_feature

Parallel Opportunities

None (sequential implementation within this WP)

Dependencies

Depends on WP03 (wrapper calls migrate_feature which must have the new behavior)

Risks & Mitigations

Risk: Migration registry collision. Mitigation: Unique ID following naming convention.
Risk: Version ordering. Mitigation: target_version="2.0.0" is higher than all existing 0.x migrations.

Work Package WP06: Upgrade Wrapper Tests (Priority: P2)

Goal: Create tests/specify_cli/upgrade/test_historical_status_migration.py — integration tests for the upgrade wrapper including detect, apply, dry-run, and cross-branch idempotency simulation. Independent Test: pytest tests/specify_cli/upgrade/test_historical_status_migration.py -v passes. Prompt: tasks/WP06-upgrade-wrapper-tests.md Estimated Size: ~350 lines

Included Subtasks

✅ T032 Test detect() with unmigrated features (returns True)
✅ T033 Test detect() with all features already migrated (returns False)
✅ T034 Test apply() calls migrate_feature and aggregates results correctly
✅ T035 Test apply() dry_run produces no files
✅ T036 Test cross-branch idempotency (run migration twice, second returns no changes)

Implementation Notes

Use tmp_path fixtures with synthetic feature dirs containing WP files
Reuse _write_wp helper pattern from test_migrate.py
Cross-branch test: run apply(), then manually mark migration_id in metadata, run apply() again → no-op

Parallel Opportunities

T032-T035 are independent test cases
T036 is the integration scenario, best written last

Dependencies

Depends on WP05 (tests import the wrapper class)

Risks & Mitigations

Risk: Test isolation (metadata.yaml shared state). Mitigation: Each test uses its own tmp_path.

Dependency & Execution Summary

WP01 (parser) ─────────────────┐
  ├→ WP02 (parser tests)       │ Wave 2 (parallel)
  └→ WP03 (engine rewrite) ────┘
       ├→ WP04 (engine tests)  ─┐ Wave 3 (parallel)
       └→ WP05 (upgrade wrapper)┘
            └→ WP06 (wrapper tests)  Wave 4

Wave 1: WP01 (foundation, no dependencies)
Wave 2: WP02 + WP03 (parallel — tests validate parser while engine rewrite proceeds)
Wave 3: WP04 + WP05 (parallel — engine tests + wrapper implementation)
Wave 4: WP06 (final tests, after wrapper is ready)

Parallelization: Up to 2 agents can work simultaneously in waves 2 and 3.

MVP Scope: WP01 + WP03 = functional migration engine. WP02 + WP04 = validated. WP05 + WP06 = upgrade integration.

Subtask Index (Reference)

Subtask ID	Summary	Work Package	Priority	Parallel?
T001	Create NormalizedHistoryEntry and Transition dataclasses	WP01	P0	No
T002	Implement normalize_entries()	WP01	P0	Yes
T003	Implement collapse_duplicates()	WP01	P0	Yes
T004	Implement pair_transitions()	WP01	P0	Yes
T005	Implement gap_fill()	WP01	P0	Yes
T006	Implement extract_done_evidence()	WP01	P0	Yes
T007	Implement build_transition_chain() orchestrator	WP01	P0	No
T008	Tests: normalize_entries	WP02	P0	Yes
T009	Tests: collapse_duplicates	WP02	P0	Yes
T010	Tests: pair_transitions	WP02	P0	Yes
T011	Tests: gap_fill	WP02	P0	Yes
T012	Tests: extract_done_evidence	WP02	P0	Yes
T013	Tests: build_transition_chain integration	WP02	P0	Yes
T014	Update WPMigrationDetail dataclass	WP03	P0	Yes
T015	Implement _write_events_atomic()	WP03	P0	Yes
T016	Implement _check_idempotency()	WP03	P0	Yes
T017	Implement _backup_events_file()	WP03	P0	Yes
T018	Rewrite migrate_feature() core loop	WP03	P0	No
T019	Add post-migration materialize() call	WP03	P0	No
T020	Update FeatureMigrationResult	WP03	P0	Yes
T021	Update TestMigrateFeature for force=True	WP04	P1	Yes
T022	Update TestAliasResolution and TestIdempotency	WP04	P1	Yes
T023	Add test_marker_idempotency	WP04	P1	Yes
T024	Add test_live_events_skip	WP04	P1	Yes
T025	Add test_migration_only_replace	WP04	P1	Yes
T026	Add test_materialization_after_migrate	WP04	P1	Yes
T027	Update CLI and JSON output tests	WP04	P1	Yes
T028	Create BaseMigration subclass	WP05	P1	No
T029	Implement detect()	WP05	P1	No
T030	Implement can_apply()	WP05	P1	No
T031	Implement apply()	WP05	P1	No
T032	Test detect() unmigrated	WP06	P2	Yes
T033	Test detect() all migrated	WP06	P2	Yes
T034	Test apply() aggregation	WP06	P2	Yes
T035	Test apply() dry_run	WP06	P2	Yes
T036	Test cross-branch idempotency	WP06	P2	No

Canonical Status (Generated)

WP01: done
WP02: done
WP03: done
WP04: done
WP05: done
WP06: done