diff --git a/CONTEXT.md b/CONTEXT.md index 61f0b53b..dd5c7acc 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -65,6 +65,21 @@ The user-facing capability for finding either a known paper or knowledge inside ### Retrieval Artifact Regenerable data derived from the Source Corpus for search: FTS indexes, embeddings, vector tables, and their companion metadata. Retrieval Artifacts may be discarded and rebuilt; they are not preservation targets or sources of truth. +### Capability State +The current usability of one PaperForge module. Installation, Library, OCR, Memory, Maintenance, and Help each report their own state; there is no global “setup complete” state. + +### Activity State +Work currently being performed by a module, such as installing, rebuilding, or stopping. Activity is independent of capability: a module may keep a warning while repair is running. + +### Attention State +Whether a module currently needs a user decision or action. Normal capability stays quiet; warnings expose one concrete primary action. + +### Maintenance Inbox +A derived view of actionable stale, failed, or corrupt work reported by the other modules. It is not an independent source of truth, and successful work leaves the inbox. + +### Managed Runtime +The verified, versioned PaperForge execution environment selected by the plugin. Installation, health checks, upgrades, and rollback operate on one runtime identity; commands fail closed when that identity is unknown or stale. + ## Architecture layers ``` diff --git a/PROJECT-MANAGEMENT.md b/PROJECT-MANAGEMENT.md index 7707fa91..cc6842f9 100644 --- a/PROJECT-MANAGEMENT.md +++ b/PROJECT-MANAGEMENT.md @@ -1,13 +1,13 @@ > **Branch:** `feat/ocr-rebuild-ux` | **Last Updated:** 2026-07-15 -> **Active work:** [Wayfinder: Make the PaperForge control center self-explanatory](https://github.com/LLLin000/PaperForge/issues/65). The capability vocabulary (#69), managed-runtime architecture (#70), six-module control-center prototype (#71), and actionable maintenance design (#72) are resolved; integrating the accepted model (#73) is the frontier. +> **Active work:** [Control-center PRD #74](https://github.com/LLLin000/PaperForge/issues/74) is split into eight dependency-linked implementation issues (#75–#82). The first slice, canonical setup/config migration (#75), is implemented and reviewed; Installation capability integration (#76) is the next unblocked issue. > > --- > -> **Current state:** Retrieval recovery is merged and live. OCR multi-key rebuild/redo now streams progress, supports cooperative stop between papers, and refreshes canonical maintenance state. The control-plane audit found four critical contract gaps; the orthogonal capability/activity/attention model and managed-runtime immutable-slot architecture are chosen across #69 and #70. OCR maintenance now uses a canonical per-row action model with confirmation gates. +> **Current state:** Retrieval recovery is merged and live. OCR multi-key rebuild/redo streams progress and refreshes canonical maintenance state. The control-plane capability vocabulary, managed-runtime architecture, prototypes, migration contract, and PRD are resolved. The first production slice now routes bare, headless, and modular setup through one `SetupPlan`, writes schema-v2 `vault_config`, reads legacy path keys only as warned fallback, forwards all configured paths, and returns non-zero on required-step failure. > -> #71 and #72 prototypes are complete: the six-module control-center HTML prototype covers 5 scenarios × 6 modules with independent capability probes, a primary attention zone, responsive single-column layout at 768px, and a plain-button scenario switcher. The maintenance inbox prototype demonstrates actionable-only filtering, single-action rows, inline issue-draft review with local redacted export, and a confirmation-first report flow. Both prototypes were independently reviewed (Critical/Important PASS) and browser-verified at 768px; design decisions documented in `docs/prototypes/`. No production plugin code was changed. +> #71/#72 prototypes and #73 migration contract are complete. PRD #74 defines the six-module production program and #75–#82 form its native dependency graph. Issue #75 passed focused verification and independent spec/quality review. > -> Next: integrate the accepted model into a production control-center plugin [#73](https://github.com/LLLin000/PaperForge/issues/73). +> Next: start [#76](https://github.com/LLLin000/PaperForge/issues/76) in a fresh Matt `/implement` session; do not continue in this context. ## 1. Architecture ### 1.1 The problem (pre-v2) @@ -58,6 +58,7 @@ raw observations → structural signatures → stable anchors/families → zone | Plugin tests + TypeScript + production build | **93 passed; typecheck/build clean** ✅ | | Live Literature-hub maintenance UI | **734 All / 700 Recommended; no captured errors** ✅ | | Maintenance regression action model | **19/19 passed** ✅ (per-row canonical action routing, redo confirmation gate, cache manifest preservation) | +| Canonical setup/config migration (#75) | **61 passed, 0 failed** ✅ (fresh/v1/v2 config, CLI routing, path forwarding, failure exit, idempotent rerun) |

@@ -81,6 +82,7 @@ raw observations → structural signatures → stable anchors/families → zone | **@ deep search** | ✅ deep CLI invocation and result-envelope consumption restored | | **Vector build control** | ✅ canonical SQLite lifecycle and health state restored | | **Control-plane contracts** | ✅ Orthogonal capability/activity/attention model and managed-runtime immutable-slot architecture chosen; documented in `docs/research/2026-07-14-capability-state-action-contract.md` and `docs/research/2026-07-14-managed-runtime-architecture.md` | +| **Setup/config migration** | ✅ Bare, headless, and modular setup share `SetupPlan`; schema-v2 `vault_config` is authoritative with warned v1 read fallback | ### 2.3 Fix Status @@ -116,6 +118,7 @@ raw observations → structural signatures → stable anchors/families → zone | 31 | — | Enable OCR_PIPELINE_V3 by default | Toggle | Full vault corpus diff (555 papers): 547/555 no diff, 5/555 v3 improvements. Set OCR_PIPELINE_V3=0 for legacy | `914acd6` | | 32 | — | OCR rebuild progress + maintenance selection contract | Feature | Added flushed, prefix-separated rebuild/redo streams; full keyed redo; cooperative stop; canonical `needs_derived_rebuild`; All/Recommended filters; selected batch progress UI | `d7b0a527`, `3a516add`, `e556c8ba` | | 33 | — | OCR maintenance canonical per-row action model | Fix | Added `maintenanceActionForRow()` for `display_action`→verb routing, `maintenanceActionRequiresConfirmation()` redo confirmation gate, batch-action filtering by canonical verb, and cache-manifest preservation. Batch actions now follow the canonical backend action instead of raw `can_rebuild`/`can_redo` booleans; destructive redo requires user confirmation; cache refresh preserves the backend manifest. | `d7b0a527` | +| 34 | — | Setup/config paths had conflicting precedence, duplicate engines, dropped user paths, and false-success exits | Migration/prefactor | Unified CLI dispatch on `SetupPlan`; v2-only writes with v1 read fallback; complete path forwarding; visible deprecation; non-zero required-step failures | `af849699`, `7b747423`, `906b3caf` | ## 3. Remaining Issues — Release-Readiness Layers @@ -153,12 +156,12 @@ The OCR maintenance slice has a canonical All/Recommended state model, selected Remaining legacy OCR issues (carried forward): ## 4. Active Queue -1. 🔴 **[Control-center Wayfinder](https://github.com/LLLin000/PaperForge/issues/65)** — map charted, audit and research gates resolved, capability model and runtime architecture chosen. Prototypes completed (#71/#72); integration (#73) is the active frontier. +1. 🔴 **[Control-center PRD #74](https://github.com/LLLin000/PaperForge/issues/74)** — published and split into eight native dependency-linked issues (#75–#82). #75 is complete; #76 is the next unblocked production tracer. 2. ✅ **[Capability-state vocabulary](https://github.com/LLLin000/PaperForge/issues/69)** — resolved at `issuecomment-4971161072`. Orthogonal availability/activity/attention axes, 6-state capability ordinal, 12 canonical verbs, backend-owned severity and primary actions, maintenance projection. 3. ✅ **[Managed runtime](https://github.com/LLLin000/PaperForge/issues/70)** — resolved at `issuecomment-4971239398`. Plugin-managed immutable runtime slots, system-Python bootstrap with validated-triplet fallback, single `active-runtime.json` pointer, `ManagedRuntime` class with `current()`/`status()`/`ensure()`, fail-closed command resolution. 4. ✅ **[Control-center prototype](https://github.com/LLLin000/PaperForge/issues/71)** — resolved with independent Critical/Important PASS review and browser verification at 768px. Six-module control-center HTML prototype covers 5 scenarios with plain-button switcher, primary attention zone, responsive layout, and capability-gated actions. Design decisions recorded in `docs/prototypes/2026-07-14-six-module-control-center.html/.md`. No production implementation before #73. 5. ✅ **[Maintenance prototype](https://github.com/LLLin000/PaperForge/issues/72)** — resolved with independent Critical/Important PASS review and browser verification at 768px. Actionable-only inbox with single-action rows, inline issue-draft review, local redacted export, and confirmation-first report flow. Design decisions recorded in `docs/prototypes/2026-07-14-maintenance-issue-reporting.html/.md`. -6. 🔴 **Control-center integration** — [#73](https://github.com/LLLin000/PaperForge/issues/73) — integrate the accepted prototype model into a production control-center plugin. Build one or more Obsidian plugin settings tabs using the capability-state-action model, managed-runtime contracts, and actionable-only maintenance pattern from #71/#72. +6. ✅ **[Migration and acceptance contract #73](https://github.com/LLLin000/PaperForge/issues/73)** — closed after five-domain code audit, user grilling, and independent acceptance reviews. The implementation frontier moved to PRD #74 and child issues #75–#82. 7. 🟡 **Downstream tooling** — section-aware chunking and separate figure/table handling. 8. ⏳ **Compatibility naming cleanup** — deferred post-release. ### 4.1 Immediate Next Steps @@ -176,7 +179,9 @@ Remaining legacy OCR issues (carried forward): - [x] Add canonical per-row action routing and redo confirmation gates - [x] Prototype six-module control center ([#71](https://github.com/LLLin000/PaperForge/issues/71)) - [x] Design actionable-only maintenance inbox ([#72](https://github.com/LLLin000/PaperForge/issues/72)) -- [ ] Integrate accepted prototype model ([#73](https://github.com/LLLin000/PaperForge/issues/73)) +- [x] Publish PRD #74 and eight agent-ready issues (#75–#82) with native dependencies +- [x] Canonicalize setup/config migration ([#75](https://github.com/LLLin000/PaperForge/issues/75)) — 61 focused tests; spec PASS; quality APPROVED +- [ ] Implement Installation/Help capability tracer in a fresh session ([#76](https://github.com/LLLin000/PaperForge/issues/76)) --- ## 5. Key File Map @@ -330,6 +335,7 @@ Remaining legacy OCR issues (carried forward): | 2026-07-15 | Maintenance inbox is actionable-only; normal and quality-ok items are absent | Routine quality scores create permanent, non-actionable noise. Maintenance shows only stale/failed/corrupt work with a concrete action; unacceptable OCR is reported through a user-reviewed GitHub Issue draft. | | 2026-07-15 | Issue-draft flow is confirmation-first, inline redacted, never auto-submitted | User reviews a prefilled GitHub draft panel inline, edits freely, then clicks "Open GitHub" to create. No token storage, no automatic submission. Docker-style opaque upload rejected. | | 2026-07-15 | Prototype reviews use independent Critical/Important pass/fail gates | Both #71 and #72 prototypes were reviewed by independent reviewer subagents on two dimensions: Critical items (correctness/safety) and Important items (readability/maintainability). All items passed before acceptance. | +| 2026-07-15 | Canonical setup is one `SetupPlan`; configuration writes converge to schema v2 | Duplicate headless/modular/bare engines dropped path inputs and disagreed on success. One engine plus `vault_config`-first reads makes migration observable, idempotent, and reversible through the warned v1 read fallback. | --- @@ -431,6 +437,7 @@ python -m ruff check paperforge/worker/ocr_*.py | 2026-07-14 | Control-center current-contract audit | Inventoried setup, configuration, runtime, readiness, persistence, cache, failure, and recovery contracts across plugin and Python. Closed #66 with a ranked contradiction matrix and preserve/migrate/remove plan; unblocked #69 while #70 remains blocked on desktop/runtime evidence. | [Contract audit](https://github.com/LLLin000/PaperForge/issues/66#issuecomment-4968837257) | | 2026-07-14 | Obsidian and desktop recovery research | Closed #67 and #68 with primary-source pattern reports. Preserved the six-module IA, independent capability probes, Obsidian-managed plugin updates, module-scoped recovery, local diagnostics, and user-reviewed issue drafts; unblocked #70. | [#67](https://github.com/LLLin000/PaperForge/issues/67#issuecomment-4970653461), [#68](https://github.com/LLLin000/PaperForge/issues/68#issuecomment-4970660288) | | 2026-07-15 | Six-module control-center + maintenance inbox prototypes completed | Closed #71 (six-module HTML prototype, 5 scenarios, plain-button switcher, responsive 768px) with independent Critical/Important PASS review and browser verification. Closed #72 (actionable-only inbox, inline issue-draft review, confirmation-first report) with same review gate. Both prototype pairs passed all review dimensions. No production code changed. | `docs/prototypes/2026-07-14-six-module-control-center.{html,md}`, `docs/prototypes/2026-07-14-maintenance-issue-reporting.{html,md}` | +| 2026-07-15 | Control-center PRD split + first production slice | Published PRD #74 and eight native dependency-linked issues (#75–#82). Implemented #75: one SetupPlan for all setup entry points, schema-v2 `vault_config`, warned v1 read fallback, complete path forwarding, visible deprecation, and non-zero required-step failures. Verification: 61/61 focused tests; independent Spec PASS / Quality APPROVED. | [PRD #74](https://github.com/LLLin000/PaperForge/issues/74), [Issue #75](https://github.com/LLLin000/PaperForge/issues/75) | ## 9. Historical Detail Archive diff --git a/docs/research/2026-07-14-wayfinder-migration-acceptance-spec.md b/docs/research/2026-07-14-wayfinder-migration-acceptance-spec.md new file mode 100644 index 00000000..ca6e3e61 --- /dev/null +++ b/docs/research/2026-07-14-wayfinder-migration-acceptance-spec.md @@ -0,0 +1,382 @@ +# Wayfinder #73 — Migration & Acceptance Specification + +**Date:** 2026-07-15 | **Parent:** [Issue #73](https://github.com/LLLin000/PaperForge/issues/73) | **Prerequisites:** #69, #70, #71, #72 + +--- + +## 0. Locked Decisions (from #69/#70, override scout speculation) + +| Domain | Decision | Source | +|--------|----------|--------| +| 维护 is a derived view | No independent probe. Aggregates non-ready states of five modules. | #69 §1 | +| Stale is never ready | TTL-expired envelope renders as `unknown`, never as stored state. | #69 §1 | +| Backend owns facts and actions | Plugin never reclassifies severity, derives actions, or decides maintenance visibility. | #69 §1 | +| Source materials preserved on destructive ops | Redo never deletes raw images/exports/config without confirmation. | #69 §1 | +| API keys migrate to Obsidian SecretStorage | `minAppVersion` becomes 1.11.4. **N:** copy to SecretStorage, readback-verify, then delete `data.json` plaintext. Failed verification retains legacy value as fallback-with-warning. **N+1:** all **plugin** credential reads use SecretStorage exclusively. **N+2:** remove legacy plaintext readers. Headless CLI may continue `.env` inputs via `embedding/_config.py`. | #73 locked (user contract) | +| macOS auto-download disabled until signed | Published checksummed artifacts must pass x64/arm64 smoke gates; no Gatekeeper bypass instructions. | #73 locked (user contract) | +| Flatpak/Snap unsupported in first release | Detect and explain native/AppImage/deb/external Python alternatives. | #73 locked (user contract) | +| Three-release cutover | N: SecretStorage copy-verify-delete, capability envelope, ManagedRuntime with read-only fallbacks; v1 READ fallback active, v1 writes avoided. N+1: all new writes use vault_config only; v1 READ fallback still accepted. N+2: remove v1 reader, old setup_complete, old resolvers, legacy snapshots. | #73 locked (user contract) | +| Python 3.11+ for new/repair runtime | Existing verified 3.10 survives N with deprecation warning. New installs and all repairs use 3.11+. All manual-install instructions say 3.11+. At N+1, 3.10 becomes `needs_repair`. | #73 locked (user contract) | +| Stay on Obsidian 1.11.4, imperative PluginSettingTab | No declarative 1.13 requirement or Workspace View. | #73 locked (user contract) | +| No credentials through ManagedRuntime | Operation subprocesses get only explicit ephemeral secrets; ManagedRuntime never receives them. | #70 §Security | +| Single resolver after Phase 2 | Every command dispatch reads from `runtime.current().pythonPath`. | #70 §Acceptance 9 | +| Orthogonal capability/activity/attention | Activity never masks availability; maintenance is derived projection. | #69 §2 | +| **Managed runtime is machine-local, shared across vaults** | Runtime directory is `~/.paperforge/runtime/{os-arch}/v{version}` outside vault, outside Obsidian Sync. Active pointer file at `~/.paperforge/runtime/active-runtime.json`. One runtime identity per machine shared by all PaperForge vaults. No secrets in runtime paths. | #73 refinement of #70 | + +--- + +## 1. Three-Release Cutover + +| Area | N | N+1 | N+2 (shim removal) | +|------|----|-----|---------------------| +| **Config migration** | v1 fallback reads top-level keys with warning; vault_config authoritative for v2+ | All writes new-only (vault_config). v1 READ fallback still accepted | Delete read-side v1 reader | +| **Plugin settings** | Add `capability_state` map alongside `setup_complete`. Migrate old `true` -> all `unknown`. `_autoUpdate` gated on installation capability, not global bool | `saveSettings` excludes path fields; all callers read from `paperforge.json` | Remove `setup_complete` from I/O; remove rendering | +| **Runtime identity** | `ManagedRuntime` class added. Machine-local `~/.paperforge/runtime/{os-arch}/v{version}`. `status()` reads `active-runtime.json` at `~/.paperforge/runtime/`; absent -> fall back to existing resolvers | Setup wizard uses `runtime.ensure()`. Existing users get one prompt for migration | Remove `resolvePythonExecutable`, `getCachedPython`, `getVectorRuntime`, `checkRuntimeVersion`, `buildRuntimeInstallCommand` | +| **Python min version** | 3.10 verified accepted with deprecation warning; new installs/repairs require 3.11+ | 3.10 becomes `needs_repair`; `ensure()` rejects <3.11 | — | +| **Snapshots** | v2 schema with `runtime_identity`, `config_revision`, `canonical_index_revision`, `source_db_revision`, `ttl_seconds` | Plugin rejects schema<2 or age>TTL as `stale` | Remove v1-only read path | +| **Maintenance cache** | Add `cache_version` field; Python backend owns `action`/`label`/`severity`/`visible` | Delete `categorizeMaintenanceRow` + derived TS types; render from backend fields | Remove cache_version compat | +| **Vector build-state** | Keep legacy JSON read path; all writes go to SQLite `build_state` | Stop writing JSON | Remove `get_vector_build_state_path` | +| **Credentials** | Copy to SecretStorage, readback-verify, then delete `data.json` plaintext. Failed verify retains legacy as fallback-with-warning. Headless CLI may continue `.env`. | All N+1 **plugin** credential reads use SecretStorage only. Headless CLI retains `.env`/env. Legacy fallback removed from plugin | Remove legacy plaintext credential readers from plugin | +| **Setup paths** | Merge headless + modular into one canonical engine. Legacy bare `setup` delegates with deprecation warning. Fix P0 pip-skip, P1 literature-dir drop, P1 JSON exit code | Bare `setup` routes exclusively to canonical modular engine (no error). Legacy TUI inactive | Delete legacy TUI code | +| **Obsidian API** | Stay on 1.11.4 imperative `PluginSettingTab`; Obsidian 1.13+ features MAY appear in research but not required | — | — | +| **Platform detection** | Flatpak/Snap -> explanation + manual alternative. macOS auto-download disabled | Add Flatpak/Snap test if user demand appears | — | + +--- + +## 2. Compatibility Migration Tables + +### 2.1 `paperforge.json` path schema + +| Aspect | Detail | +|--------|--------| +| **Current** | Python: `top_level OVERRIDES vault_config`. Plugin: `vault_config OVERRIDES top_level`. `savePaperforgeJson` deletes top-level path keys. `load_vault_config` strips `schema_version` from output. | +| **Target** | N: `v1` fallback reads top-level keys with warning; `v2` vault_config only. Plugin and Python share same precedence. | +| **Schema version** | `schema_version: 2` in `paperforge.json` (file format). `load_vault_config` returns version with paths. | +| **Rollback** | v1 READ fallback active N and N+1; removed at N+2. Starting N+1, all writes are vault_config-only. Downgraded plugin reads vault_config-only file (reads v1 if present). | +| **Test seam** | `unit/config/test_load_vault_config.py` — v1 vs v2 fixture. `unit/plugin/test_readPaperforgeJson.ts` — same. | +| **Files** | `paperforge/config.py:166-337` (load_vault_config, paperforge_paths). `plugin/src/main.ts:224-300` (readPaperforgeJson, savePaperforgeJson). `plugin/src/services/memory-state.ts:79-116` (readPathConfig). `paperforge.json` (root config). | + +### 2.2 Global `setup_complete` -> per-module capability_state + +| Aspect | Detail | +|--------|--------| +| **Current** | `PaperForgeSettings.setup_complete` bool at `constants.ts:73`. Invalidated only when `paperforge.json` missing (`settings.ts:191-196`). Gates `_autoUpdate` (`main.ts:78`). Contradictory wizard validation (`step3-gate.ts:3-10` vs `modals.ts:766-775`). | +| **Target** | `capabilityState: Record` alongside `setup_complete` in N. Migrate old `true` -> all `unknown`. `_autoUpdate` gated on `installation.capability_state`. | +| **Versioning** | Plugin `data.json` schema version tracked internally. Migration runs once on load. | +| **Idempotence** | Re-migration at same state writes same capabilityState. | +| **Rollback** | **Most dangerous** — changes auto-update gate. Mitigation: `override_to_ready` escape hatch in plugin settings on N. | +| **Test seam** | `unit/plugin/test_capability_state.ts` (load old data.json). `unit/plugin/test_auto_update_gate.ts`. | +| **Files** | `plugin/src/constants.ts:69-113` (PaperForgeSettings). `plugin/src/main.ts:78,307-325` (loadSettings, _autoUpdate). `plugin/src/settings.ts:180-230` (render setup_complete). `plugin/src/views/modals.ts:617-775` (Wizard). `plugin/src/views/step3-gate.ts:3-10`. | + +### 2.3 Duplicate Python resolvers -> ManagedRuntime +| Aspect | Detail | +|--------|--------| +| **Current** | `resolvePythonExecutable` (python-bridge.ts:62-117) — one-shot. `getCachedPython` (memory-state.ts:274-303) — process-global cache, Windows-only venv candidates. `getVectorRuntime` (memory-state.ts:173-221) — independent Windows-only crawl. All non-functional on POSIX. | +| **Target** | `ManagedRuntime` class (plugin/src/services/managed-runtime.ts). Directory: `~/.paperforge/runtime/{os-arch}/v{version}` outside vault. Single pointer at `~/.paperforge/runtime/active-runtime.json`. `current()` sync, fails closed. `status()` async probe. `ensure()` builds/verifies slots. N: coexists, absent -> current resolver. N+1: setup uses ensure(). N+2: old resolvers removed. | +| **Versioning** | `active-runtime.json` schema_version. `status()` returns `stale: true` when cache outside TTL. | +| **Idempotence** | Deterministic given same filesystem and perms. Caching prevents re-resolution. | +| **Rollback** | Manual `python_path` override still takes priority. Old slots kept (2) before cleanup. | +| **Test seam** | `unit/plugin/test_managed_runtime.ts` (mocked fs+exec). `unit/plugin/test_resolver_cache_invalidation.ts`. `integration/test_runtime_lifecycle.py`. `e2e/test_runtime_rollback.py`. | +| **Files** | `plugin/src/services/python-bridge.ts:62-117`. `plugin/src/services/memory-state.ts:173-303`. `~/.paperforge/runtime/` (machine-local runtime dir). `.worktrees/retrieval-recovery/docs/research/2026-07-14-managed-runtime-architecture.md §Interface §Directory Layout §Migration`. | +### 2.4 Memory/vector snapshot freshness + +| Aspect | Detail | +|--------|--------| +| **Current** | `state_snapshot.py:17-71` — schema_version=1, updated_at only. Plugin `buildSnapshot` (memory-state.ts:305-353) — no freshness check. Stale green renders as `ready`. | +| **Target** | v2 schema with `runtime_identity`, `config_revision`, `canonical_index_revision`, `source_db_revision`, `computed_at`. Plugin rejects schema<2 or age>TTL as `stale`. Backfill v1 on first read. | +| **Versioning** | `schema_version` gates check. Plugin rejects <2 -> stale -> triggers refresh. | +| **Idempotence** | Overwrite-always write. Same inputs -> same snapshot (modulo timestamps). | +| **Rollback** | Backfill corruption -> stale triggers re-probe; safe. | +| **Test seam** | `unit/embedding/test_snapshot_v2.py`. `unit/plugin/test_snapshot_freshness.ts`. `integration/test_snapshot_lifecycle.py`. | +| **Files** | `paperforge/memory/state_snapshot.py:17-71`. `plugin/src/services/memory-state.ts:305-380` (buildSnapshot, isMemoryReady, isVectorReady). | + +### 2.5 OCR maintenance cache + +| Aspect | Detail | +|--------|--------| +| **Current** | `ocrMaintenanceCachePath` (ocr-maintenance-ui.ts:127-134) hardcodes `System/PaperForge/cache/`. `categorizeMaintenanceRow` (62-118) type-over-type duplication of backend logic. `MaintenanceCache` (37-40) has no version field. | +| **Target** | Path uses `resolveVaultPaths()` for correct paperforge dir + `cache/ocr_maintenance.json`. Delete `categorizeMaintenanceRow` + derived types. Add `cache_version` field to `MaintenanceCache`. Render from backend fields. | +| **Versioning** | `cache_version` integer in cache. Bump on display schema changes. Miss -> full refresh. | +| **Idempotence** | Overwrite-always cache write. Same backend data -> same cache (modulo timestamps). | +| **Rollback** | Old cache at wrong path -> stale -> full refresh, slightly slower first load. Old plugin reads old hardcoded path (no effect). `cache_version` gate ensures old cache schema rejected. | +| **Test seam** | `unit/plugin/test_maintenance_cache_path.ts`. `unit/plugin/test_maintenance_display.ts`. `unit/plugin/test_cache_version.ts`. `unit/plugin/test_incremental_refresh.ts`. | +| **Files** | `plugin/src/services/ocr-maintenance-ui.ts:37-243`. `paperforge/worker/ocr_maintenance.py:284-393` (compute_maintenance_manifest, collect_maintenance_rows). | + +### 2.6 Embedding credential resolution + +| Aspect | Detail | +|--------|--------| +| **Credential handoff protocol** | Plugin launcher reads SecretStorage immediately before an OCR or Memory subprocess, passes the single required token in an allowlisted ephemeral env var (`PADDLEOCR_API_TOKEN`, `VECTOR_DB_API_KEY`). Token is NEVER forwarded to `ManagedRuntime.ensure()/status()/current()`, NEVER persisted to logs, NEVER included in diagnostic dumps, NEVER injected into child-process env beyond the single targeted subprocess. Headless CLI independently reads explicit env/`.env` via `embedding/_config.py` (no SecretStorage path). | +| **Target (N)** | `redact_env()` filter on child process env (strip `PADDLEOCR_*`, `VECTOR_DB_*`, `OPENAI_*` patterns). Remove explicit `VECTOR_DB_API_KEY` from child env. Add `credential_source` field. **SecretStorage migration:** copy from data.json/.env to SecretStorage, readback-verify the stored value against original, then delete data.json plaintext. If readback fails (wrong value returned), retain legacy value as fallback-with-warning. | +| **Target (N+1)** | All **plugin** credential reads use SecretStorage exclusively. Legacy `.env`/data.json readers in plugin code removed. Headless CLI retains `.env`/env inputs via `embedding/_config.py` (unchanged). `credential_last_4_for_verification` only in diagnostics. | +| **Target (N+2)** | Remove legacy plaintext credential readers from plugin entirely. | +| **Versioning** | Migration warning if both `.env` and SecretStorage have key (indicates incomplete migration). | +| **Idempotence** | Re-running migration at same state: SecretStorage has value, data.json cleanly deleted. Safe re-run. | +| **Test seam** | `unit/plugin/test_credential_redaction.ts`. `integration/test_credential_migration.py` (copy->verify->delete). `unit/plugin/test_child_env_filter.ts`. `unit/plugin/test_secret_storage_readback.ts`. `integration/test_credential_handoff_ephemeral.py` — verify ephemeral env allowlist, no leak to ManagedRuntime, no log/diag persistence | +| **Files** | `paperforge/embedding/_config.py:8-29`. `plugin/src/services/python-bridge.ts:469-482` (paperforgeEnrichedEnv). `plugin/src/settings.ts:1251-1254`. `plugin/src/views/modals.ts:718-750` (diagnostic copy). `tests/conftest.py:64-69` (test .env). | + +| Aspect | Detail | +|--------|--------| +| **Current** | Three setup implementations (headless, modular, TUI). P0: source-checkout headless skips pip (`setup_wizard.py:774-817`). P1: `--literature-dir` silently dropped (`cli.py:749-762`). P1: Modular JSON exits 0 despite failures (`setup/plan.py:81-84`). P1: Modular directory semantics differ from canonical path builder. | +| **Target** | Merge headless + modular into one canonical engine. N: legacy bare `setup` delegates with deprecation warning — paves path. N+1: bare `setup` routes exclusively to canonical modular engine (no error). N+2: delete legacy TUI code. Fix P0 pip-skip, P1 literature-dir drop, P1 JSON exit code throughout. | +| **Versioning** | CLI: bare `setup` delegates/warns in N; routes to canonical engine in N+1; TUI removed N+2. All steps data-op idempotent. | +| **Idempotence** | Re-running after fix correctly installs runtime. Setup already incremental. | +| **Rollback** | New validation may be too strict (network unreachable), but clear error is correct behavior per #66. Legacy TUI unavailable after N+1 upgrade; prepare users. | +| **Test seam** | `integration/test_headless_setup.py` (no network -> non-zero exit). `unit/test_cli_dispatch.py` (literature_dir forwarded). `unit/setup/test_plan_json_exit.py`. `e2e/test_setup_cli.py`. | +| **Files** | `paperforge/setup_wizard.py:428-865`. `paperforge/setup/plan.py:19-96`. `paperforge/cli.py:733-766`. `paperforge/config.py:166-337` (paperforge_paths). | + +--- + +## 3. Security & Privacy Threat Gates + +| Gate | Requirement | Blocking? | Test | +|------|-------------|-----------|------| +| SG-01 | Strips `PADDLEOCR_*`, `VECTOR_DB_*`, `OPENAI_*` before all non-targeted subprocesses (pip install, ManagedRuntime ensure/status, diagnostic probes, etc.). Does NOT strip the single allowlisted token from its one targeted OCR/Memory subprocess per §2.6 handoff protocol. | **BLOCKING** (N) | `unit/plugin/test_child_env_filter.ts` — verify env stripped for install/ensure/diag but allowlisted token reaches its target subprocess | +| SG-02 | **Forbidden:** broad `process.env` forwarding to all child processes (e.g. `paperforgeEnrichedEnv()` before pip install); injecting credentials into unrelated subprocesses. **Allowed:** single SecretStorage-derived token in allowlisted env var (`PADDLEOCR_API_TOKEN`, `VECTOR_DB_API_KEY`) for the one targeted OCR/Memory subprocess, immediately before launch, per §2.6 handoff protocol. Headless CLI reads env/`.env` independently via `_config.py`. | **BLOCKING** (N) | `unit/plugin/test_child_env_allowlist.ts` — verify only allowlisted vars reach their target subprocess; no credential in pip/install subprocess; headless CLI path unchanged | +| SG-03 | Diagnostic clipboard / export redacts all credential values, vault path, Zotero path, paper titles to `[redacted]`. | **BLOCKING** (N) | `unit/plugin/test_diagnostic_redaction.ts` — all patterns redacted | +| SG-04 | Issue draft modal: editable title, redacted fields, no auto-submit, no token in URL, manual GitHub open (`window.open`, not `window.location`). | **BLOCKING** (N) | `integration/test_issue_draft_privacy.py` — URL parse + no-auto-upload + DOM | +| SG-06 | `ManagedRuntime.ensure()/status()/current()` never receives credentials. Plugin passes token via allowlisted ephemeral child-process env var immediately before OCR/Memory subprocess only. Headless CLI reads env/`.env`. Machine-local runtime path `~/.paperforge/runtime/` contains no secrets. | **BLOCKING** (N) | `integration/test_runtime_no_credential_leak.py`. `integration/test_credential_handoff_ephemeral.py` — verify token present in OCR subprocess env but absent from ManagedRuntime calls, logs, and diagnostic output | +| SG-07 | Bootstrap Python path never stored — only venv path from `active-runtime.json`. | BLOCKING (N) | `unit/plugin/test_active_runtime_no_bootstrap_path.ts` | +| SG-08 | `~/.paperforge/runtime/active-runtime.json` stores version metadata and pythonPath only — no credentials, no vault paths, no bootstrap-Python path. | BLOCKING (N) | `unit/plugin/test_active_runtime_contents.ts` — JSON shape check | +| SG-09 | OCR upload consent modal (`PaperForgeOcrPrivacyModal`) remains as-is; issue draft consent explicit before browser open. | BLOCKING (N) | `e2e/test_ocr_upload_consent.py` (manual check: modal present) | +| SG-10 | No FileHandler logger; stderr-only StreamHandler; JSONL logs (reading-log, project-log, correction-log) contain paper keys only, no credential values. | Post-release audit | `unit/test_logging_no_file_handler.py` + code review | + +**Rejected (scout speculation, overridden by locked decisions):** +- "Use OS keychain for credentials" — locked decision says Obsidian SecretStorage (1.11.4), not OS keychain. (#73 contract) +- "Flatpak credential isolation" — Flatpak/Snap explicitly unsupported in first release. (#73 contract) + +--- + +## 4. Platform Support Matrix + +| Triplet | Bootstrap | Auto-Download | Release Gate | macOS Signing | Notes | +|---------|-----------|---------------|--------------|---------------|-------| +| **win-x64** | `py -3.11` -> `py -3` -> registered Python | Published, checksummed, CI-smoke-tested | **Gate: pointer atomicity on win-x64** | N/A | LongPathsEnabled may be needed | +| **macos-x64** | `/usr/bin/python3` -> Homebrew -> `which python3` | **Disabled until signed/notarized** | **Gate: signed artifact + pointer atomicity on macos-x64** | Required ($99/yr) | No Gatekeeper bypass. Manual Python: install 3.11+ | +| **macos-arm64** | Same as macos-x64 | **Disabled until signed/notarized** | **Gate: signed artifact + pointer atomicity on macos-arm64** | Required ($99/yr) | x64/arm64 smoke gates required before enable | +| **linux-x64** | `/usr/bin/python3` -> `$PATH python3` | Published, checksummed, CI-smoke-tested | **Gate: pointer atomicity on linux-x64** | N/A | `python3-venv` separate on Debian/Ubuntu | +| **win-arm64** | Not available from upstream | Not available | Not validated | N/A | Manual: install Python 3.11+ from python.org | +| **linux-arm64** | Published, not CI-smoke-tested | Not validated | Not validated | N/A | Manual: install Python 3.11+ via package manager | +| Flatpak/Snap | Detect -> explain native alternatives | N/A | **First release: unsupported** | N/A | Detect `container`/`flatpak` env indicator. Manual: install Python 3.11+ natively | + +**MacOS signing release gate** (blocking before auto-download enable): +1. Apple Developer Program enrollment ($99/yr) confirmed. +2. Notarization workflow built into CI pipeline. +3. Downloaded python-build-standalone binary signed and notarized. +4. x64 and arm64 smoke tests pass in CI. +5. Checksums published in release-owned checksums file. +6. Verification test: `codesign -dv` on artifact, `spctl --assess --type execute` on first launch. + +**Cross-platform pointer atomicity gate** (blocking per triplet, before that triplet is supported): +On every supported triplet, `write-temp-file + rename` must be verified atomic in CI before the triplet is marked supported. +Runs on the machine-local path `~/.paperforge/runtime/` (same-filesystem rename guaranteed on NTFS/APFS/ext4). + +**Platform test gaps (post-release hardening, not blocking):** +- E2E on Windows/macOS CI (currently ubuntu-only). (PlatformAcceptance §Finding) +- Chaos tests on Windows/macOS. (PlatformAcceptance §Finding) +- Path-with-spaces unit test (zero coverage today). (PlatformAcceptance §Test Gaps) +- `MAX_PATH` / LongPathsEnabled acceptance on Windows. (PlatformAcceptance §Test Gaps) +- Flatpak/Snap vault write access test. (PlatformAcceptance §Unresolved 3) + +--- + +## 5. Accessibility Gates + +| Gate | Requirement | Source | Method | +|------|-------------|--------|--------| +| AG-01 | Module cards use plain `