---
phase: 23-canonical-asset-index-safe-rebuilds
plan: 03
type: execute
wave: 3
depends_on:
  - 23-02
files_modified:
  - paperforge/worker/ocr.py
  - paperforge/worker/deep_reading.py
  - paperforge/worker/repair.py
  - paperforge/worker/sync.py
  - tests/test_asset_index_integration.py
autonomous: true
requirements:
  - ASSET-03
  - ASSET-04
  - MIG-02

must_haves:
  truths:
    - "After OCR completes, incremental refresh is called for the affected zotero_key instead of full rebuild"
    - "After deep-reading status sync, incremental refresh is called for the affected zotero_key"
    - "After repair operations, incremental refresh is called for affected keys"
    - "Incremental refresh is triggered by default; full rebuild only runs when --rebuild-index is passed or schema_version mismatches"
  artifacts:
    - path: "paperforge/worker/ocr.py"
      provides: "Post-OCR incremental refresh"
      modifies: "call site _sync.run_index_refresh -> asset_index.refresh_index_entry"
    - path: "paperforge/worker/deep_reading.py"
      provides: "Post-deep-reading incremental refresh"
      adds: "asset_index.refresh_index_entry call after status sync"
    - path: "paperforge/worker/repair.py"
      provides: "Post-repair incremental refresh"
      adds: "asset_index.refresh_index_entry call after repair"
    - path: "paperforge/worker/sync.py"
      provides: "Default incremental path in run_index_refresh"
    - path: "tests/test_asset_index_integration.py"
      provides: "Integration tests for full cycle"
      min_lines: 80
  key_links:
    - from: "ocr.py :: OCR loop end"
      to: "asset_index.py :: refresh_index_entry(vault, ocr_key)"
      via: "incremental refresh per OCR'd zotero_key"
    - from: "deep_reading.py :: run_deep_reading loop"
      to: "asset_index.py :: refresh_index_entry(vault, dr_key)"
      via: "incremental refresh after status sync per key"
    - from: "repair.py :: repair operations"
      to: "asset_index.py :: refresh_index_entry(vault, repaired_key)"
      via: "incremental refresh after fixing paths/states"

---

<objective>
Wire incremental index refresh into all worker operations that modify paper state (OCR, deep-reading, repair) so they update only the affected index entries instead of rebuilding the entire index. Make the default sync path use incremental refresh on individual keys rather than full rebuild.

Purpose: Workers that modify a single paper's state (OCR completion, deep-reading status sync, path repair) should NOT trigger a complete index rebuild. Incremental refresh by key is faster, safer, and avoids unnecessary writes.

Output: All four worker call sites updated + integration tests.
</objective>

<execution_context>
@C:/Users/Lin/.opencode/get-shit-done/workflows/execute-plan.md
</execution_context>

<context>
@.planning/STATE.md
@.planning/ROADMAP.md (Phase 23)
@.planning/phases/23-canonical-asset-index-safe-rebuilds/23-CONTEXT.md (D-11)
@.planning/phases/23-canonical-asset-index-safe-rebuilds/23-01-PLAN.md (asset_index.py — build_index, refresh_index_entry)
@paperforge/worker/asset_index.py (full file — build_index, refresh_index_entry)
@paperforge/worker/ocr.py (line 1813 — current _sync.run_index_refresh(vault) post-OCR)
@paperforge/worker/deep_reading.py (lines 25-137 — run_deep_reading, status sync loop)
@paperforge/worker/repair.py (lines 1-30 — imports, repair operations overview)
@paperforge/worker/sync.py (lines 1669-1792 — run_index_refresh, rebuild_index parameter)
</context>

<tasks>

<task type="auto">
  <name>Task 1: Switch OCR post-processing from full rebuild to incremental refresh by key</name>
  <files>paperforge/worker/ocr.py</files>

  <read_first>
  @paperforge/worker/ocr.py (lines 1806-1817 — the post-OCR sync+index block)
  @paperforge/worker/ocr.py (lines 1-30 — existing imports)
  @.planning/phases/23-canonical-asset-index-safe-rebuilds/23-CONTEXT.md (D-11)
  </read_first>

  <action>
  In `paperforge/worker/ocr.py`, modify the post-OCR index refresh block (currently lines 1811-1813):

  **Current code:**
  ```python
  try:
      _sync.run_selection_sync(vault)
      _sync.run_index_refresh(vault)
  except Exception as e:
      logger.error("Post-OCR sync failed: %s", e)
  ```

  **Replace with incremental refresh:**
  ```python
  from paperforge.worker.asset_index import refresh_index_entry

  try:
      _sync.run_selection_sync(vault)
      # Incremental refresh: update only the affected paper(s)
      # The OCR queue may have processed multiple keys; use the last key from context
      # or fall back to full rebuild if no single key is identifiable
      ocr_keys = [r.get("key", "") for r in ocr_queue if r.get("queue_status") == "done"]
      for ocr_key in ocr_keys:
          if ocr_key:
              refresh_index_entry(vault, ocr_key)
      if verbose:
          print(f"ocr: refreshed {len(ocr_keys)} index entries incrementally")
  except ImportError:
      # Fallback: if asset_index module not available (pre-migration), use full rebuild
      _sync.run_index_refresh(vault)
  except Exception as e:
      logger.error("Post-OCR index refresh failed: %s", e)
  ```

  Add the import at the top of the file (with the other imports). The import is:
  ```python
  from paperforge.worker.asset_index import refresh_index_entry
  ```

  **Important:** Keep the existing `_sync.run_selection_sync(vault)` call — that still does the library-record sync. Only the index refresh part changes from full rebuild to incremental.

  **Edge cases:**
  - OCR queue has no "done" items: no refresh needed (the variable `changed` tracks this — only refresh if `changed > 0`)
  - OCR queue is empty: skip refresh entirely
  - `refresh_index_entry` handles legacy format detection internally (falls back to full rebuild)
  </action>

  <verify>
  <automated>grep -n "refresh_index_entry\|oc: refreshed" paperforge/worker/ocr.py</automated>
  </verify>

  <acceptance_criteria>
  - `paperforge/worker/ocr.py` imports `refresh_index_entry` from `paperforge.worker.asset_index`
  - Post-OCR block calls `refresh_index_entry(vault, ocr_key)` instead of `_sync.run_index_refresh(vault)`
  - Fallback to full rebuild exists for pre-migration state (ImportError catch)
  </acceptance_criteria>

  <done>OCR post-processing uses incremental refresh by key instead of full rebuild</done>
</task>

<task type="auto">
  <name>Task 2: Add incremental refresh to deep-reading status sync</name>
  <files>paperforge/worker/deep_reading.py</files>

  <read_first>
  @paperforge/worker/deep_reading.py (lines 25-137 — run_deep_reading function)
  @paperforge/worker/deep_reading.py (lines 1-17 — existing imports)
  @.planning/phases/23-canonical-asset-index-safe-rebuilds/23-CONTEXT.md (D-11)
  </read_first>

  <action>
  In `paperforge/worker/deep_reading.py`, add incremental index refresh after the library-record status sync loop.

  1. Add import at the top:
  ```python
  from paperforge.worker.asset_index import refresh_index_entry
  ```

  2. After the loop that syncs `deep_reading_status` in library records (lines 56-66, where `record_path.write_text(...)` is called), add an incremental refresh for each synced key:
  ```python
  # Refresh canonical index for each synced paper
  for record in records:
      key = record["zotero_key"]
      if key:
          try:
              refresh_index_entry(vault, key)
          except Exception:
              logger.warning("Failed to refresh index entry for %s", key)
  ```

  Place this block right after the `synced += 1` loop completes and the report lines are written (after line 79, before the pending_queue reporting block).

  **Important:** Keep the `synced` counter for progress reporting. The index refresh should happen even when `synced == 0` if there are changes in the formal note content that affect the index (e.g., deep_reading_path field).

  **Edge cases:**
  - deep-reading runs but nothing changed (`synced == 0`): still refresh for papers that have pending status but were already up-to-date (needed for deep_reading_path field)
  - `refresh_index_entry` raises: log warning, do not abort the deep-reading worker
  - Multiple papers synced: refresh each one incrementally
  </action>

  <verify>
  <automated>grep -n "refresh_index_entry" paperforge/worker/deep_reading.py</automated>
  </verify>

  <acceptance_criteria>
  - `paperforge/worker/deep_reading.py` imports `refresh_index_entry` from `paperforge.worker.asset_index`
  - After the deep-reading status sync loop, `refresh_index_entry` is called for each paper
  - Failure to refresh logs warning but does not abort
  </acceptance_criteria>

  <done>Deep-reading post-processing calls incremental refresh for each synced paper</done>
</task>

<task type="auto">
  <name>Task 3: Add incremental refresh to repair operations and verify default sync behavior</name>
  <files>
    paperforge/worker/repair.py
    paperforge/worker/sync.py
  </files>

  <read_first>
  @paperforge/worker/repair.py (lines 1-60 — imports, repair function signatures)
  @paperforge/worker/sync.py (lines 1669-1746 — run_index_refresh, rebuild_index parameter)
  @.planning/phases/23-canonical-asset-index-safe-rebuilds/23-CONTEXT.md (D-11)
  </read_first>

  <action>
  **A. Add incremental refresh to repair.py:**

  In `paperforge/worker/repair.py`:
  1. Add import: `from paperforge.worker.asset_index import refresh_index_entry`
  2. Find where repair fixes paths or states for individual papers. The repair operations that modify paper state should trigger incremental refresh.
  3. After each repair action that affects a specific paper (e.g., `--fix-paths` or status repair), call:
     ```python
     try:
         refresh_index_entry(vault, repaired_key)
     except Exception as e:
         logger.warning("Failed to refresh index for %s: %s", repaired_key, e)
     ```

  **B. Verify sync.py default behavior:**

  In `paperforge/worker/sync.py`, verify the `run_index_refresh()` function:
  - The `rebuild_index` parameter (added in Plan 02) controls whether to call `asset_index.build_index()` (full rebuild) or something else
  - Currently `run_index_refresh` always calls `asset_index.build_index()` for the full rebuild path
  - Keep this behavior: when called without args (from CLI without `--rebuild-index`), it still does a full rebuild by default because selection-sync may affect many papers
  - The incremental path is triggered from OCR, deep-reading, and repair (individual paper changes)
  - Add a docstring to `run_index_refresh()` explaining this convention:
    ```python
    """Refresh the canonical asset index.
    
    Default behavior: full rebuild. This is the safe default because
    selection-sync may affect many papers. Workers that modify individual
    papers (ocr, deep-reading, repair) use asset_index.refresh_index_entry()
    for incremental refresh by key.
    
    Args:
        vault: Path to the vault root.
        verbose: If True, print detailed progress.
        rebuild_index: If True, force full rebuild (default: True for sync).
    """
    ```

  **Edge cases:**
  - Repair with no specific paper (library-wide operations): skip incremental refresh, no action needed
  - Sync with `--rebuild-index`: forced full rebuild regardless of current state
  - Sync without `--rebuild-index`: still does full rebuild (safe default for multi-paper changes)
  </action>

  <verify>
  <automated>grep -n "refresh_index_entry" paperforge/worker/repair.py; grep -n "def run_index_refresh" paperforge/worker/sync.py</automated>
  </verify>

  <acceptance_criteria>
  - `paperforge/worker/repair.py` imports `refresh_index_entry`
  - Per-paper repair operations call `refresh_index_entry` after fixing
  - `run_index_refresh` in sync.py has docstring explaining full-rebuild default convention
  </acceptance_criteria>

  <done>Repair wired to incremental refresh, sync.py convention documented</done>
</task>

<task type="auto">
  <name>Task 4: Integration tests for incremental refresh across worker operations</name>
  <files>tests/test_asset_index_integration.py</files>

  <read_first>
  @paperforge/worker/asset_index.py (full file)
  @paperforge/worker/ocr.py (modified — post-OCR incremental refresh)
  @paperforge/worker/deep_reading.py (modified — post-deep-reading incremental refresh)
  @tests/test_asset_index.py (existing unit tests)
  </read_first>

  <action>
  Create `tests/test_asset_index_integration.py` with integration-level tests:

  Test functions:

  ```python
  def test_incremental_refresh_preserves_unrelated_entries():
      """refresh_index_entry only modifies the targeted entry's fields."""
      # Setup: create a test index with 3 entries
      # Call refresh_index_entry for key of entry 1
      # Read back index
      # Assert entries 2 and 3 have identical content (no side effects)
      pass

  def test_incremental_refresh_adds_new_key():
      """refresh_index_entry appends a new key not yet in the index."""
      pass

  def test_incremental_refresh_fallback_on_legacy():
      """refresh_index_entry falls back to build_index when index is legacy format."""
      pass

  def test_ocr_calls_incremental_not_full():
      """Verify that OCR post-processing calls refresh_index_entry, not run_index_refresh."""
      # Structural check: grep ocr.py for the call pattern
      pass

  def test_deep_reading_calls_incremental():
      """Verify that deep_reading.py calls refresh_index_entry."""
      pass

  def test_workspace_paths_in_entry():
      """Each entry in the built index contains all 5 workspace path fields."""
      pass

  def test_path_fields_consistent_after_incremental():
      """After incremental refresh, the targeted entry has consistent path fields."""
      pass
  ```

  Run the tests:
  ```bash
  pytest tests/test_asset_index_integration.py -v --tb=short 2>&1
  ```
  If any test fixture depends on a real vault, use unittest.mock to mock `paperforge_paths` and file I/O.

  **Important:** These are structural/integration tests that verify call sites and data consistency. They should use mocking to avoid depending on real vault fixtures.
  </action>

  <verify>
  <automated>pytest tests/test_asset_index_integration.py -v --tb=short 2>&1</automated>
  </verify>

  <acceptance_criteria>
  - `tests/test_asset_index_integration.py` exists with at least 5 test functions
  - `pytest tests/test_asset_index_integration.py -v --tb=short` exits 0
  - Tests cover: incremental preserves unrelated entries, fallback on legacy, workspace paths, and structural verification of OCR/deep-reading call sites
  </acceptance_criteria>

  <done>Integration tests pass for incremental refresh across all worker operations</done>
</task>

</tasks>

<verification>
1. `grep -n "refresh_index_entry" paperforge/worker/ocr.py` — OCR wired
2. `grep -n "refresh_index_entry" paperforge/worker/deep_reading.py` — deep-reading wired
3. `grep -n "refresh_index_entry" paperforge/worker/repair.py` — repair wired
4. `pytest tests/test_asset_index_integration.py -v --tb=short 2>&1` — integration tests pass
5. `python -c "from paperforge.worker.asset_index import refresh_index_entry; print('OK')"` — module importable
</verification>

<success_criteria>
- OCR post-processing calls refresh_index_entry for each completed key instead of full rebuild
- Deep-reading status sync triggers incremental refresh for each synced paper
- Repair triggers incremental refresh for each repaired paper
- run_index_refresh() has docstring clarifying full-rebuild convention
- Integration tests verify call sites and data consistency
</success_criteria>

<output>
After completion, create `.planning/phases/23-canonical-asset-index-safe-rebuilds/23-03-SUMMARY.md`
</output>