---
phase: 02-paddleocr-and-pdf-path-hardening
plan: 02
type: execute
wave: 2
depends_on: ["02-01"]
files_modified:
  - paperforge/ocr_diagnostics.py
  - tests/test_ocr_classify.py
  - pipeline/worker/scripts/literature_pipeline.py
autonomous: true
requirements:
  - OCR-03
  - OCR-04
  - OCR-05

must_haves:
  truths:
    - "OCR failures distinguish blocked (fixable config/path issues) from error (runtime/API issues)"
    - "meta.json contains actionable fix suggestion for every failure"
    - "OCR polling handles schema changes defensively without crashing"
    - "Raw diagnostic snippets are recorded safely in meta.json on schema mismatch"
  artifacts:
    - path: "paperforge/ocr_diagnostics.py"
      provides: "classify_error() function with full exception taxonomy"
      exports: ["classify_error"]
    - path: "tests/test_ocr_classify.py"
      provides: "Unit tests for all exception-to-state mappings"
    - path: "pipeline/worker/scripts/literature_pipeline.py"
      provides: "Defensive exception handling and polling in run_ocr()"
      contains: "classify_error", "meta\['suggestion'\]", "raw_response"
  key_links:
    - from: "pipeline/worker/scripts/literature_pipeline.py"
      to: "paperforge/ocr_diagnostics.py"
      via: "import classify_error"
      pattern: "from paperforge.ocr_diagnostics import classify_error"
    - from: "pipeline/worker/scripts/literature_pipeline.py"
      to: "meta.json"
      via: "write_json with state, error, suggestion, raw_response"
      pattern: "meta\['suggestion'\]"
---

<objective>
Implement OCR failure classification taxonomy and defensive polling in the OCR worker.

Purpose: Replace the single generic "PaddleOCR request failed" error with specific, actionable failure states (blocked vs error) and fix suggestions. Make polling resilient to API schema changes.
Output: classify_error() function, defensive run_ocr() exception handling, polling safety, comprehensive tests.
</objective>

<execution_context>
@C:/Users/Lin/.opencode/get-shit-done/workflows/execute-plan.md
@C:/Users/Lin/.opencode/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/STATE.md
@.planning/ROADMAP.md
@.planning/REQUIREMENTS.md
@.planning/phases/02-paddleocr-and-pdf-path-hardening/02-CONTEXT.md

<interfaces>
From pipeline/worker/scripts/literature_pipeline.py (run_ocr current exception handling):
```python
except requests.RequestException as e:
    meta['ocr_status'] = 'error'
    meta['error'] = f'PaddleOCR request failed: {e}'
    queue_row['queue_status'] = 'error'
    write_json(paths['ocr'] / key / 'meta.json', meta)
    changed += 1
    continue
```

From pipeline/worker/scripts/literature_pipeline.py (current polling code):
```python
response = requests.get(f"{job_url}/{meta['ocr_job_id']}", headers={'Authorization': f'bearer {token}'}, timeout=60)
response.raise_for_status()
payload = response.json()['data']
state = payload['state']
```

Failure taxonomy from CONTEXT.md D-03:
| State | Meaning | User Action |
|-------|---------|-------------|
| `blocked` | Fixable issue (config/path/token) | Fix issue, then re-run `paperforge ocr` |
| `error` | Runtime failure (API error, timeout, schema mismatch) | Check `meta.json` error field for fix suggestion |
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Implement classify_error() with full exception taxonomy</name>
  <files>paperforge/ocr_diagnostics.py, tests/test_ocr_classify.py</files>
  <read_first>
    - pipeline/worker/scripts/literature_pipeline.py lines 2404-2413 (current exception handling)
    - pipeline/worker/scripts/literature_pipeline.py lines 2342-2376 (current polling code)
    - .planning/phases/02-paddleocr-and-pdf-path-hardening/02-CONTEXT.md section 3.3 (failure taxonomy decisions)
  </read_first>
  <behavior>
    - classify_error(ConnectionError, None) → ('blocked', 'Check PADDLEOCR_JOB_URL in .env and re-run `paperforge ocr`')
    - classify_error(TimeoutError, None) → ('error', 'OCR service timed out. Retry with `paperforge ocr` or check network.')
    - classify_error(HTTPError with 401 response, response_401) → ('blocked', 'PaddleOCR API key invalid or missing. Set PADDLEOCR_API_TOKEN and re-run `paperforge ocr`')
    - classify_error(HTTPError with 404 response, response_404) → ('error', 'OCR job not found. Re-run `paperforge ocr` to resubmit.')
    - classify_error(HTTPError with 503 response, response_503) → ('error', 'OCR provider error. Retry later with `paperforge ocr`.')
    - classify_error(JSONDecodeError, None) → ('error', 'PaddleOCR API response format changed. Check `meta.json` raw response and update client.')
    - classify_error(KeyError('data'), None) → ('error', 'PaddleOCR API response missing expected fields. Provider may have updated schema.')
    - classify_error(FileNotFoundError, None) → ('blocked', 'PDF file not found. Check Zotero attachment and re-run `paperforge ocr`.')
    - classify_error(ValueError('generic'), None) → ('error', 'Unexpected error: generic. Retry with `paperforge ocr` or run `paperforge ocr doctor`.')
  </behavior>
  <action>
    Create or extend `paperforge/ocr_diagnostics.py` with `classify_error(exception, response)` function.

    Signature: `def classify_error(exception: Exception, response: requests.Response | None) -> tuple[str, str]:`

    Mapping rules (per D-03):
    - `requests.exceptions.ConnectionError` → ('blocked', 'Check PADDLEOCR_JOB_URL in .env and re-run `paperforge ocr`')
    - `requests.exceptions.Timeout` or `requests.exceptions.ReadTimeout` → ('error', 'OCR service timed out. Retry with `paperforge ocr` or check network.')
    - `requests.exceptions.HTTPError`:
      - If response.status_code == 401 → ('blocked', 'PaddleOCR API key invalid or missing. Set PADDLEOCR_API_TOKEN and re-run `paperforge ocr`')
      - If response.status_code == 404 → ('error', 'OCR job not found. Re-run `paperforge ocr` to resubmit.')
      - If response.status_code >= 500 → ('error', 'OCR provider error. Retry later with `paperforge ocr`.')
      - Otherwise → ('error', f'OCR HTTP error {response.status_code}. Retry with `paperforge ocr` or run `paperforge ocr doctor`.')
    - `json.JSONDecodeError` → ('error', 'PaddleOCR API response format changed. Check `meta.json` raw response and update client.')
    - `KeyError` → ('error', 'PaddleOCR API response missing expected fields. Provider may have updated schema.')
    - `FileNotFoundError` → ('blocked', 'PDF file not found. Check Zotero attachment and re-run `paperforge ocr`.')
    - Any other Exception → ('error', f'Unexpected error: {exception}. Retry with `paperforge ocr` or run `paperforge ocr doctor`.')

    Create `tests/test_ocr_classify.py` with tests for each mapping. Use `unittest.mock.MagicMock` to create mock Response objects with `.status_code` for HTTPError tests. Use `requests.HTTPError` with `response=mock_response`.

    Do NOT create the ocr_doctor function here (that is Plan 02-03). Only classify_error.
  </action>
  <verify>
    <automated>pytest tests/test_ocr_classify.py -v</automated>
  </verify>
  <done>classify_error() exists with full taxonomy, all 9 exception types map to correct (state, suggestion) pairs, tests pass</done>
</task>

<task type="auto" tdd="true">
  <name>Task 2: Wire classification and defensive polling into run_ocr()</name>
  <files>pipeline/worker/scripts/literature_pipeline.py</files>
  <read_first>
    - pipeline/worker/scripts/literature_pipeline.py lines 2294-2429 (full run_ocr function)
    - paperforge/ocr_diagnostics.py (classify_error function from Task 1)
  </read_first>
  <behavior>
    - On API submission exception: meta gets classified state, error string, and suggestion
    - On polling JSON parse failure: meta gets 'error', suggestion, and truncated raw_response
    - queue_status always matches meta['ocr_status']
  </behavior>
  <action>
    Modify `pipeline/worker/scripts/literature_pipeline.py` in two locations within `run_ocr()`:

    **Change A — Exception handling in submission loop (around lines 2404-2413):**
    Replace:
    ```python
    except requests.RequestException as e:
        meta['ocr_status'] = 'error'
        meta['error'] = f'PaddleOCR request failed: {e}'
        queue_row['queue_status'] = 'error'
    ```
    With:
    ```python
    except Exception as e:
        from paperforge.ocr_diagnostics import classify_error
        state, suggestion = classify_error(e, getattr(e, 'response', None))
        meta['ocr_status'] = state
        meta['error'] = str(e)
        meta['suggestion'] = suggestion
        queue_row['queue_status'] = state
    ```
    Keep the `write_json(...)` and `changed += 1` and `continue` lines after this block unchanged.

    **Change B — Defensive polling in status check loop (around lines 2342-2376):**
    Replace:
    ```python
    response = requests.get(f"{job_url}/{meta['ocr_job_id']}", headers={'Authorization': f'bearer {token}'}, timeout=60)
    response.raise_for_status()
    payload = response.json()['data']
    state = payload['state']
    ```
    With:
    ```python
    response = requests.get(f"{job_url}/{meta['ocr_job_id']}", headers={'Authorization': f'bearer {token}'}, timeout=60)
    response.raise_for_status()
    try:
        payload = response.json()['data']
        state = payload['state']
    except (json.JSONDecodeError, KeyError) as e:
        from paperforge.ocr_diagnostics import classify_error
        state, suggestion = classify_error(e, None)
        meta['ocr_status'] = state
        meta['error'] = f'API schema mismatch during polling: {e}'
        meta['suggestion'] = suggestion
        meta['raw_response'] = response.text[:1000]  # truncate for safety per OCR-05
        queue_row['queue_status'] = state
        write_json(paths['ocr'] / key / 'meta.json', meta)
        changed += 1
        active_submitted = max(0, active_submitted - 1)
        continue
    ```
    Ensure `import json` is already present at the top of the file (it is, used elsewhere).

    Do NOT change the existing `if not queue_row.get('has_pdf'):` block — that was replaced by preflight in Plan 02-01.
  </action>
  <verify>
    <automated>
      grep -n "classify_error" pipeline/worker/scripts/literature_pipeline.py
      grep -n "meta\['suggestion'\]" pipeline/worker/scripts/literature_pipeline.py
      grep -n "raw_response" pipeline/worker/scripts/literature_pipeline.py
    </automated>
  </verify>
  <done>run_ocr() uses classify_error for both submission exceptions and polling schema failures; meta.json gets state, error, suggestion, and raw_response on schema mismatch</done>
</task>

</tasks>

<verification>
- [ ] `pytest tests/test_ocr_classify.py -v` passes
- [ ] `grep -n "classify_error" pipeline/worker/scripts/literature_pipeline.py` returns 2 matches (submission + polling)
- [ ] `grep -n "meta\['suggestion'\]" pipeline/worker/scripts/literature_pipeline.py` returns 2 matches
- [ ] `grep -n "raw_response" pipeline/worker/scripts/literature_pipeline.py` returns 1 match
- [ ] All requirement IDs OCR-03, OCR-04, OCR-05 are addressed
</verification>

<success_criteria>
- classify_error() maps all 9 exception types to correct (state, suggestion) per D-03 taxonomy
- run_ocr() submission exceptions produce classified states instead of generic 'error'
- run_ocr() polling is defensive against JSONDecodeError and KeyError, recording raw_response truncated to 1000 chars
- meta.json contains 'suggestion' field on every failure path
</success_criteria>

<output>
After completion, create `.planning/phases/02-paddleocr-and-pdf-path-hardening/02-02-SUMMARY.md`
</output>
