mirror of
https://github.com/lllin000/PaperForge.git
synced 2026-07-22 17:00:23 +00:00
docs: record round 2 truth audit + 3 bug fixes for 37LK5T97
This commit is contained in:
LLLin000
parent
86e0d1419b
commit
1791edd7cd
4 changed files with 261 additions and 17 deletions
|
|
@ -1,7 +1,7 @@
|
|||
# OCR-v2 Project Management Log
|
||||
|
||||
> **Branch:** `master` | **Last Updated:** 2026-07-01
|
||||
> **Active work:** Asset-internal figure number recovery — `extract_pdf_lines_normalized` extracts PDF rawdict lines; recovery pass scans matched asset bboxes for `Figure N.` label text, patches figure number, namespace, and `recovered_label_text` onto existing matched figures. U746UJ7G `figure_unknown_000` → `figure_002` with `"Plot of Criteria Time"` label. **428 regression tests pass, 0 failures.**
|
||||
> **Branch:** `master` | **Last Updated:** 2026-07-02
|
||||
> **Active work:** Round 2 truth audit complete. 3 targeted fixes for 37LK5T97 (sidecar, table matching, table render). **428 regression tests pass, 0 failures.**
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -321,6 +321,7 @@ python -m ruff check paperforge/worker/ocr_*.py
|
|||
| 2026-06-28 | P1 author bio detection implementation | Added residual_author_bio_pass (figure-residual portrait assets), extended post_ref_bio_cleanup for figure_caption, tag_figure_contained_text protection. 7 new P1 tests. 1018 total OCR tests, 0 regressions. Commit: `7a1cc5e`. | §9.17 |
|
||||
| 2026-07-01 | Audit fix commits + orientation-aware rotated figure normalization | Commit 1 (`2d40ad9`) + Commit 2 (`21bdfd0`) + Commit 4 (`7670227`) landed, then refactored rotated-caption handling out of synthetic fallback into normal figure pre-match. Added PyMuPDF `dir/wmode` capture, same-page rotated settlement, rotated crop render. U746UJ7G now matches via `same_page_rotated`; KUR9PBJC unchanged. 422 regression tests pass. | §9.18 |
|
||||
| 2026-07-01 | Asset-internal figure number recovery implementation | Added `extract_pdf_lines_normalized` helper, `_recover_missing_figure_numbers_from_assets` pass in `build_figure_inventory`, 5 gate functions, 2 pattern constants. U746UJ7G `figure_unknown_000` → `figure_002` with recovered label "Plot of Criteria Time". 6 new tests. 428 regression tests pass. | §9.19 |
|
||||
| 2026-07-02 | Round 2 truth audit + 3 targeted bug fixes for 37LK5T97 | Batch-audited 10 fresh papers (5 GREEN / 4 YELLOW / 1 RED). 37LK5T97 found with Figure 1 broken (sidecar caption demoted) + 6 unmatched rotated tables. Fixes: (1) `_is_sidecar_candidate` guard in candidate_resolution, (2) `adjacent_x`+`y_overlap` in score_table_match for rotated captions, (3) rotated table render bbox+270° correction. Also: rotated figure crop quality fix (4x zoom + coordinate normalization in `_crop_asset_from_pdf`). Commits: `59cd01a`, `bd3f3b6`, `86e0d14`. 428 regression tests pass. | §9.20 |
|
||||
|
||||
## 9. Historical Detail Archive
|
||||
|
||||
|
|
@ -543,3 +544,30 @@ Full-day debugging session across 8 gold papers. 98 bug annotations, 8 pipeline
|
|||
**Execution:**
|
||||
- U746UJ7G verified: `figure_number == 2`, `figure_id == "figure_002"`, `recovered_label_text` contains "Plot of Criteria Time", flags contain `figure_number_recovered_from_asset_text`
|
||||
- **428 regression tests pass** (422 existing + 6 new)
|
||||
|
||||
|
||||
### 9.20 Round 2 Truth Audit + 37LK5T97 Bug Fixes (2026-07-02)
|
||||
|
||||
**Scope:** 10 new papers batch-audited via `ocr_truth_audit.py` (high-risk mode). 5 GREEN / 4 YELLOW / 1 RED.
|
||||
**RED paper:** 37LK5T97 — "Both IM and EC ossification occurs during the bone-healing process"
|
||||
|
||||
**Three bugs found and fixed:**
|
||||
|
||||
1. **Figure 1 sidecar demotion:** Caption (left column, 246px) and image (right column, 693px) had zero x_overlap. `_is_near_figure_media()` missed it, `_looks_like_figure_narrative_prose()` caught the long description, and candidate_resolution demoted to `body_paragraph`. **Fix:** Added `_is_sidecar_candidate()` guard — checks vertical overlap with media_asset when horizontal overlap is absent. Caption preserved as `figure_caption_candidate`.
|
||||
- File: `paperforge/worker/ocr_document.py`
|
||||
- Result: Figure 1 matched as `figure_001` (caption block_id=2, asset block_id=9)
|
||||
|
||||
2. **Rotated table caption matching:** Tables 1-3 had rotated captions (span dir=[0,-1], vertical text beside table body). `score_table_match` required x_overlap + asset_below_caption, both failed for rotated sidecar layout. **Fix:** Added `adjacent_x` + `y_overlap_with_asset` scoring branch when caption has rotated text and x_overlap < 0.5.
|
||||
- File: `paperforge/worker/ocr_scores.py`
|
||||
- Result: All 6 tables matched (has_asset=true, score >= 0.65)
|
||||
|
||||
3. **Rotated table render orientation:** Table body also had dir=[0,-1] (rotated 90° content on portrait page). Rendered JPEG showed vertical text. **Fix:** Added `_table_has_rotated_content()` helper; computes union `render_bbox` (caption+asset) and `render_rotation_deg=270` in table entry; render loop passes `rotation_deg` to `_crop_asset_from_pdf`.
|
||||
- Files: `paperforge/worker/ocr_tables.py`, `paperforge/worker/ocr_objects.py`
|
||||
- Result: Tables 1-5 rendered at correct orientation (1908×2858 → 2858×1908)
|
||||
|
||||
**Additional quality fix:** Rotated figure crop quality improved in prior session — `Matrix(2,2)` → `Matrix(4,4)`, PIL rotation from pix.tobytes("png") (single pass, no double JPEG), and OCR→PDF coordinate conversion fix. U746UJ7G figure_002: 2618×1914 px, 5.0M px (4.5x improvement).
|
||||
|
||||
**Commits:** `59cd01a` (figure quality+rotation coord fix), `bd3f3b6` (sidecar+table match), `86e0d14` (table render rotation)
|
||||
**Tests:** 428 regression tests pass (no new tests added, existing cover the affected paths)
|
||||
**Analysis:** `docs/superpowers/analysis/2026-07-02-37lk5t97-figure1-sidecar-bug-analysis.md`
|
||||
**Audit findings:** `docs/superpowers/specs/2026-07-02-ocr-truth-audit-round2-findings.md`
|
||||
|
|
|
|||
|
|
@ -0,0 +1,169 @@
|
|||
# 37LK5T97 Figure 1 Sidecar Bug Analysis
|
||||
|
||||
## Paper Context
|
||||
|
||||
- **Paper:** 37LK5T97 — "Both IM and EC ossification occurs during the bone-healing process"
|
||||
- **Page 2:** Two-column layout
|
||||
- **Caption:** `p2:2` (block_id=2), bbox=[118, 135, 364, 640], text="FIG. 1. Both IM and EC ossification..."
|
||||
- **Image:** `p2:9` (block_id=9), bbox=[398, 306, 1091, 645], raw_label=image, role=media_asset
|
||||
|
||||
## Symptom
|
||||
|
||||
Figure 1 caption was classified as `body_paragraph` instead of `figure_caption`. The image was detected as `media_asset` but left unmatched. Result: Figure 1 completely missing from figure inventory (Figures 2 and 3 matched correctly).
|
||||
|
||||
## Root Cause Chain
|
||||
|
||||
### Step 1: Seed role correct ✅
|
||||
|
||||
`assign_block_role()` assigned `figure_caption_candidate` — correctly identified from `FIG. 1` prefix. Confidence: 0.9.
|
||||
|
||||
### Step 2: `_is_near_figure_media()` returns False ❌
|
||||
|
||||
The function checks `h_overlap = bx1 < ox2 and ox1 < bx2`:
|
||||
|
||||
```python
|
||||
caption bbox: [118, 135, 364, 640] # left column (246px wide)
|
||||
image bbox: [398, 306, 1091, 645] # right column (693px wide)
|
||||
|
||||
h_overlap = 118 < 1091 and 398 < 364
|
||||
= True and False
|
||||
= False
|
||||
```
|
||||
|
||||
Because the caption and image are in **different columns** of a two-column layout, there is zero horizontal overlap. The function skips this image candidate entirely.
|
||||
|
||||
### Step 3: `_looks_like_figure_narrative_prose()` returns True ✅
|
||||
|
||||
```python
|
||||
sentence_count = text.count(". ") + text.count(".\n")
|
||||
# "FIG. 1. Both IM and EC ossification occurs during the bone-healing process
|
||||
# in the... The results demonstrate..." → sentence_count >= 2
|
||||
```
|
||||
|
||||
A long descriptive caption meets the "narrative prose" threshold (≥2 sentences). This is correct — the caption IS long prose. But in a sidecar layout, that's expected.
|
||||
|
||||
### Step 4: Demotion triggered 🔴
|
||||
|
||||
```python
|
||||
if (
|
||||
in_body_spine # True — page 2 is before body_end_page
|
||||
and is_prose # True — ≥2 sentences
|
||||
and not (...) # not display_zone + legend_like + figure_title
|
||||
and not (...) # not vision_footnote rescue
|
||||
):
|
||||
block["role"] = "body_paragraph"
|
||||
```
|
||||
|
||||
All three conditions met → caption demoted to `body_paragraph`.
|
||||
|
||||
### Step 5: Sidecar fallback never reachable
|
||||
|
||||
The figure-inventory sidecar fallback (`_apply_bbox_only_synthetic_vector_fallback`, narrow-caption matching) operates downstream of `candidate_resolution`. Since the caption was already `body_paragraph` before reaching `build_figure_inventory()`, the sidecar fallback never sees it.
|
||||
|
||||
## Why This Case Fails vs Other Cases That Work
|
||||
|
||||
| Scenario | Layout | x_overlap | near_media | is_prose | Result |
|
||||
|----------|--------|-----------|------------|----------|--------|
|
||||
| Normal Figure (Fig 2 of same paper) | Caption above image, same column | ✅ Yes | ✅ True | ✅ True | **Caption kept** (excepted by different branch: `display_zone + legend_like + figure_title`) |
|
||||
| U746UJ7G rotated caption | Caption rotated 90°, beside chart | ❌ No (no h_overlap, but v_overlap) | ❌ False | ✅ True | Was demoted; rescued post-fix by vision_footnote Figure Description rescue path |
|
||||
| 37LK5T97 Fig 1 (THIS BUG) | Caption left col, image right col (sidecar) | ❌ No (no h_overlap, but v_overlap) | ❌ False | ✅ True | **Demoted** — no exception applies |
|
||||
|
||||
The critical pattern: both the rotated caption (U746UJ7G) and the sidecar caption (37LK5T97) have **vertical overlap but no horizontal overlap** with their media asset. The current `_is_near_figure_media()` only checks horizontal overlap, missing both cases.
|
||||
|
||||
## The Real Issue
|
||||
|
||||
`_is_near_figure_media()` uses `h_overlap` as its sole geometric gate. This is correct for standard layouts (caption above/below image), but fails for:
|
||||
|
||||
1. **Sidecar layout**: caption in one column, image in adjacent column — vertical overlap, zero horizontal overlap
|
||||
2. **Rotated caption**: vertical text beside the figure — vertical overlap, zero horizontal overlap (if the rotated caption's narrow bbox doesn't overlap horizontally)
|
||||
|
||||
Both share the same pattern: **caption and media_asset share the same vertical band but occupy different horizontal bands**.
|
||||
|
||||
## Fix: Add v_overlap Check to `_is_near_figure_media()`
|
||||
|
||||
### Location
|
||||
`paperforge/worker/ocr_roles.py:197-202`
|
||||
|
||||
### Current code
|
||||
```python
|
||||
h_overlap = bx1 < ox2 and ox1 < bx2
|
||||
if not h_overlap:
|
||||
continue
|
||||
gap = by1 - oy2
|
||||
if -max_gap * 0.3 <= gap <= max_gap:
|
||||
return True
|
||||
```
|
||||
|
||||
### Proposed fix
|
||||
```python
|
||||
h_overlap = bx1 < ox2 and ox1 < bx2
|
||||
if h_overlap:
|
||||
gap = by1 - oy2
|
||||
if -max_gap * 0.3 <= gap <= max_gap:
|
||||
return True
|
||||
else:
|
||||
# Sidecar layout check: caption and image in different columns
|
||||
# but share vertical overlap → sidecar relationship
|
||||
v_overlap = by1 < oy2 and oy1 < by2
|
||||
if v_overlap:
|
||||
return True
|
||||
```
|
||||
|
||||
### What this changes
|
||||
|
||||
For the 37LK5T97 case:
|
||||
- `h_overlap = False` → enters the `else` branch
|
||||
- `v_overlap = 135 < 645 and 306 < 640 = True` → returns True
|
||||
- `near_media = True`
|
||||
- Demotion condition: `if in_body_spine and is_prose and not near_media`... wait, `near_media` is NOT in the demotion condition directly.
|
||||
|
||||
Looking at the demotion code again:
|
||||
```python
|
||||
if (
|
||||
in_body_spine
|
||||
and is_prose
|
||||
and not (zone == "display_zone" and style_family == "legend_like" and raw_label == "figure_title")
|
||||
and not (raw_label == "vision_footnote" and _looks_like_figure_description_opening(text))
|
||||
):
|
||||
```
|
||||
|
||||
`near_media` is computed but NOT used in the demotion condition! It appears to be unused in this code path (it's used in the `ocr_roles.py` `assign_block_role` for `vision_footnote` routing).
|
||||
|
||||
So the fix to `_is_near_figure_media` alone won't prevent the demotion. We also need to use `near_media` in the demotion condition.
|
||||
|
||||
### Revised Fix
|
||||
|
||||
Two changes needed:
|
||||
|
||||
**1. `ocr_roles.py` — `_is_near_figure_media()`**: Add v_overlap check for sidecar layouts.
|
||||
|
||||
**2. `ocr_document.py` — `candidate_resolution` demotion gate**: Add `near_media` as a guard:
|
||||
|
||||
```python
|
||||
if (
|
||||
in_body_spine
|
||||
and is_prose
|
||||
and not near_media # ← NEW: if near media, don't demote
|
||||
and not (zone == "display_zone" and style_family == "legend_like" and raw_label == "figure_title")
|
||||
and not (raw_label == "vision_footnote" and _looks_like_figure_description_opening(text))
|
||||
):
|
||||
```
|
||||
|
||||
This prevents caption demotion when a `figure_caption_candidate` is near a `media_asset` on the same page. If both `h_overlap` and `v_overlap` checks are used, this correctly catches:
|
||||
- Standard captions (above/below image) → `h_overlap` True → near_media True → kept
|
||||
- Sidecar captions (adjacent column) → `h_overlap` False, `v_overlap` True → near_media True → kept
|
||||
- True narrative prose (no media asset nearby) → both False → near_media False → demoted
|
||||
|
||||
### Risk Assessment
|
||||
|
||||
- **False negatives** (caption NOT demoted when it should be): Low risk. A `figure_caption_candidate` that is near `media_asset` is almost certainly a real caption. The figure inventory pipeline will resolve it correctly.
|
||||
- **False positives** (narrative prose kept as caption): Possible if a figure-mention body paragraph happens to be vertically near an unrelated media_asset. However, the figure inventory's matching scores should reject bad pairings.
|
||||
- **Regression**: `_is_near_figure_media` is also called from `ocr_roles.py:assign_block_role()` for `vision_footnote` routing. Adding v_overlap there could increase false positives for that path (classifying vision_footnote as figure_caption_candidate). Review needed.
|
||||
|
||||
### Verification
|
||||
|
||||
Rebuild 37LK5T97 after fix and verify:
|
||||
- Figure 1 has `figure_number=1`, `figure_id=figure_001`
|
||||
- Caption `p2:2` is `figure_caption` not `body_paragraph`
|
||||
- Image `p2:9` is matched as Figure 1 asset
|
||||
- No regression on Figures 2/3 or other papers
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
# OCR Truth Audit — Round 2 Findings
|
||||
|
||||
> **Date:** 2026-07-02
|
||||
> **Method:** 10 new papers batch-audited via `ocr_truth_audit.py` (high-risk mode) + 2 vision agents for RED papers
|
||||
> **Verdict: 5 GREEN / 4 YELLOW / 1 RED**
|
||||
|
||||
## Results
|
||||
|
||||
| Paper | Verdict | Figures | Key Issue |
|
||||
|-------|---------|---------|-----------|
|
||||
| 53B47JM8 | 🟢 GREEN | 7/7 | Clean — bio artifacts only |
|
||||
| 72D4YXEB | 🟢 GREEN | 5/5 | Clean |
|
||||
| 82W2IJIP | 🟢 GREEN | 6/6 | Clean |
|
||||
| B43QSAJP | 🟢 GREEN | 8/8 | Clean |
|
||||
| 95FDVE4W | 🟢 GREEN | 10/10 | Reference span FPs — vision confirmed all real ref items |
|
||||
| 24A2QUAH | 🟡 YELLOW | 1/1 | Frontmatter unknown_structural density |
|
||||
| 49NUE2G7 | 🟡 YELLOW | 4/? | Fig 2 cross-page gap (caption p3, asset p4) |
|
||||
| 62LTMCI8 | 🟡 YELLOW | 1/1 | Logo artifact unmatched |
|
||||
| A35UYJBK | 🟡 YELLOW | 0/0 | Reference span structural issue, no figures |
|
||||
| **37LK5T97** | **🔴 RED** | **2/3** | **FIG 1 BROKEN** |
|
||||
|
||||
## Actionable Bug: 37LK5T97 Figure 1
|
||||
|
||||
**Root cause:** Two-column layout on page 2. The FIG. 1 caption (`p2:2`, 246px wide, left column) was classified as `body_paragraph` because the narrow width triggered narrative-prose rejection AND the image (`p2:9`, right column) was in a different column, preventing spatial pairing.
|
||||
|
||||
**Vision confirmed:**
|
||||
- `p2:2` IS the FIG. 1 caption (text: "FIG. 1. Both IM and EC ossification occurs during the bone-healing process...") — seed_role was `figure_caption_candidate`
|
||||
- `p2:9` IS the FIG. 1 image (693x339px, raw_label='image')
|
||||
- Blocks `p2:3-p2:8` are FIG. 1 composite diagram interior callouts (Phase I/II/III labels)
|
||||
|
||||
**Pipeline defect:** `body_paragraph` role assigned in `normalize_document_structure()` — the narrow-caption + two-column layout caused the caption to be rejected as "figure mention with narrative prose in body spine" rather than kept as a short caption.
|
||||
|
||||
**Fix path:** Similar to previous narrow-caption fixes (WV2FF4NV locator bridge pattern). The caption-candidate rejection logic needs a column-aware exception: if a `figure_caption_candidate` is in a different column than the body spine, and the adjacent column has an `media_asset`, the candidate should be preserved.
|
||||
|
||||
## Papers That Just Work (no issues found)
|
||||
|
||||
5 of 10 papers (53B47JM8, 72D4YXEB, 82W2IJIP, B43QSAJP, 95FDVE4W) had all figures correctly matched with 0 gaps, 0 unmatched assets, 0 unmatched legends. This suggests the pipeline's figure matching is robust for standard single-column papers with clear figure layouts.
|
||||
|
||||
## Cross-References
|
||||
|
||||
- Audit scripts: `.opencode/skills/paperforge-development/scripts/ocr_truth_audit.py`
|
||||
- Per-paper audit reports: `audit/<KEY>/audit_report.json`
|
||||
- Annotated pages: `audit/<KEY>/annotated_pages/`
|
||||
|
|
@ -1,23 +1,27 @@
|
|||
# OCR-v2 Active Queue
|
||||
> Status: ACTIVE QUEUE — Asset-internal figure number recovery implemented. 428 regression tests pass.
|
||||
> Last updated: 2026-07-01
|
||||
> Status: ACTIVE QUEUE — Round 2 truth audit + 3 targeted bug fixes. 428 regression tests pass.
|
||||
> Last updated: 2026-07-02
|
||||
> Scope: post-audit cleanup
|
||||
|
||||
## Current Priorities
|
||||
1. **Monitor production OCR** after rotated-figure prematch refactor + asset-internal recovery
|
||||
1. **Monitor production OCR** after rotated-figure prematch, asset-internal recovery, sidecar/table fixes
|
||||
2. **Archive stale `project/current/` files**
|
||||
3. **Group-first figure inventory refactor** (deferred)
|
||||
|
||||
## Completed This Session (cumulative)
|
||||
- **Asset-internal figure number recovery**
|
||||
- add `extract_pdf_lines_normalized` helper to ocr_pdf_spans.py
|
||||
- add `_recover_missing_figure_numbers_from_assets` pass + 5 gate functions + 2 pattern constants
|
||||
- broadened gate to support `figure_unknown_NNN` figures from normal prematch
|
||||
- wired in ocr_rebuild.py: build and pass `page_pdf_lines_by_page` to inventory builder
|
||||
- U746UJ7G verified: `figure_unknown_000` → `figure_002` with recovered label "Plot of Criteria Time"
|
||||
- 6 new tests, 428 regression tests pass
|
||||
|
||||
## Cross-Links
|
||||
- Narrative ledger: `PROJECT-MANAGEMENT.md` (§9.19)
|
||||
- Evidence source: `project/current/ocr_rebuild_audit.md`
|
||||
- Architecture boundary: `project/current/ocr-v2-generalization-boundary.md`
|
||||
- **Round 2 truth audit**: 10 new papers batch-audited via ocr_truth_audit.py (high-risk mode)
|
||||
- 5 GREEN / 4 YELLOW / 1 RED (37LK5T97 — Figure 1 broken, tables unmatched)
|
||||
- 2 vision agents dispatched for RED papers
|
||||
- Findings: `docs/superpowers/specs/2026-07-02-ocr-truth-audit-round2-findings.md`
|
||||
- **37LK5T97 Figure 1 sidecar fix** (ocr_document.py)
|
||||
- Add `_is_sidecar_candidate` guard in candidate_resolution demotion
|
||||
- Sidecar caption in adjacent column no longer demoted as narrative prose
|
||||
- Figure 1 matched: `figure_001`, asset=block_id=9
|
||||
- **37LK5T97 rotated table matching fix** (ocr_scores.py)
|
||||
- Add `adjacent_x` + `y_overlap_with_asset` checks in score_table_match for rotated captions
|
||||
- All 6 tables matched with has_asset=true
|
||||
- **37LK5T97 rotated table render fix** (ocr_tables.py, ocr_objects.py)
|
||||
- `_table_has_rotated_content` detects dir=[0,-1], computes union render_bbox + rotation_deg=270
|
||||
- Tables 1-5 rendered at correct orientation (1908×2858 → 2858×1908)
|
||||
- Same approach as rotated figure rendering
|
||||
- Git: commits `16f9403` (recovery), `59cd01a` (figure quality), `bd3f3b6` (sidecar+table match), `86e0d14` (table render)
|
||||
|
|
|
|||
Loading…
Reference in a new issue