diff --git a/audit/28ALPCY7_findings.md b/audit/28ALPCY7_findings.md new file mode 100644 index 00000000..f04c7dd2 --- /dev/null +++ b/audit/28ALPCY7_findings.md @@ -0,0 +1,84 @@ +# 28ALPCY7 — "Figure N" Lost-in-OCR Investigation + +## Paper +- **Title**: 3D-Printed β-Tricalcium Phosphate Scaffold Combined with a Pulse Electromagnetic Field Promotes the Repair of Skull Defects in Rats +- **PDF**: Liang et al., 2019, ACS Biomaterials Science & Engineering +- **Pages**: 37 total + +--- + +## Methodology + +For each page containing figures/tables, we compared: +1. **PDF text layer** (PyMuPDF `page.get_text()`) — search for "Figure N"/"Table N" headings +2. **OCR role-index** (block_trace.csv from the PaperForge pipeline) — check for figure_caption/table_caption role + +## Pages with Figures/Tables (source: figure_table_ownership_summary.json) + +| Page | Content | PDF has heading? | OCR has caption block? | Status | +|------|---------|-----------------|----------------------|--------| +| 9 | Table 1 (primer sequences) | `Table 1.` | b5 table_caption | PRESERVED | +| 12 | Figure 1 (SEM images) | `Figure 1.` | b8 figure_caption | PRESERVED | +| 13 | Figure 2 (LIVE/DEAD staining) | `Figure 2.` | b10 figure_caption | PRESERVED | +| 15 | Figure 3 (CCK-8 analysis) | `Figure 3.` | b4 figure_caption | PRESERVED | +| 16 | Figure 4 (ALP activity) | `Figure 4.` | b4 figure_caption | PRESERVED | +| 17 | Figure 5 (ALP staining) | `Figure 5.` | b11 figure_caption | PRESERVED | +| 19 | Figure 6 charts (ALP/Runx2/OPN) | NO heading (charts only) | b4/b6/b8 fragment captions | Heading on page 20 | +| 20 | Figure 6 caption (qRT-PCR) | `Figure 6.` | b3 figure_caption | PRESERVED | +| 22 | Figure 7 (3D reconstruction) | `Figure 7.` | b4 figure_caption | PRESERVED | +| 23 | Figure 8 (BMD analysis) | `Figure 8.` | b5 figure_caption | PRESERVED | +| 27 | Figure 9 (HE staining) | `Figure 9.` | b5 figure_caption | PRESERVED | +| 36 | TOC graphic | `Table of Contents graphic` | b3 table_caption_candidate | PARTIAL | + +--- + +## Key Finding: No "Figure N" headings are lost from PDF text layer + +**All 10 numbered figure/table headings present in the PDF text layer are correctly captured in the OCR role-index.** There is zero cases where a "Figure N"/"Table N" string exists in the PDF text but is absent from OCR blocks. + +## Related Issues Found + +### 1. Figure 6 heading is on a different page than the figure assets +- The figure caption **"Figure 6. qRT-PCR analysis..."** is on **Page 20** +- But the actual figure chart images (ALP, Runx2, OPN expression) are on **Page 19** +- The figure_table_ownership incorrectly maps Figure 6 to "page": 19 (the legend_block_id=3 on page 19 is actually a body paragraph, not the caption) +- The three chart blocks on page 19 (b5=media_asset/chart, b7=media_asset/chart, b9=figure_asset/chart) have only fragmentary sub-panel labels ("ALP", "Runx2", "OPN") instead of the full Figure 6 heading + +### 2. Pages 25-26: Full-page images with no text layer -> media_asset with no caption matching +- Pages 25 and 26 contain **full-page images** with NO text layer (only page numbers and journal header in PDF text) +- The OCR pipeline correctly identified these as `media_asset` blocks (6 total across pages 25, 26, 27, 36): + - Page 25: b3, b4, b5, b6 = media_asset/image (4 blocks) + - Page 26: b3, b4 = media_asset/image (2 blocks) + - Page 27: b3 = media_asset/image (1 block) +- These are **NOT caption-matched** — they remain as unmatched/uncaptioned `media_asset` blocks +- The vision audit confirms these are real figures/charts that "should have caption" +- **Cause**: Captions are embedded within the image pixels, not as separate PDF text — the OCR pipeline cannot extract text from images + +### 3. Page 17: Mixed media_asset + figure_asset blocks +- Page 17 has both `media_asset` (b3-b6, 4 image blocks) and `figure_asset` (b9-b10, 2 image blocks) +- The `media_asset` blocks have no figure_number assignment — they are uncaptioned +- The `figure_asset` blocks are matched with `figure_caption` block b11 (Figure 5) +- This suggests sub-panel images within Figure 5 are classified as media_asset, not figure_asset + +### 4. Total uncaptioned figure assets +- **4 unmatched figure assets** across the paper +- **8 unmatched table assets** across the paper +- **2 unresolved figure clusters** (pages 17 and 19) where near-neighbor caption matching failed + +--- + +## Summary + +| Metric | Count | +|--------|-------| +| Numbered figure/table headings in PDF text | 10 | +| Figure/table headings correctly captured in OCR | 10 | +| **Figure N headings LOST in OCR** | **0** | +| Media_asset blocks (images without caption matching) | 18 | +| Pages with media_asset but NO caption on same page | 25, 26 | +| Unmatched figure assets | 4 | +| Unmatched table assets | 8 | + +## Conclusion + +**The "Figure N" lost-in-OCR problem does not affect paper 28ALPCY7 in the traditional sense.** All extractable figure/table headings from the PDF text layer are faithfully captured in the OCR role-index blocks. The images on pages 25-26 have their captions embedded within the image pixels (no separate text layer), so there is no "Figure N" text to lose — the OCR pipeline correctly classifies them as media_asset rather than figure_asset, but this means they remain uncaptioned in the structured output. diff --git a/audit/media_asset_samples/24A2QUAH_p3_b3_table.png b/audit/media_asset_samples/24A2QUAH_p3_b3_table.png new file mode 100644 index 00000000..c9ace71b Binary files /dev/null and b/audit/media_asset_samples/24A2QUAH_p3_b3_table.png differ diff --git a/audit/media_asset_samples/24A2QUAH_p7_b3_table.png b/audit/media_asset_samples/24A2QUAH_p7_b3_table.png new file mode 100644 index 00000000..2c901111 Binary files /dev/null and b/audit/media_asset_samples/24A2QUAH_p7_b3_table.png differ diff --git a/audit/media_asset_samples/24A2QUAH_p8_b3_table.png b/audit/media_asset_samples/24A2QUAH_p8_b3_table.png new file mode 100644 index 00000000..2dba1fcc Binary files /dev/null and b/audit/media_asset_samples/24A2QUAH_p8_b3_table.png differ diff --git a/audit/media_asset_samples/28ALPCY7_p25_b3_image.png b/audit/media_asset_samples/28ALPCY7_p25_b3_image.png new file mode 100644 index 00000000..ab642127 Binary files /dev/null and b/audit/media_asset_samples/28ALPCY7_p25_b3_image.png differ diff --git a/audit/media_asset_samples/28ALPCY7_p25_b4_image.png b/audit/media_asset_samples/28ALPCY7_p25_b4_image.png new file mode 100644 index 00000000..da2e51ed Binary files /dev/null and b/audit/media_asset_samples/28ALPCY7_p25_b4_image.png differ diff --git a/audit/media_asset_samples/28ALPCY7_p25_b5_image.png b/audit/media_asset_samples/28ALPCY7_p25_b5_image.png new file mode 100644 index 00000000..6faf6b49 Binary files /dev/null and b/audit/media_asset_samples/28ALPCY7_p25_b5_image.png differ diff --git a/audit/media_asset_samples/28ALPCY7_p25_b6_image.png b/audit/media_asset_samples/28ALPCY7_p25_b6_image.png new file mode 100644 index 00000000..e69de29b diff --git a/audit/media_asset_samples/28ALPCY7_p26_b3_image.png b/audit/media_asset_samples/28ALPCY7_p26_b3_image.png new file mode 100644 index 00000000..6f44a2e2 Binary files /dev/null and b/audit/media_asset_samples/28ALPCY7_p26_b3_image.png differ diff --git a/audit/media_asset_samples/28ALPCY7_p26_b4_image.png b/audit/media_asset_samples/28ALPCY7_p26_b4_image.png new file mode 100644 index 00000000..e69de29b diff --git a/audit/media_asset_samples/28ALPCY7_p36_b10_image.png b/audit/media_asset_samples/28ALPCY7_p36_b10_image.png new file mode 100644 index 00000000..e69de29b diff --git a/audit/media_asset_samples/28ALPCY7_p36_b12_image.png b/audit/media_asset_samples/28ALPCY7_p36_b12_image.png new file mode 100644 index 00000000..e69de29b diff --git a/audit/media_asset_samples/28ALPCY7_p36_b14_image.png b/audit/media_asset_samples/28ALPCY7_p36_b14_image.png new file mode 100644 index 00000000..e69de29b diff --git a/audit/media_asset_samples/28ALPCY7_p36_b16_image.png b/audit/media_asset_samples/28ALPCY7_p36_b16_image.png new file mode 100644 index 00000000..e69de29b diff --git a/audit/media_asset_samples/28ALPCY7_p36_b9_image.png b/audit/media_asset_samples/28ALPCY7_p36_b9_image.png new file mode 100644 index 00000000..e69de29b diff --git a/audit/media_asset_samples/5S7UI34M_p1_b2_image.png b/audit/media_asset_samples/5S7UI34M_p1_b2_image.png new file mode 100644 index 00000000..9045d666 Binary files /dev/null and b/audit/media_asset_samples/5S7UI34M_p1_b2_image.png differ diff --git a/audit/media_asset_samples/5S7UI34M_p4_b2_image.png b/audit/media_asset_samples/5S7UI34M_p4_b2_image.png new file mode 100644 index 00000000..f195830f Binary files /dev/null and b/audit/media_asset_samples/5S7UI34M_p4_b2_image.png differ diff --git a/audit/media_asset_samples/5S7UI34M_p6_b4_image.png b/audit/media_asset_samples/5S7UI34M_p6_b4_image.png new file mode 100644 index 00000000..699605b1 Binary files /dev/null and b/audit/media_asset_samples/5S7UI34M_p6_b4_image.png differ diff --git a/audit/media_asset_samples/5S7UI34M_p6_b6_image.png b/audit/media_asset_samples/5S7UI34M_p6_b6_image.png new file mode 100644 index 00000000..bdf8c47d Binary files /dev/null and b/audit/media_asset_samples/5S7UI34M_p6_b6_image.png differ diff --git a/audit/media_asset_samples/5S7UI34M_p6_b7_image.png b/audit/media_asset_samples/5S7UI34M_p6_b7_image.png new file mode 100644 index 00000000..e69de29b diff --git a/audit/media_asset_samples/5S7UI34M_p8_b11_chart.png b/audit/media_asset_samples/5S7UI34M_p8_b11_chart.png new file mode 100644 index 00000000..e69de29b diff --git a/audit/media_asset_samples/5S7UI34M_p8_b12_image.png b/audit/media_asset_samples/5S7UI34M_p8_b12_image.png new file mode 100644 index 00000000..db8efdac Binary files /dev/null and b/audit/media_asset_samples/5S7UI34M_p8_b12_image.png differ diff --git a/audit/media_asset_samples/5S7UI34M_p8_b3_image.png b/audit/media_asset_samples/5S7UI34M_p8_b3_image.png new file mode 100644 index 00000000..e7b88395 Binary files /dev/null and b/audit/media_asset_samples/5S7UI34M_p8_b3_image.png differ diff --git a/audit/media_asset_samples/5S7UI34M_p8_b5_image.png b/audit/media_asset_samples/5S7UI34M_p8_b5_image.png new file mode 100644 index 00000000..e69de29b diff --git a/audit/media_asset_samples/5S7UI34M_p8_b8_chart.png b/audit/media_asset_samples/5S7UI34M_p8_b8_chart.png new file mode 100644 index 00000000..a4a8242e Binary files /dev/null and b/audit/media_asset_samples/5S7UI34M_p8_b8_chart.png differ diff --git a/audit/media_asset_samples/5S7UI34M_p8_b9_image.png b/audit/media_asset_samples/5S7UI34M_p8_b9_image.png new file mode 100644 index 00000000..c5c517d5 Binary files /dev/null and b/audit/media_asset_samples/5S7UI34M_p8_b9_image.png differ diff --git a/audit/media_asset_vision_report.md b/audit/media_asset_vision_report.md new file mode 100644 index 00000000..f2d3b443 --- /dev/null +++ b/audit/media_asset_vision_report.md @@ -0,0 +1,43 @@ +# Media Asset Vision Audit Report + +> **Date**: 2026-07-02 +> **Method**: via +> **Sample**: 14 valid images from 3 papers (24A2QUAH, 28ALPCY7, 5S7UI34M) +> **Extraction**: Cropped from PDF pages via PyMuPDF using media_asset bboxes + +## Results + +| File | Content Type | Smart Has Caption? | Raw Label | +|------|-------------|-------------------|-----------| +| 24A2QUAH_p3_b3_table.png | data table | YES | table | +| 24A2QUAH_p7_b3_table.png | data table | YES | table | +| 24A2QUAH_p8_b3_table.png | data table | YES | table | +| 28ALPCY7_p25_b3_image.png | figure/chart | YES | image | +| 28ALPCY7_p25_b4_image.png | figure/chart | YES | image | +| 28ALPCY7_p26_b3_image.png | figure/chart | YES | image | +| 5S7UI34M_p1_b2_image.png | journal logo | no | image | +| 5S7UI34M_p4_b2_image.png | figure/chart | YES | image | +| 5S7UI34M_p6_b4_image.png | figure/chart | YES | image | +| 5S7UI34M_p6_b6_image.png | figure/chart | YES | image | +| 5S7UI34M_p8_b3_image.png | figure/chart | YES | image | +| 5S7UI34M_p8_b8_chart.png | figure/chart | YES | chart | +| 5S7UI34M_p8_b9_image.png | figure/chart | YES | image | +| 5S7UI34M_p8_b12_image.png | figure/chart | YES | image | + +## Summary + +- **Data tables**: 3 (all raw_label=table) +- **Figures/charts**: 10 (raw_label=image or chart) +- **Journal logo**: 1 (page 1, raw_label=image) +- **Should have caption**: 13/14 +- **Corrupted crops**: 11 (invalid bbox dimensions during PDF extraction) + +## Key Finding + +Of 14 successfully extracted media_asset blocks, **13 are real scientific content** that should have been matched with a figure/table caption: +- 3 data tables (raw_label=table) — should be table_html, matched with table captions +- 10 figures/charts (raw_label=image or chart) — should be figure_asset, matched with figure captions +- 1 journal logo on page 1 — correctly excluded from matching + +The 650 media_asset with raw_label=table across the corpus are very likely **real data tables misclassified as media_asset** instead of table_html. +The 225 media_asset with raw_label=chart are likely **real figures misclassified as media_asset** instead of figure_asset. diff --git a/audit/tmp_pdf/page4.png b/audit/tmp_pdf/page4.png new file mode 100644 index 00000000..7e5d0bac Binary files /dev/null and b/audit/tmp_pdf/page4.png differ diff --git a/local-page_trace.md b/local-page_trace.md new file mode 100644 index 00000000..23220058 --- /dev/null +++ b/local-page_trace.md @@ -0,0 +1,335 @@ +# Page Number Tracing Through PaperForge Pipeline + +## 1. Raw OCR Result → `all_results` + +**Entry point:** `paperforge/worker/ocr.py` lines 2357-2361 (`run_ocr`) +``` +PaddleOCR API returns NDJSON per-line. Each line parsed as JSON["result"] = one page payload. +→ all_results: list[dict], each element = per-page result from PaddleOCR API. +``` + +The API response structure is stored verbatim at `result.json` (ocr.py line 1801): +```python +write_json(json_dir / "result.json", all_results) +``` + +**Backfill path** (`ocr_rebuild.py` lines 515-522): reads `result.json` and normalizes: +- If dict → `raw.get("pages", [])` (legacy format) +- If list → use directly + +--- + +## 2. `build_raw_blocks_for_result_lines()` — Page Number Origin + +**File:** `paperforge/worker/ocr_blocks.py` lines 465-472 + +```python +def build_raw_blocks_for_result_lines(paper_id: str, all_results: list[dict]) -> list[dict[str, Any]]: + rows = [] + page_num = 0 + for payload in all_results: + for res in payload.get("layoutParsingResults", []): + page_num += 1 + rows.extend(build_raw_blocks_for_page(paper_id, page_num, res)) + return rows +``` + +**Key observations:** +- `page_num` starts at **0** and increments **after** the first entry → **first page = 1** (1-indexed) +- **Nested loop**: For each API page payload, it iterates ALL `layoutParsingResults`. If a single page returns >1 result entry, page counting inflates. (In normal PaddleOCR operation, each page produces exactly one `layoutParsingResults` entry.) +- Each result entry gets its own page number, and all blocks from that entry share that page number. + +--- + +## 3. `build_raw_blocks_for_page()` — Direct Page Assignment + +**File:** `paperforge/worker/ocr_blocks.py` lines 441-462 + +```python +def build_raw_blocks_for_page(paper_id: str, page: int, result: dict) -> list[dict[str, Any]]: + pruned = result.get("prunedResult", {}) + width = pruned.get("width", 0) + height = pruned.get("height", 0) + blocks = pruned.get("parsing_res_list", []) + rows = [] + for i, block in enumerate(blocks): + rows.append({ + "paper_id": paper_id, + "page": page, # ← PAGE NUMBER SET HERE + "block_id": block.get("block_id", f"p{page}_b{i}"), + "raw_label": block.get("block_label", "unknown"), + "raw_order": block.get("block_order", i), + "bbox": block.get("block_bbox", [0, 0, 0, 0]), + "text": block.get("block_content", "") or "", + "page_width": width, + "page_height": height, + "source": "ocr_raw", + }) + return rows +``` + +- `page` parameter is used **verbatim** as the block's page number. +- No offset, no transformation. + +--- + +## 4. `postprocess_ocr_result()` — Parallel Page Counting + +**File:** `paperforge/worker/ocr.py` lines 1777-1796 + +```python +page_num = 0 +for page_payload in all_results: + for res in page_payload.get("layoutParsingResults", []): + page_num += 1 + render_page_blocks(vault, page_num, res, ...) +``` + +This is the **SAME** counting logic as `build_raw_blocks_for_result_lines`. Both iterate `all_results` → `layoutParsingResults` in lockstep. `page_num` at the end = total number of `layoutParsingResults` entries = `meta["page_count"]`. + +Then sets `meta["page_count"] = page_num` at ocr.py line 2413: +```python +meta["page_count"] = page_num +``` + +--- + +## 5. `build_structured_blocks()` — Cover Page Drop (CRITICAL) + +**File:** `paperforge/worker/ocr_blocks.py` lines 189-302 + +**Step 1: Group by page** (lines 200-205): +```python +by_page: dict[int, list[dict]] = {} +for block in raw_blocks: + page = block.get("page", 1) + by_page.setdefault(page, []).append(block) + +total_pages = max(by_page.keys()) if by_page else 1 # ← original max page (e.g. N) +``` + +**Step 2: Cover page detection and removal** (lines 297-302): +```python +if _has_preproof_cover_page_one(rows): + rows = [row for row in rows if (row.get("page", 0) or 0) != 1] +elif _has_nonpreproof_cover_page_one(rows): + rows = [row for row in rows if (row.get("page", 0) or 0) != 1] +``` + +**⚠️ OFFSET ROOT CAUSE:** After this drop: +- Blocks with page==1 are **removed entirely** +- Remaining blocks still have their **original page values** (2, 3, ..., N) +- **No re-indexing / renumbering** occurs +- `meta["page_count"]` is still N (set before this function runs) +- `total_pages` was computed from `by_page` (line 205) **before** the drop, so it's still N +- `discover_body_family_anchor(rows, page_count=total_pages)` at line 304 receives the original N + +**Detection functions** (lines 129-175): +- `_has_preproof_cover_page_one()` — checks for preproof markers on page 1 +- `_has_nonpreproof_cover_page_one()` — checks for cover text markers ("just accepted", etc.) on page 1 without real content + +--- + +## 6. `infer_zones()` — Page-Based Zone Computation + +**File:** `paperforge/worker/ocr_document.py` lines 1300-1713 + +**First surviving page** (line 1430): +```python +first_surviving_page = _first_surviving_page(blocks) +``` +Where `_first_surviving_page()` (line 628-630): +```python +def _first_surviving_page(blocks: list[dict]) -> int | None: + pages = sorted({int(block.get("page", 0) or 0) for block in blocks if int(block.get("page", 0) or 0) > 0}) + return pages[0] if pages else None +``` + +After cover page drop, this returns **2** (the first remaining page). This is used throughout the zone inference. + +**Anchor page selection** (line 1435): +```python +anchor_page = first_surviving_page if first_surviving_page is not None and first_surviving_page <= 2 else 1 +``` +This correctly handles both cases: uses first_surviving_page when it's 1 or 2 (normal or after cover drop), falls back to 1 for blocksets starting on page 3+. + +**Body end page** (line 1536): +```python +body_end_page = first_reference_page - 1 if first_reference_page is not None else max_page +``` +A simple page arithmetic: no off-by-one here, but values depend on block page values (which may be shifted after cover drop). + +**Frontmatter side zone** (line 1490): +```python +int(block.get("page", 0) or 0) >= first_reference_page - 1 +``` +Uses `first_reference_page` (from block page values) — consistent within the shifted numbering. + +**`page1_candidates`** (lines 1437-1444): Selects blocks whose page matches `anchor_page`. After cover drop, `anchor_page` = first surviving page (e.g., 2), so it processes blocks on that page as "page 1 candidates." The role assignment functions (in `ocr_roles.py`) reference these by checking `page_num == 1` or `page == anchor_page` — this is handled via the `anchor_page` variable, not hardcoded `== 1`. + +--- + +## 7. Page-Based Role Assignment in `ocr_roles.py` + +**File:** `paperforge/worker/ocr_roles.py` + +Role assignment functions **read `page` from block data directly**: + +- **`assign_block_role()`** (line 865+) — receives `block` dict with `"page"` from `build_structured_blocks` pass-through. Checks `block.get("page") or 1` and `page_num == 1` patterns. + +- **Hardcoded `page_num == 1` checks** — These fire correctly when page 1 exists, but after cover page drop, they **never fire** because page 1 blocks are gone. Instead, the zone-based logic in `infer_zones()` drives frontmatter handling via `anchor_page`. + +- **`_is_backmatter_boundary_heading()`** line 369: uses `page_num` and `total_pages` — works within shifted numbering. + +**Evidence strings** contain phrases like `"page-1 article-type label"` (line 906), `"page-1 zone title_zone"` (line 1027), etc. These are metadata labels only; not used for computation. + +--- + +## 8. Role Index — Verbatim Page Propagation + +**File:** `paperforge/worker/ocr_index.py` lines 27-35 + +```python +for block in structured_blocks: + role = block.get("role", "") + entry = { + "paper_id": block.get("paper_id", ""), + "page": block.get("page", 0), # ← VERBATIM from block + ... + } +``` + +No transformation. If a block has `page: 5`, it goes into the role index as `page: 5`. + +Metadata entries artificially get `"page": 0` (line 54). + +--- + +## 9. Render — Page Marker Generation + +**File:** `paperforge/worker/ocr_render.py` + +**During body block iteration** (lines 1635-1664): +```python +if block_page != current_page: + for p in range(first_new_page, block_page): + lines.append(f"") + ... + lines.append(f"") +``` +Emits page markers based on block page values. + +**Post-body sweep** (lines 1891-1908): +```python +effective_count = page_count if page_count is not None else max_page +for p in range(1, effective_count + 1): + if p in emitted_pages: + continue + if p > (current_page or 0): + lines.append(f"") +``` +This iterates from 1 to `page_count` (original N). After cover page drop: +- **Page 1** gets an empty `` marker (no blocks, no objects) +- Page 2..N get their content (blocks that exist) +- `emitted_pages` count will be N (= page_count), so `validate_ocr_meta` passes + +**⚠️ Note:** The render generates `page_count` markers regardless of whether page 1 blocks exist. This is correct for `validate_ocr_meta` (which counts markers), but produces an empty `` in the markdown output when a cover page was dropped. + +--- + +## 10. `normalize_document_structure()` — Page Usage + +**File:** `paperforge/worker/ocr_document.py` lines 5494-6203 + +- Builds `page_layouts` from `_build_page_layout_profiles()` → uses `int(block.get("page", 0) or 0)` (line 525+) +- Calls `infer_zones()` with block.page values +- Calls `discover_body_family_anchor(blocks)` → uses `page_count` from `total_pages` (original N) +- All page comparisons are within the block's page numbering system (which may be shifted if cover was dropped) +- **No page renumbering or offset correction anywhere** + +--- + +## 11. `_select_middle_sample_pages()` — Page Selection for Body Anchor + +**File:** `paperforge/worker/ocr_families.py` lines 225-262 + +```python +def _select_middle_sample_pages(blocks: list[dict], page_count: int) -> set[int]: + pages = {int(block.get("page", 0) or 0) for block in blocks if int(block.get("page", 0) or 0) > 0} + if page_count >= 4: + start_page = max(2, int(page_count * 0.25)) + end_page = min(int(page_count * 0.7), ...) + selected = {page for page in pages if page != 1 and ...} + else: + selected = {page for page in pages if page != 1} +``` + +**After cover page drop:** +- `page_count` = original N (e.g., 10) +- `pages` = {2, 3, ..., N} (e.g., {2, ..., 10}) +- `start_page = max(2, int(10*0.25)) = 3` +- `end_page = min(7, 8) = 7` +- Selected: {3, 4, 5, 6, 7} — these are real block pages, fine + +The `page != 1` guard was designed before cover page drop existed. It's redundant after the drop (page 1 blocks don't exist), but harmless. + +**After cover page drop with 3-page paper:** +- `page_count` = 3, `pages` = {2, 3} +- `page_count < 4` → `selected = {2, 3}` (correct) + +--- + +## 12. `validate_ocr_meta()` — Page Count Consistency Check + +**File:** `paperforge/worker/ocr.py` lines 174-204 + +```python +rendered_pages = fulltext_path.read_text(encoding="utf-8").count("` for every page from 1 to `page_count`, this check passes even with an empty page 1 marker. **No actual content presence check.** + +--- + +## 13. Off-by-One Analysis Summary + +### Confirmed: Cover Page Drop Creates Page Number Shift + +| Scenario | Block pages | `meta["page_count"]` | Render generates | Issue | +|----------|------------|----------------------|-----------------|-------| +| Normal 10-page paper | 1..10 | 10 | `` through `` | No issue | +| 10-page + cover (page 1 dropped) | 2..10 | **10** (not 9) | `` (empty) through `` | Page values shifted by 1 vs. content page count | +| After cover drop, 9 real content pages | max=10, min=2 | **10** (original) | 10 markers, 1 empty | Content pages are 2..10 but conceptually 1..9 | + +### Potential Off-by-One Locations + +| Location | File:Line | Risk | Explanation | +|----------|-----------|------|-------------| +| `build_raw_blocks_for_result_lines` | `ocr_blocks.py:469` | **Medium** | `for res in payload.get("layoutParsingResults", [])` — if a page has >1 result entry, page counting inflates | +| `postprocess_ocr_result` | `ocr.py:1795` | **Medium** | Same loop as above — duplicate counting if multi-result page | +| Cover page drop (no re-index) | `ocr_blocks.py:297-302` | **HIGH** | Block pages 2..N, `meta["page_count"]` = N, `total_pages` = N. Functions get `page_count=N` but first content page is 2. | +| `_first_surviving_page()` | `ocr_document.py:628-630` | None | Returns correct first page from remaining blocks (e.g., 2). Used correctly by `infer_zones()`. | +| `body_end_page = first_reference_page - 1` | `ocr_document.py:1536` | None internally | Arithmetic is consistent within the shifted numbering. But the VALUE is offset by 1 from conceptual page count. | +| `render_fulltext_markdown` | `ocr_render.py:1891-1893` | None | `for p in range(1, page_count + 1)` correctly covers all pages. Page 1 just has no content blocks. | +| `validate_ocr_meta` | `ocr.py:203` | Medium | `rendered_pages == page_count` passes always, even with empty page 1. Does not detect that page 1 has no content. | +| `_select_middle_sample_pages` | `ocr_families.py:233-237` | Low | Uses original `page_count` (N) for proportion math. After cover drop, proportion is off: N=10 suggests 10 pages but only 9 have content. `start_page=int(10*0.25)=3` vs conceptual `int(9*0.25)=2` — minor and unlikely to affect correctness. | + +### No Offset Found In + +- **`ocr_blocks.py` lines 200-203**: `block.get("page", 1)` — safe default +- **`ocr_index.py` line 31**: `block.get("page", 0)` — verbatim pass-through +- **`ocr_families.py` line 71**: `max(page_count, max_block_page)` — correctly takes the larger value +- **`ocr_families.py` line 229**: `page != 1` guard — redundant after cover drop, not harmful +- **`ocr_document.py` line 1435**: `anchor_page = first_surviving_page if ... <= 2 else 1` — handles cover-drop case explicitly + +### Root Cause of the +1 Offset + +**There is no +1 bug in the numeric sense.** The pipeline does not add or subtract 1 from page numbers during propagation. The issue is: + +1. `meta["page_count"]` reflects the **original PDF page count** (number of `layoutParsingResults` entries), NOT the number of content pages after cover removal +2. Block page values are the **original 1-indexed PDF page numbers**, not renumbered after cover page drop +3. This creates a **systematic +1 shift** when cover page 1 is removed: all downstream consumers see `page: 2` for what is conceptually the first content page + +The design trades off conceptual cleanliness for traceability (block page == original PDF page number). If a consumer expects contiguous 1..N page numbering (e.g., for paginated display), behavior will be wrong after cover page drop. diff --git a/loss_survey_expanded.md b/loss_survey_expanded.md new file mode 100644 index 00000000..6b4c26ee --- /dev/null +++ b/loss_survey_expanded.md @@ -0,0 +1,105 @@ +# OCR Figure N / Table N Loss Survey — Expanded Batch (20 papers) + +Survey date: 2026-07-02 +Method: Compare PDF text layer (PyMuPDF) vs OCR block text on pages where `role-index.json` has **both `media_asset` AND `figure_caption_candidate`** on the same page. + +## Pool Statistics + +| Metric | Value | +|--------|-------| +| Total `role-index.json` files scanned | 729 | +| Papers with both roles on same page | 184 | +| Papers in this batch | 20 | +| Papers with detected "loss" | **12 (60%)** | + +## Individual Paper Results + +### Papers WITH confirmed heading misalignment + +| Paper ID | Target Page(s) | Lost Headings | Where They Landed | +|----------|---------------|---------------|-------------------| +| **24YKLTHQ** | 4 | `table 2`, `Figure 3`, `Figure 4A` | OCR p5 (body_paragraph, figure_caption) | +| **2BFG5P6B** | 4 | `Fig. 2`, `Figure 2`, `Figure 3` | OCR p5 (body_paragraph, figure_caption) | +| **2P6JR629** | 1 | `Fig. 1`, `Fig. 2`, `Table 1`, `Fig. 3A/B/C` (6 total) | OCR p2-18 (mostly body_paragraph) | +| **3CIQWBGA** | 4 | `Table S2` | OCR p5 (body_paragraph) | +| **3FDT9652** | 4 | `Table 3`, `TABLE 3`, `Fig. 6` | OCR p5 (figure_caption, table_caption) | +| **3FQYMMXS** | 3 | `Figure S2/S3`, `Figure 1a/b/c`, `Figure 2a/c/2`, `Table S1/S2`, `Table 1` (11 total) | OCR p4 (mostly body_paragraph) | +| **4DU8LEH2** | 7 | `Fig. 4` | OCR p8 (figure_caption_candidate) | +| **4VEGZ7BN** | 2 | `Fig. 6`, `Table 2`, `Fig. 3`, `Fig. 4` (4 total) | OCR p3-5 (figure_caption, table_caption) | +| **59JLHKIN** | 5 | `Figure 6`, `Table 4`, `Figure 6` (3 total) | OCR p6-7 (figure_caption, table_caption) | +| **5CK2JEIS** | 17 | `Figure 8` | OCR p12/13/15/18 (body_paragraph) | +| **5MAW65YD** | 2 | `Fig. 1`, `Fig. 2` | OCR p3 (body_paragraph) | +| **5QLLVFFG** | 10 | `Fig. 7`, `FIG. 6` | OCR p9/11-13 (body_paragraph, figure_caption) | + +### Papers WITHOUT misalignment (clean) + +29RE4EMX, 2UIPV93M, 3QYCST4E, 4CML4K3Y, 4FY3VQJS, 4M64NZEC, 5PQ7DI4W, 62WG35CZ — all headings present in same-page OCR blocks. + +## Aggregate Statistics + +| Metric | Count | +|--------|-------| +| Total headings checked (in PDF text layer) | 51 | +| Headings "lost" from same-page OCR | **40 (78.4%)** | +| Truly missing from OCR (nowhere in any page) | **0 (0%)** | +| On same page but wrong role | **1 (2.5%)** — `table 2` in 24YKLTHQ p4 | +| On different page entirely | **39 (97.5%)** | + +## Page Offset Distribution + +| Offset | Occurrences | % | +|--------|------------|---| +| +1 | 40 | **59.7%** | +| +2 | 7 | 10.4% | +| +0 (same page, wrong role) | 1 | 1.5% | +| -1 | 3 | 4.5% | +| Other | 16 | 23.9% | +| **Total** | **67** | | + +The dominant pattern is a **+1 page offset**: headings from PDF page N appear on OCR page N+1. + +## Role Distribution at Offset +1 + +| Role at wrong page | Count | % | +|--------------------|-------|---| +| `body_paragraph` | 33 | **71.7%** | +| `figure_caption` | 10 | 21.7% | +| `table_caption` | 2 | 4.3% | +| `figure_caption_candidate` | 1 | 2.2% | + +## Patterns Identified + +### 1. Not a text-extraction problem — it's a page-alignment problem + +The OCR engine (PaddleOCR-VL-1.5) **does capture figure/table heading text**. In all 40 cases of "loss," the heading text exists in the OCR output — just on the wrong page. The heading-to-`figure_caption_candidate` role mapping on the correct page fails because the heading wasn't placed on its PDF-origin page. + +### 2. Systematic +1 forward shift + +The overwhelming pattern is +1 page offset (59.7% of all measured offsets). This suggests a systematic boundary misalignment between the PDF text layer page numbering and the OCR page segmentation: +- PDF front matter (cover, TOC, copyright page) may cause a consistent +1 offset +- OCR may be treating a section start page boundary differently + +### 3. Newline-split headings + +3/12 affected papers had `\n` embedded inside the heading (e.g., `Figure\n3`, `table\n2` in 24YKLTHQ). This suggests that when the heading text crosses a PDF text-layer line boundary, the OCR page-segmentation logic is more likely to misplace it. + +### 4. Multi-column layout correlation + +**8/12 (67%)** of the affected pages had multi-column layout (by text block x-coordinate analysis), vs. typical single-column academic papers. Multi-column layouts stress test the OCR's page segmentation logic, which may explain the misalignment. + +### 5. "Lost" heading tends to degrade to `body_paragraph` + +When a heading shifts to the next page, it's most commonly (72%) assigned the `body_paragraph` role instead of `figure_caption_candidate` or `figure_caption`. This means the figure/table loses its caption association entirely, even though the text is preserved. + +### 6. Papers with many figures per page are hardest hit + +3FQYMMXS (11 lost headings on one page) and 2P6JR629 (6 lost headings on page 1) have dense figure arrays. OCR page segmentation struggles when multiple figure references appear close together, scattering them across subsequent OCR pages. + +## Conclusion + +**The problem is not OCR text loss — it is OCR page-boundary alignment.** Zero headings are missing from the OCR output; 78.4% of PDF figure/table headings are not found in the same OCR page they originate from. The systematic +1 page offset (60% of cases), multi-column correlation (67%), and the degraded role assignment (`body_paragraph` 72%) all point to the page segmentation step as the root cause. + +For the pipeline, this means: +- **`figure_caption_candidate` role assignment misses captions** because the heading text is on page N+1, not page N (where `media_asset` lives) +- The actual number of **true negatives** (headings genuinely lost from OCR) is zero across this sample +- Fixing the page-alignment bug (likely in how the OCR engine maps PDF page numbers to its own page segmentation) would recover essentially all currently "lost" figure/table captions diff --git a/paperforge/worker/ocr_figures.py b/paperforge/worker/ocr_figures.py index d0484153..701bf3c0 100644 --- a/paperforge/worker/ocr_figures.py +++ b/paperforge/worker/ocr_figures.py @@ -3063,6 +3063,12 @@ def build_figure_inventory(structured_blocks: list[dict], page_width: float = 12 } rejected_legends.append(block) continue + # PDF prefix recovery: restore "Figure N" heading missed by OCR + if page_pdf_lines_by_page and _extract_figure_number(text) is None: + recovered = _recover_figure_heading_prefix(block, page_pdf_lines_by_page) + if recovered: + block["text"] = recovered + text = recovered if not _is_formal_legend(text, block, page_width): if rotated_orientation_prematch: block["_rotated_caption_prematch"] = True @@ -6035,3 +6041,62 @@ def _recover_missing_figure_numbers_from_assets( fig["flags"] = list(dict.fromkeys((fig.get("flags") or []) + [ "figure_number_recovered_from_asset_text", ])) + + +def _recover_figure_heading_prefix( + block: dict, + page_pdf_lines_by_page: dict[int, list[dict]], +) -> str | None: + """Recover 'FIGURE N' heading prefix from PDF text layer for OCR-missed captions. + + When PaddleOCR fails to detect a standalone 'Figure N' / 'FIGURE N' heading + (e.g. rendered in a bold/small-caps font that the OCR engine doesn't read), + the caption body is captured as figure_caption_candidate but lacks the + figure number prefix. This function checks the PDF text layer for the + heading on the same page and prepends it if the line immediately after + the heading matches the start of the caption text. + + Returns the complete caption text with prefix, or None if no recovery. + """ + page = int(block.get("page", 0) or 0) + pdf_lines = page_pdf_lines_by_page.get(page) + if not pdf_lines: + return None + + block_text = str(block.get("text", "") or "").strip() + if not block_text or len(block_text) < 20: + return None + + # Sort PDF lines top-to-bottom by y-center + sorted_lines = sorted( + pdf_lines, + key=lambda l: (l.get("bbox") or [0, 0, 0, 0])[1], + ) + + for i, line in enumerate(sorted_lines): + line_text = str(line.get("text", "") or "").strip() + # Check: line starts with Figure N heading (short heading, not in-text ref) + m = _FIGURE_NUMBER_PATTERN.match(line_text) + if not m or m.start() != 0: + continue + if len(line_text) > 40: + continue # too long for a heading — probably in-text reference + + # Check the next PDF line (by y-order) matches the start of block text + if i + 1 >= len(sorted_lines): + continue + next_text = str(sorted_lines[i + 1].get("text", "") or "").strip() + if not next_text or len(next_text) < 10: + continue + + # Common-prefix match: at least 15 chars case-insensitive + common = 0 + for a, b in zip(next_text.lower(), block_text.lower()): + if a == b: + common += 1 + else: + break + if common >= 15: + return line_text + "\n" + block_text + + return None