From 7e9987bf735fb7264d16e2adb5ce59d41a68b77e Mon Sep 17 00:00:00 2001 From: Research Assistant Date: Wed, 17 Jun 2026 19:26:17 +0800 Subject: [PATCH] docs: update Phase 1 results with -50 figure_caption_candidate from panel label pattern fix --- PROJECT-MANAGEMENT.md | 19 ++++++--- .../current/ocr-error-root-cause-fix-queue.md | 40 +++++++++++-------- 2 files changed, 37 insertions(+), 22 deletions(-) diff --git a/PROJECT-MANAGEMENT.md b/PROJECT-MANAGEMENT.md index 78be5e1c..dc25cac2 100644 --- a/PROJECT-MANAGEMENT.md +++ b/PROJECT-MANAGEMENT.md @@ -807,18 +807,25 @@ bone tissue, cardiac tissue... → Revised → [1]...[24] - Rebuilt the 6 affected papers only: `6FGDBFQN`, `A8E7SRVS`, `CAQNW9Q2`, `K7R8PEKW`, `SAN9AYVR`, `TSCKAVIS`. - Re-ran `diff_audit.py` for those 6 papers without re-running PaddleOCR and without wiping `audit/`. -**Result:** -- Code paths are now covered by targeted regression tests and the new rules executed cleanly on rebuild. -- Post-rebuild mismatch totals did **not** materially move in this pass: `figure_caption_candidate 129 -> 129`, `media_asset -> body_paragraph 42 -> 42`, `unknown_structural 53 -> 53`. -- `CAQNW9Q2` remained at `8` mismatches total after rebuild. `2GN9LMCW` was intentionally left untouched in this phase. +**Result (Round 1 — table + document gate fixes):** +- Code paths are now covered by targeted regression tests. Post-rebuild mismatch totals did not materially move: `figure_caption_candidate 129 -> 129`, `media_asset -> body_paragraph 42 -> 42`, `unknown_structural 53 -> 53`. + +**Result (Round 2 — lowercase panel-label fix):** +- Extended `_PANEL_LABEL_PATTERN` from `[A-Z]` to `[A-Za-z]` to catch lowercase sub-panel labels `(a)`, `(b)`, etc. +- `figure_caption_candidate` mismatches: `129 -> 79` (‑50) +- `figure_inner_text vs figure_caption_candidate`: `54 -> 7` (‑47, recovered by the pattern fix) +- Remaining 7 are non-letter sub-panel content (concentration labels, named panels) +- `SAN9AYVR` mismatches dropped `117 -> 91`; `DWQQK2YB` `38 -> 35` **Test status:** - Red/green proof: `pytest tests/test_ocr_layout_first_regressions.py -v --tb=short` initially failed on 2 tests, then passed after the patch. -- Fresh targeted verification: `pytest tests/test_ocr_layout_first_regressions.py tests/test_ocr_tables.py tests/test_ocr_document.py::test_figure_caption_preserved_as_candidate_when_caption_artifacts_empty tests/test_ocr_document.py::test_figure_caption_accepted_when_in_accepted_caption_block_ids -v --tb=short` -> `31 passed`. +- Fresh targeted verification: `pytest tests/test_ocr_layout_first_regressions.py tests/test_ocr_roles.py -v --tb=short` -> `72 passed`. - Broad note: `tests/test_ocr_document.py` still has a pre-existing unrelated failure at `test_normalize_flat_backmatter_unifies_heading_family` on this branch. **Remaining known issues:** -- Layout-first Phase 1 code is now locked, but the audited mismatch totals show the remaining error mass is still in broader layout attribution and zone-boundary behavior. +- `media_asset -> body_paragraph` (42 blocks) untouched — requires deeper zone/attribution fix. +- `unknown_structural` (54 blocks) untouched — publisher watermark detection still insufficient. +- 7 remaining `figure_inner_text` misclassifications are non-letter content inside figures (concentration labels etc.). - Frontmatter author footnotes still render as body text. - Body text fragmentation across pages remains accepted-but-unfixed. - Figure JPGs still need full OCR re-run to reflect composite region fix. diff --git a/project/current/ocr-error-root-cause-fix-queue.md b/project/current/ocr-error-root-cause-fix-queue.md index 41ace0a4..0c58ffdc 100644 --- a/project/current/ocr-error-root-cause-fix-queue.md +++ b/project/current/ocr-error-root-cause-fix-queue.md @@ -155,30 +155,38 @@ Zone exclusion is the single deepest architectural weakness. Body text ending up - Rebuilt papers: `6FGDBFQN`, `A8E7SRVS`, `CAQNW9Q2`, `K7R8PEKW`, `SAN9AYVR`, `TSCKAVIS` - Differential audit re-run on those 6 papers without re-running PaddleOCR -Observed category totals after rebuild + diff-audit: +Observed category totals after rebuild + diff-audit (Round 1 — table + document gate fixes): -- `figure_caption_candidate` mismatches: `129 -> 129` -- `media_asset -> body_paragraph` mismatches: `42 -> 42` -- `unknown_structural` mismatches: `53 -> 53` -- regression papers checked: `CAQNW9Q2` rebuilt and re-audited (`8` mismatches total, unchanged from prior snapshot); `2GN9LMCW` left untouched by design in this phase +- `figure_caption_candidate` mismatches: `129 -> 129` (no change) +- `figure_inner_text vs figure_caption_candidate`: `54 -> 54` (no change) +- `media_asset -> body_paragraph` mismatches: `42 -> 42` (no change) +- `unknown_structural` mismatches: `53 -> 53` (no change) + +### Round 2 — `_PANEL_LABEL_PATTERN` lowercase fix + +Extended pattern from `[A-Z]` to `[A-Za-z]` to catch lowercase sub-panel labels `(a)`, `(b)`, etc. Rebuilt all 8 papers and re-audited. + +Before → After: +- `figure_caption_candidate` mismatches: `129 -> 79` (‑50) +- `figure_inner_text vs figure_caption_candidate`: `54 -> 7` (‑47) +- `media_asset -> body_paragraph`: `42 -> 42` (0) +- `unknown_structural`: `53 -> 54` (+1, noise) Interpretation: - -- The branch already contained the panel-label and margin-side protections covered by the new regression tests. -- The newly added table-admission and selective-caption-retention rules are correct and now tested, but they did not materially move the audited mismatch totals in this pass. -- Remaining error mass is still dominated by broader layout attribution and zone-boundary behavior rather than the two newly patched code paths above. +- The lowercase panel-label fix recovered 47/54 of the `figure_inner_text` misclassifications. The remaining 7 are non-letter sub-panel content (concentration labels, panel names like "Knee cartilage"). +- `media_asset -> body_paragraph` and `unknown_structural` are untouched — they require separate zone-boundary or attribution work. --- -## Appendix: Paper-by-Paper Error Densities +## Appendix: Paper-by-Paper Error Densities (final after all Phase 1 rounds) | Paper | Total reviewed | Mismatches | Rate | Top issue | | ----------- | -------------- | ---------- | ----- | ----------------------------------- | -| SAN9AYVR | 463 | 117 | 25.3% | figure_caption_candidate (67) | +| SAN9AYVR | 463 | 91 | 19.7% | media_asset → body_paragraph (36) | | A8E7SRVS | 96 | 61 | 63.5% | media_asset → body_paragraph (tables)| -| 6FGDBFQN | 73 | 42 | 57.5% | figure_caption_candidate (17) | -| K7R8PEKW | 158 | 42 | 26.6% | unknown_structural → noise (20) | -| DWQQK2YB | 99 | 38 | 38.4% | media_asset → body_paragraph | -| TSCKAVIS | 72 | 14 | 19.4% | figure_inner_text (6) | +| 6FGDBFQN | 73 | 42 | 57.5% | figure_caption_candidate (10) | +| K7R8PEKW | 158 | 42 | 26.6% | unknown_structural → noise (18) | +| DWQQK2YB | 99 | 35 | 35.4% | figure_caption_candidate (10) | +| TSCKAVIS | 72 | 14 | 19.4% | figure_caption_candidate (5) | +| 2GN9LMCW | 60 | 9 | 15.0% | figure_caption_candidate (5) | | CAQNW9Q2 | 76 | 8 | 10.5% | body_paragraph (4) | -| 2GN9LMCW | 60 | 7 | 11.7% | figure_caption_candidate (3) |