Add _merge_box_content() post-processing to normalize_document_structure:
detects structured_insert blocks whose text starts with Box N or Key
insights, then consumes following body-like blocks as box content.
Consumed blocks get role=noise + render_default=False to suppress
double-rendering. Column-crossing boxes (common in multi-column
layouts) are handled by removing the x-overlap heuristic.
In _normalize_backmatter_roles_after_boundary, respect upstream
seed_role classification for frontmatter_noise blocks. Only convert
frontmatter_noise to backmatter_body when seed_role differs (indicating
misclassification), not when it matches (indicating intentional
watermark/noise suppression).
Also remove internal reader_status labels (EXACT_MATCH, LEGEND_ONLY,
ASSET_GROUP_ONLY) from rendered fulltext output.
- ocr_render.py: extend consumed frontmatter roles skip from page 1 to pages <= 2,
preventing paper_title appearing twice on pre-proof papers
- ocr_document.py: add image/media block promotion in backmatter zone normalization
so raw image blocks become media_asset instead of backmatter_body; this fixes
build_figure_inventory producing 0 matched_figures. Also check seed_role in
_sanitize_reference_zone_boundary since it runs before role resolution
- scripts/dev/ocr_rebuild_paper.py: single-paper rebuild + block_trace regeneration
entry point (will be wired into CLI later)
- tests: boundary protection and completeness check tests
- DW trace regenerated with correct role distribution
- CAQ: Move correspondence detection before zone-based furniture check
(ocr_roles.py). Correspondence footnote now correctly gets
frontmatter_support instead of frontmatter_noise.
- DW: Accept raw_label=doc_title on page 2 as paper_title repeat
(ocr_structural_gate.py). Fixes title re-verification that was
held to unknown_structural.
- DW: Tag keywords block as structured_insert when abstract span
stop_reason=keywords (ocr_document.py). Uses abstract span boundary
signal instead of text matching.
- Zone fallback: Fix unassigned role resolution in ref_heading_pages
and last_body_heading_top pre-scans (ocr_document.py). Blocks with
role=unassigned now fall through to seed_role. Also populate
ref_heading_pages from first reference_item on pages without a
reference_heading.
- Updated expectations for DW (7 FAIL, down from 9) and CAQ (0 FAIL,
down from 3). All remaining DW FAILs are user-decided skips.
In post_reference_backmatter_zone, blocks with role=reference_item are
outside the verified reference range. The structural gate cannot verify
them and holds them to unknown_structural. Fix: the zone normalization
now converts ALL non-heading blocks (including reference_item) to
backmatter_body, which the gate accepts via _SAFE_PRESERVED_ROLES.
DW expectations: 18 FAIL -> 9 FAIL (fixed 8 biography blocks + 1 equal
contribution note). Remaining 9 are pre-existing known bugs.
The source-anchor override for authors matched block_id=2 on EVERY page,
not just page 1. Added page check: only accept anchored role if block is
on the same page as the source anchor. Prevents section headings,
reference items, and figure captions from being misclassified as authors.
Also updated regenerate_traces.py sys.path to use main checkout instead
of worktree, and regenerated both DW (287 blocks) and CAQ (153 blocks)
traces.
Adds document-level promotion rules in normalize_document_structure() so
that backmatter_heading_candidate blocks get promoted to backmatter_heading
when followed by tail-like body text. Uses text evidence (conflict-of-interest
phrases, backmatter vocabulary) + local follower evidence (short body
paragraphs) — no paper-specific checks or literal page-number logic.
Closes task 6 of the OCR-v2 pipeline.
- Pass source_frontmatter_anchors to normalize_document_structure so gate
has source-backed IDs before running (fixes paper_title/authors always HELD)
- Add table_caption CANDIDATE handler (was falling through to unknown_structural)
- Author matching: handle & splitting, strip trailing labels, initial-based
matching for abbreviated names, subset match for truncated author lists
- Override frontmatter_noise preservation when seed_role is VERIFY_REQUIRED
- Accept authors role from source anchor regardless of seed_role
- Enrich source_metadata authors from formal-library.json fallback
- Heading merge: use vertical_gap <= 30px instead of sentence-end heuristic
All 280 tests pass. TSCKAVIS/DWQQK2YB/CAQNW9Q2 rebuilt and verified.
