Three-part fix:
1. ocr_roles.py: add inline <table> check before raw_label=table → media_asset
fallback. Blocks starting with <table> now get table_html directly regardless
of raw_label.
2. ocr_structural_gate.py: add table_html verifier — inline <table> HTML is
self-identifying, accepted without structural verification.
3. ocr_document.py: remove table_html→table_html_candidate conversion step.
This role was never handled by any downstream pipeline, causing blocks to
be downgraded to unknown_structural.
Validated on AH6Q7DLC (worst case, 30 blocks): 29/30 now correctly table_html,
1 remaining is reference_item (bibliography table, different classification).
Full corpus data pending rebuild.
585 figure/table/role tests pass.
1. _looks_like_initial_lastname_byline: require at least one lowercase letter.
Prevents journal taglines like 'A RESEARCH VISION' from matching the
initial-lastname pattern (e.g. 'J. Smith'). Fixes 24YKLTHQ block 7.
2. _is_first_page_body_start: metadata headings no longer trigger body_start.
Page 1 section_heading for 'INFORMACIÓN DEL ARTÍCULO' (Article Information)
is frontmatter metadata, not body content. Only known body-heading terms
(introduction, methods, results, discussion, conclusion) trigger body_start.
3. build_document_abstract_span backward scan: exclude non-English keyword
headings ('palabras clave', 'mots clés', 'schlagwörter') from the abstract
body candidate set. Prevents Spanish keywords block from being collected
as abstract body.
These changes fix 24YKLTHQ: block 17 (abstract) now correctly assigned
abstract_body instead of body_paragraph, block 7 no longer mislabeled as
authors, and page 1 frontmatter zone covers 5 more blocks.
- CAQ: Move correspondence detection before zone-based furniture check
(ocr_roles.py). Correspondence footnote now correctly gets
frontmatter_support instead of frontmatter_noise.
- DW: Accept raw_label=doc_title on page 2 as paper_title repeat
(ocr_structural_gate.py). Fixes title re-verification that was
held to unknown_structural.
- DW: Tag keywords block as structured_insert when abstract span
stop_reason=keywords (ocr_document.py). Uses abstract span boundary
signal instead of text matching.
- Zone fallback: Fix unassigned role resolution in ref_heading_pages
and last_body_heading_top pre-scans (ocr_document.py). Blocks with
role=unassigned now fall through to seed_role. Also populate
ref_heading_pages from first reference_item on pages without a
reference_heading.
- Updated expectations for DW (7 FAIL, down from 9) and CAQ (0 FAIL,
down from 3). All remaining DW FAILs are user-decided skips.
The source-anchor override for authors matched block_id=2 on EVERY page,
not just page 1. Added page check: only accept anchored role if block is
on the same page as the source anchor. Prevents section headings,
reference items, and figure captions from being misclassified as authors.
Also updated regenerate_traces.py sys.path to use main checkout instead
of worktree, and regenerated both DW (287 blocks) and CAQ (153 blocks)
traces.
- Pass source_frontmatter_anchors to normalize_document_structure so gate
has source-backed IDs before running (fixes paper_title/authors always HELD)
- Add table_caption CANDIDATE handler (was falling through to unknown_structural)
- Author matching: handle & splitting, strip trailing labels, initial-based
matching for abbreviated names, subset match for truncated author lists
- Override frontmatter_noise preservation when seed_role is VERIFY_REQUIRED
- Accept authors role from source anchor regardless of seed_role
- Enrich source_metadata authors from formal-library.json fallback
- Heading merge: use vertical_gap <= 30px instead of sentence-end heuristic
All 280 tests pass. TSCKAVIS/DWQQK2YB/CAQNW9Q2 rebuilt and verified.
