Commit graph

1175 commits

Author SHA1 Message Date
Research Assistant
ee5cd0df4e fix: render section_heading with Markdown heading prefix 2026-06-07 12:53:04 +08:00
Research Assistant
088fb4144d fix: prevent article-type running headers from becoming section headings 2026-06-07 12:52:02 +08:00
Research Assistant
644f14dad6 fix: gate non-body insert detection by spine reliability 2026-06-07 12:00:18 +08:00
Research Assistant
a723993bb9 feat: add explicit no-span degraded OCR mode 2026-06-07 11:48:58 +08:00
Research Assistant
10fe2251b5 feat: recover source PDFs for old-paper span backfill 2026-06-07 11:47:09 +08:00
Research Assistant
2404a486f0 feat: expose OCR body spine quality and anchor diagnostics 2026-06-07 11:45:44 +08:00
Research Assistant
6bdaddb366 fix: prevent author bylines from becoming section headings 2026-06-07 11:45:11 +08:00
Research Assistant
dcc07140b0 feat: anchor OCR body baselines from stable middle pages 2026-06-07 11:43:36 +08:00
Research Assistant
ab2f581ae9 fix: complete container backmatter detection with backmatter_boundary_candidate support and multi-column container child promotion 2026-06-07 01:28:26 +08:00
Research Assistant
822895d77e fix: remove redundant figure caption prose check on resolved captions 2026-06-07 01:04:15 +08:00
Research Assistant
0fbd4e186c fix: extend render fallback normalization guards to all tail and candidate roles 2026-06-07 01:02:06 +08:00
Research Assistant
ccff2aef3b refactor: replace dead reorder import with direct column-major fallback 2026-06-07 01:00:45 +08:00
Research Assistant
0ac980ebbe fix: remove dead _verified_by_seed guard from reference demotion 2026-06-07 01:00:22 +08:00
Research Assistant
3e748dcb5d fix: gate weak boundary signals through backmatter_boundary_candidate 2026-06-07 00:59:22 +08:00
Research Assistant
bf8c1be53a fix: restore column-major page ordering in legacy render path 2026-06-07 00:30:52 +08:00
Research Assistant
e5d2877f3c fix: restore render fallback for synthetic test inputs, preserve funding/backmatter roles 2026-06-06 21:54:15 +08:00
Research Assistant
32c87a903d test: add tail reading order stabilization regression test 2026-06-06 19:37:08 +08:00
Research Assistant
ab24f19b91 fix: figure_id missing from matched figures, subfigure narrative hard reject 2026-06-06 19:31:44 +08:00
Research Assistant
b017e78828 fix: proper Funding handling in container backmatter, fix single-column tail reorder 2026-06-06 19:25:55 +08:00
Research Assistant
45768a6d5f fix: restore figure pipeline, fix Funding suppression, tighten candidate resolution 2026-06-06 19:11:51 +08:00
Research Assistant
f2e62c5773 fix: include figure_caption_candidate in non_body_insert detection 2026-06-06 18:54:03 +08:00
Research Assistant
fe1d9467ca fix: make figure inventory distrust bad legend roles with prose narrative check 2026-06-06 18:50:47 +08:00
Research Assistant
9bfd4354ca fix: add candidate resolution and activation gates to document normalization 2026-06-06 18:46:20 +08:00
Research Assistant
e071a0b6f5 fix: demote ambiguous seed roles to candidates (figure_caption, backmatter_heading) 2026-06-06 18:40:24 +08:00
Research Assistant
5c7e439696 fix: render mixed tail pages with layout-aware reference zones 2026-06-06 17:57:55 +08:00
Research Assistant
5cbf6ae066 fix: make reference rescue depend on local reference zones 2026-06-06 17:39:15 +08:00
Research Assistant
32e4a67f8d refactor: add local reference zones and layout-aware tail boundary detection 2026-06-06 17:35:08 +08:00
Research Assistant
9f0bc39692 feat: derive layout-aware tail reading segments 2026-06-06 17:28:03 +08:00
Research Assistant
2c7ab8aa62 feat: persist layout-aware OCR tail analysis artifacts 2026-06-06 17:25:47 +08:00
Research Assistant
c21f04d7b6 feat: add page layout profiling for OCR document analysis 2026-06-06 17:23:41 +08:00
Research Assistant
c56ce87a1c fix: add relational-operator spacing and Greek-letter preservation to math normalization 2026-06-06 17:19:33 +08:00
Research Assistant
cdf7deffc9 fix: catch body-width continuation fragments in non_body_insert detection
Block 35 in SAN9AYVR was a biography continuation fragment that escaped
non_body_insert detection because it shared the same width and font as
the body spine.  Added a second-pass expansion in
_detect_non_body_insert_clusters that catches orphan continuation
fragments (lowercase start, same font as insert cluster, adjacent to
detected insert) and marks them as non_body_insert.

Regression test: test_non_body_insert_catches_continuation_fragment
2026-06-06 17:14:43 +08:00
Research Assistant
06d7abb2fa feat: unify OCR math normalization across render and objects 2026-06-06 17:02:01 +08:00
Research Assistant
4ecdee5ff6 fix: close residual non-body insert leakage from figure_caption blocks 2026-06-06 16:50:49 +08:00
Research Assistant
e41895d4d6 fix: add geometry guard to non_body_insert rescue — narrow blocks (<35% page width) stay non_body_insert regardless of font match; require strong non_body_insert_family quality for reinstatement 2026-06-06 16:22:46 +08:00
Research Assistant
0175fc3ab3 fix: tighten non_body_insert detection and rescue — exclude frontmatter_noise from role filter, require moderate+ family quality for reinstatement 2026-06-06 16:15:38 +08:00
Research Assistant
463852462f fix: stop legend-only figures from using fake unmatched assets 2026-06-06 16:08:26 +08:00
Research Assistant
dcfbe8bb64 fix: exclude non-body insert media from figure asset pool 2026-06-06 16:06:36 +08:00
Research Assistant
cb3f459ba2 fix: prioritize formal figure legends over panel subcaptions 2026-06-06 16:01:57 +08:00
Research Assistant
69a1156449 fix: expand non_body_insert detection to catch frontmatter_noise blocks; use relative page bound
- Expand role filter in _detect_non_body_insert_clusters from body_paragraph
  only to include frontmatter_noise and unknown_structural
- Replace absolute page<=3 gate with relative bound based on body_end_page:
  max_early_page = min(3, body_end_page/4 + 1)
- Added test_detect_non_body_insert_catches_frontmatter_noise_blocks
2026-06-06 15:56:14 +08:00
Research Assistant
67bd36c997 refactor: make document structure the single render truth
- build_structured_blocks now returns (rows, doc_structure) tuple
- ocr.py and ocr_rebuild.py pass document_structure to render_fulltext_markdown
- bare except Exception replaced with structured logging
2026-06-06 15:52:18 +08:00
Research Assistant
3dc06eb9db feat: derive section family profiles from normalized OCR structure 2026-06-06 15:49:50 +08:00
Research Assistant
562ae5ca93 fix: author anchor & non_body_insert — remove open-ended authors, add Zotero anchor, font-family cluster detection, furniture/received/published word-fix, affiliation keyword cleanup 2026-06-06 14:08:32 +08:00
Research Assistant
1e10f9edd6 fix: add non_body_insert guard in frontmatter candidate extraction 2026-06-06 13:51:51 +08:00
Research Assistant
f40548e8a1 feat: enhance body spine detection with font-family collection from span metadata 2026-06-06 13:51:16 +08:00
Research Assistant
fd00a5add2 fix: guard rescue from promoting non_body_insert blocks to body roles 2026-06-06 13:50:24 +08:00
Research Assistant
b4951597e7 feat: add non_body_insert role for early-page non-body insert clusters 2026-06-06 13:48:30 +08:00
Research Assistant
936902a7d6 feat: add Zotero-anchored author alignment check in resolve_metadata 2026-06-06 13:33:20 +08:00
Research Assistant
c974ba7480 fix: remove open-ended authors text heuristics, anchor to page-1 zone only 2026-06-06 13:28:46 +08:00
Research Assistant
2bf7e73fff feat: persist document_structure.json, thread structure truth through downstream pipeline 2026-06-06 13:10:56 +08:00