Commit graph

823 commits

Author SHA1 Message Date
Research Assistant
3caeb2be1f fix: bind table notes to table ownership contract 2026-06-13 17:01:59 +08:00
Research Assistant
1a6e9a8630 fix: prevent summary-page caption from duplicating formal figure-page caption 2026-06-13 16:59:21 +08:00
Research Assistant
8209e01870 fix: enforce body-column completion before reference zone on mixed pages 2026-06-13 16:51:10 +08:00
Research Assistant
1ed6cd8030 fix: deny body ordinal openings from becoming reference_item 2026-06-13 16:44:02 +08:00
Research Assistant
7b8ed4be0e fix: close OCR v2 real-paper structural, object, and render gaps 2026-06-12 17:41:57 +08:00
Research Assistant
e4d050c298 fix: close OCR v2 real-paper structural and object gaps 2026-06-12 17:24:57 +08:00
Research Assistant
e9a3608488 fix: tighten OCR real-paper abstract span boundaries
Add three exclusion layers to build_document_abstract_span():
- seed_role-based frontmatter exclusion (authors, email, affiliation,
  doi, correspondence, highlights, author_contribution, data_availability)
- text-based Highlights block exclusion
- structured abstract subhead exclusion (background, objective,
  methods, results, conclusions, etc.)

These prevent author lines, affiliations, Highlights, Keywords, and
structured abstract labels from leaking into the abstract body.
2026-06-12 17:12:51 +08:00
Research Assistant
8dfc0a0425 feat: close OCR object gate for reader outputs
When figure_caption_candidate blocks lack explicit marker_type or caption_text in the strict inventory, _normalize_bucket now falls back to the structured block's text and infers figure_number/marker_type from the text content. This ensures candidate captions propagate through the reader-figure pipeline to the renderer and health summary.
2026-06-12 17:03:25 +08:00
Research Assistant
669c3bfe07 fix: repair OCR renderer object emit and tail heading suppression 2026-06-12 16:52:32 +08:00
Research Assistant
e4aa9da015 fix: remove unsafe OCR gate fallbacks 2026-06-12 16:47:07 +08:00
Research Assistant
c44dd17fb9 fix: repair OCR reference zone heading acceptance 2026-06-12 16:43:33 +08:00
Research Assistant
31d7cad7c7 test: add missing plan-specified gate health and render tests 2026-06-12 15:21:14 +08:00
Research Assistant
d9f76917e9 style: ruff format after OCR role gate implementation 2026-06-12 15:11:33 +08:00
Research Assistant
ee90b3fa41 test: add OCR v2 structural contract audits 2026-06-12 15:05:49 +08:00
Research Assistant
04cbc9a443 feat: merge OCR role gate into health 2026-06-12 13:53:40 +08:00
Research Assistant
5c66e9d92b fix: render OCR fulltext from verified artifacts 2026-06-12 13:52:45 +08:00
Research Assistant
a65d992ab8 fix: prevent OCR role mutation after structural gate 2026-06-12 13:49:14 +08:00
Research Assistant
37106695e5 feat: install OCR verified role gate 2026-06-12 13:48:07 +08:00
Research Assistant
d1ff2179e9 feat: add OCR role gate context adapters 2026-06-12 12:19:11 +08:00
Research Assistant
6069a01928 feat: verify OCR reference zone from document artifacts 2026-06-12 12:17:20 +08:00
Research Assistant
8fea106c1b feat: build OCR document abstract spans 2026-06-12 12:15:48 +08:00
Research Assistant
23e95a72fd feat: add OCR structural gate safe defaults 2026-06-12 12:10:57 +08:00
Research Assistant
679da738e9 fix(ocr): cross-page block_id collision in reader normalization causes wrong figure_number
_index_structured_blocks used block_id as dict key, but PaddleOCR assigns
the same block_id for blocks on different pages. When A8E7SRVS page 6
block_id=5 (Fig.5) overwrites page 5 block_id=5 (Fig.1), reader output
rendered Figure 5 with Fig.1 caption text.

Add _index_blocks_by_page_and_id using (page, block_id) tuples for
page-aware lookup in _normalize_bucket. Keep serialized block_index
clean (plain int|str keys) to preserve JSON compatibility.

Add regression test for cross-page block_id collision.
2026-06-11 21:48:59 +08:00
Research Assistant
bef3ff540e fix: reader cards skip caption for matched figures, suppress backmatter at all heading levels, restore figure embeds alongside cards 2026-06-11 19:58:23 +08:00
Research Assistant
3bd51dee25 fix: suppress panel labels, backmatter headings, fix figure placement — add render-hygiene audit gate 2026-06-11 19:49:52 +08:00
Research Assistant
adb640c98e fix: close OCR figure reader contract — page-scoped block ids, legacy render suppression, per-paper audit gate 2026-06-11 16:25:12 +08:00
Research Assistant
41ebec7d89 fix: tighten OCR strict-figure safety and audit hygiene 2026-06-11 14:53:34 +08:00
Research Assistant
4cbad1c943 fix: make OCR figure rendering reader-primary
- Move consumed_caption_block_ids skip to top of block loop (before role
  dispatch) so body_paragraph / backmatter_body blocks with consumed
  captions are also skipped, not just figure_caption blocks.
- Create _emit_page_objects() helper to eliminate 4 duplicated page-
  emission loops. Reader figures are primary: when present, legacy
  matched_figures and unresolved_clusters are skipped for that page.
- Add tests: consumed-caption dedupe for body_paragraph, reader-figures-
  preferred-over-legacy-matched_figures.
2026-06-11 14:16:28 +08:00
Research Assistant
8c709177e7 fix: restore OCR reader normalization contract
Preserve gate-critical fields in _normalize_bucket():
- marker_type: fall back to source_item['marker_type'] when marker_signature is absent
- strict_reject, linked_legend_block_id, cluster_area_ratio, width_ratio, height_ratio, media_block_count: carry through from source_item

Fixes 6 failing tests that relied on these fields surviving normalization.
2026-06-11 13:33:54 +08:00
Research Assistant
5e67b13f88 fix: stop leaking OCR debug figure artifacts into fulltext 2026-06-11 02:49:33 +08:00
Research Assistant
24bd3868c0 fix: restore figure semantics in OCR reader normalization 2026-06-11 02:41:28 +08:00
Research Assistant
7697e86463 test: lock OCR figure reader contract on real papers 2026-06-10 20:50:57 +08:00
Research Assistant
a0a6e7f42e feat: add strict OCR sequence match promotion 2026-06-10 20:48:57 +08:00
Research Assistant
3fdda60adf feat: render OCR reader figures without debug leakage 2026-06-10 20:47:25 +08:00
Research Assistant
c86700b6f2 feat: persist OCR reader figure artifacts and health metrics 2026-06-10 20:43:54 +08:00
Research Assistant
7ec1f66da7 feat: materialize OCR reader figures and hold semantics
- Rewrite _materialize_reader_figure() to handle all source types:
  matched_figures -> EXACT_MATCH, held/ambiguous with candidates ->
  GROUPED_APPROXIMATE, held/ambiguous without candidates -> LEGEND_ONLY,
  unmatched_legends -> LEGEND_ONLY, unresolved_clusters -> ASSET_GROUP_ONLY
- Add _materialize_hold_outcome() for reader_hold vs audit_hold semantics
- Add _candidate_asset_ids_from_item() for flat candidate_asset_ids lists
- GROUPED_APPROXIMATE now requires visual candidates (no candidates -> LEGEND_ONLY)
- legend_block_id no longer consumed by audit_hold, only reader_hold
- Coverage now tracks gap (total - accounted) correctly
- 8 reader figure tests, 280 total tests passing
2026-06-10 20:26:55 +08:00
Research Assistant
b2d76f7d39 feat: add OCR reader input gating and deduplicated coverage 2026-06-10 20:23:18 +08:00
Research Assistant
df95b946a7 feat: add OCR reader figure schema and stable ids
- _stable_reader_figure_id: deterministic ID generation for reader figures
  (figure_number-based or visual_group fallback with page+asset)
- synthesize_reader_figures: shell that normalizes inventory, materializes
  reader figures with separate reader_status (RESOLVED/GROUPED_APPROXIMATE/
  DEFERRED/CAPTION_ONLY/ORPHAN_ASSETS) and strict_status (preserved from
  source bucket)
- _materialize_reader_figure: maps normalized items to reader figure dicts
- ReaderCoverage dataclass extended with as_dict()
- Tests: status separation, stable ID generation, normalization round-trip
2026-06-10 20:19:16 +08:00
Research Assistant
68ee5f735a feat: normalize strict OCR figure inventory for reader synthesis 2026-06-10 20:16:21 +08:00
Research Assistant
4d0c99dd76 fix: close OCR real-paper anchor-first regressions 2026-06-10 18:18:48 +08:00
Research Assistant
aeb89bd626 fix: close OCR figure and legend object gaps
Add figure legend completeness check ensuring every numbered formal
legend lands in an explicit outcome bucket (matched/held/ambiguous/
unresolved_cluster/unmatched). No legend may disappear without inventory
explanation.

Changes:
- Add compute_figure_legend_completeness() to ocr_figures.py
- Integrate completeness into build_figure_inventory() return value
- Add figure_legend_completeness fields to health report (ocr_health.py)
- Gaps degrade overall health status and add degraded reasons
- Add 9 new tests for completeness edge cases
2026-06-10 13:24:08 +08:00
Research Assistant
adb33b2588 fix: honor OCR display and frontmatter authority in render 2026-06-10 13:13:59 +08:00
Research Assistant
f35470fb70 fix: make non-body families authoritative against body flow 2026-06-10 13:00:44 +08:00
Research Assistant
b9b661dbad fix: exclusion functions check seed_role when role is unassigned
_exclude_tail_nonref_from_body_flow and
_exclude_frontmatter_side_from_body_flow now resolve effective_role
from seed_role when role is 'unassigned' (post-Task-2 state).
Excluded blocks also update seed_role to prevent resolve_final_role
from overriding the exclusion.

Regression from Task 2's seed_role split: exclusion functions ran
before resolve_final_role and saw 'unassigned' instead of
'body_paragraph', so tail non-ref content leaked into body.
2026-06-10 12:49:32 +08:00
Research Assistant
03f21e324b fix: type OCR old-style references as reference markers 2026-06-10 12:41:39 +08:00
Research Assistant
02a3eda56a test: add OCR real-paper regression tests and cleanup 2026-06-10 12:36:15 +08:00
Research Assistant
5758feb6c7 refactor: reorder OCR anchor-first pipeline 2026-06-10 12:35:45 +08:00
Research Assistant
b56b7e287e refactor: split OCR seed and final roles 2026-06-10 12:06:16 +08:00
Research Assistant
d8d766b56e fix: keep OCR tail support out of body flow 2026-06-10 01:54:43 +08:00
Research Assistant
e3fdc8b942 fix: close OCR reference zone authority on real papers 2026-06-10 01:50:08 +08:00