Commit graph

9 commits

Author SHA1 Message Date
LLLin000
914acd6aa5 feat: enable OCR_PIPELINE_V3 by default
The V3 pipeline has been tested against the full 555-paper vault vs legacy:

  No diff: 547 (98.6%)
  Diff:     5 (0.9%) — all in v3's favor: 3 papers find 1 more figure
                        (v3 softens role before matching), 2 papers shift
                        1 block boundary (body_paragraph vs frontmatter_noise)
  Error:    3 (0.5%) — 2 missing raw data, 1 pre-existing v3 pipeline edge case

The 5 diffs are marginal improvements, not regressions. V3's shadow-normalize
approach defers role commit until after figure/table matching, so captions
that legacy normalize prematurely hardened to 'body_paragraph' are correctly
found as figure captions.

Changes:
- _ocr_pipeline_v3_enabled() defaults to True now; set OCR_PIPELINE_V3=0 to
  revert to legacy normalize-then-match order
- Test updated to assert default=True
- Full vault corpus diff script + result report archived
2026-07-04 20:04:39 +08:00
LLLin000
6f479ae46d test: widen v3 parity to six replay fixtures
- add fixture-backed replay parity for 37LK5T97, 8CCATQE3, and 5MAW65YD
- keep existing DWQQK2YB, VAMSAZMG, and PJBMGVTF gates
- update OCR project status and queue to 105-test state
2026-07-04 19:15:26 +08:00
LLLin000
ae3b5ce07c test: expand v3 replay parity corpus
- add fixture-backed parity replay for VAMSAZMG and PJBMGVTF
- keep DWQQK2YB as existing replay gate
- sync post_match render/index defaults to legacy contract
- update OCR status docs to 102-test state
2026-07-04 19:11:31 +08:00
LLLin000
2a4cda9782 test: add DWQQK2YB v3 parity gate
- sync render/index defaults in post_match_normalize to match legacy
- add fixture-backed real-paper parity test for DWQQK2YB
- update OCR project status and active queue to 100-test state
2026-07-04 18:33:32 +08:00
LLLin000
7856d6e202 fix: pre-merge blockers A-D
- A: page-qualified object writeback lookup (key by (page, block_id))
- B: contained figure text routed through ownership evidence
- C: v3 path rescue_roles_with_document_context equivalence
- D: regression tests for all 4 blockers
  - test_apply_object_writebacks_respects_page_for_duplicate_block_ids
  - test_contained_figure_text_stamps_ownership_evidence
  - test_post_match_normalize_runs_rescue_roles
  - test_v3_synthetic_parity_with_legacy_reference_boundaries
2026-07-04 16:40:53 +08:00
LLLin000
1a81817d7f test: add OCR pipeline v3 parity gate 2026-07-04 16:06:06 +08:00
LLLin000
cb685f73d0 C2: Implement post_match_normalize with shadow plus tail settlement, wire full v3 orchestration
- Rewrite ocr_post_match_normalize.py: shadow-normalize rows, commit final
  role/role_source/role_confidence/role_candidate/render_default/index_default
  from shadow, then run settle_tail_and_backmatter
- Wire v3 path in ocr.py postprocess_ocr_result:
  - seed_only normalize mode when OCR_PIPELINE_V3 is truthy
  - pre_match_normalize call (import inside if block)
  - post_match_normalize call (import inside if block)
  - Legacy path unchanged
- Add C2 test test_post_match_normalize_commits_shadow_role_back_to_public_role
2026-07-04 16:05:03 +08:00
LLLin000
d82949ee58 C1: shadow normalize in pre_match_normalize + _match_role helpers
- Rewrite pre_match_normalize to run a shadow normalize pass that
  populates role_candidate on each block while preserving public role
- Add _match_role(block) helper to ocr_figures.py (role_candidate > role > seed_role)
- Port matching-time role reads in figure builders to use _match_role
- Add _match_role(block) helper to ocr_tables.py (same priority)
- Port matching-time role reads in table builders to use _match_role
- Port role reads in FigureCorpus.from_blocks and TableCorpus.from_blocks
- Add 3 C1 tests to test_ocr_pipeline_v3.py
- All 325 OCR tests pass (6 C0+C1 + 4 rendering + 5 writeback + 310 figures)
2026-07-04 16:02:55 +08:00
LLLin000
c9d78fca44 C0: add OCR_PIPELINE_V3 toggle, seed-only build_structured_blocks, stub modules
- Add _ocr_pipeline_v3_enabled() toggle function in ocr.py
- Add normalize_mode='legacy'|'seed_only' param to build_structured_blocks()
- Add seed_only guard that skips legacy normalize pipeline
- Create ocr_pre_match_normalize.py stub
- Create ocr_post_match_normalize.py stub
- Create test_ocr_pipeline_v3.py with 3 tests
2026-07-04 15:54:53 +08:00