Fix 1 (units.py): object unit_id now includes the entry's block_id
instead of the node's block_span, fixing PRIMARY KEY collisions when
a section has multiple objects (e.g. two figure_captions).
Fix 2 (builder.py): retrieval units are now persisted during
build_from_index(). Scans each paper's OCR output directory for
structure-tree.json, builds body+object units and manifest, then
upserts into the DB tables and meta. Also clears body_units and
object_units at rebuild start alongside the other tables.
1. show_in_base — separate hidden from maintenance vs hidden from both
2. Unified is_degraded check — done_degraded and health yellow/red
share one entry point
3. Hash error_summary in manifest instead of raw string
4. Fix Promise.withResolvers for ES compat
1. compute_use_cases output uses status/gates/reasons (not recommended/gate_results)
2. write_feedback validates every mark has result_hash and fulltext_hash
3. load_readiness_policy(policy=...) bypasses user override (reproducible)
4. confidence_and_fallbacks yellow for degraded=True or weak span coverage
Minor: append_mark doesn't mutate caller dict; has_figure_evidence includes
unmatched_legend_count and held_count.
Creates new ocr_quality.py module as pure function layer between
build_ocr_health() and evaluate_readiness(). All 5 indicators:
- rendered_text_integrity
- body_reference_structure
- figure_table_integrity
- metadata_frontmatter_quality
- confidence_and_fallbacks
10 unit tests covering shape, thresholds, applicability,
health_profile spelling, inventory precedence, and run_integrity.
8 spec corrections from review:
1. PR 1: field-based return, not lines; insertion before title
2. PR 2: explicit body_paragraph escape in assign_block_role
3. PR 2: caption syntax supports delimiter + title patterns
4. PR 3: use existing ref_heading_block, no new scan
5. PR 3: same_page_tail_blocks + column check, test body rendering
6. PR 4: split None into full_width vs ambiguous_center
7. PR 4: column check before safe_auto_match, no sidecar exception
8. PR 2: fixture verification criteria per-fixture, not blanket
Based on layout truth audit findings (6 bug patterns), designs 4
independent PRs with precise code locations, helper contracts,
column-aware zone inference, caption formalness tightening,
and render frontmatter fallback. Includes before/after Mermaid
diagrams, test specs, and execution order.
Workstream X: 11 papers audited across 6 layout classes,
5 verified via vision subagents. 6 real bug patterns identified
with code locations and root cause analysis.
Bug patterns:
A - _is_obviously_formal_figure_caption heuristic too aggressive
(ocr_roles.py:193)
B - Cross-column figure asset mis-assignment ignores boundaries
(ocr_figures.py:704)
C - Render frontmatter skip silently drops authors/affiliations
(ocr_render.py)
D - Two-column same-page boundary body→backmatter zone pollution
(ocr_document.py)
E - Frontmatter headings misclassified (ocr_roles.py)
F - Supplementary-only PDFs not handled
Full report: docs/superpowers/analysis/2026-07-05-layout-truth-audit-findings.md
Also: codebase-memory-mcp 0.8.1 installed and repo indexed.
The V3 pipeline has been tested against the full 555-paper vault vs legacy:
No diff: 547 (98.6%)
Diff: 5 (0.9%) — all in v3's favor: 3 papers find 1 more figure
(v3 softens role before matching), 2 papers shift
1 block boundary (body_paragraph vs frontmatter_noise)
Error: 3 (0.5%) — 2 missing raw data, 1 pre-existing v3 pipeline edge case
The 5 diffs are marginal improvements, not regressions. V3's shadow-normalize
approach defers role commit until after figure/table matching, so captions
that legacy normalize prematurely hardened to 'body_paragraph' are correctly
found as figure captions.
Changes:
- _ocr_pipeline_v3_enabled() defaults to True now; set OCR_PIPELINE_V3=0 to
revert to legacy normalize-then-match order
- Test updated to assert default=True
- Full vault corpus diff script + result report archived
figure_inner_text role means the text IS part of the figure content
(forest plot y-axis labels, nomogram variable names, panel markers).
The earlier fix listed the text as a separate below the image — that
was wrong: the text already lives inside the cropped image when the
crop bbox covers it.
Now extract_and_write_objects expands the crop bbox to include all
figure_inner_text blocks owned by the figure (via _object_owner_id,
same page). The cropped jpg contains the labels; render only emits
image + Legend.
Verified: M84CTEM9 figure_a001 (forest plot) now crops to 787x363
(previously 444x358 — missed the y-axis labels); figure_a002
(nomogram) still includes its variable labels. 105 focused tests pass.
