1. Stale test: test_extract_objects_renders_same_page_once_for_multiple_crops
now asserts render_pdf_page_cached is NOT called (new PageRenderContext path)
and pages/page_001.jpg is NOT created.
2. Extract _resolve_object_crop_pdf_path() helper with 4-case test.
Phase 4 now uses it instead of inline fallback logic.
Test locks in: Phase 1 resolved path beats stale meta['source_pdf'].
3. PageRenderContext: use fitz.csRGB in get_pixmap() for safe CMYK/n>3 handling.
Removed fragile pix.n mode detection.
4. Added test_extract_and_write_objects_with_use_disk_page_cache_false_and_valid_pdf
6 fixes bundled:
1. Cleanup: remove accidental empty file '6s}'
2. Rebuild determinism: add 'use_disk_page_cache' gate to _crop_asset_from_pdf.
Rebuild (use_disk_page_cache=False) never reads or writes pages/page_XXX.jpg.
All rendering goes through PageRenderContext in-memory.
Legacy callers keep backward-compat behavior (default True).
3. PageRenderContext safety:
- Only use when page_width>0 and page_height>0 and not rotation_deg
- Fix Pixmap.n mode detection (L/RGB/RGBA -> convert to RGB)
- Fix Image.Resampling name bug (Image -> PILImage)
- Rotated crops always fall back to direct PDF clip path
4. Phase 1+2a merged traversal: always extract PDF lines on every page,
regardless of whether span_metadata already exists. Fixes figure inventory
data gaps when pages have pre-existing span coverage.
5. Phase 4: use resolved source_pdf_path from Phase 1, not ocr_meta['source_pdf']
fallback (which may be stale or missing).
6. Added 5 determinism tests
Add PageRenderContext class that renders each page once into memory
(PIL Image via get_pixmap + frombytes), shared across all crop tasks
in a single rebuild session. No full-page JPG is written to disk.
- _crop_asset_from_pdf: new fast path (page_render_context kwarg)
- Three task functions pass context through
- extract_and_write_objects opens PDF once, creates context, closes after
- Cold Phase 4a: 4.0s → 1.24s (60p, 30 unique pages, 42 crops)
- Full cold rebuild: 10.9s → 7.17s
- Extract PDF lines from rawdict data that was already fetched for span backfill,
eliminating the separate page.get_text('dict') pass (~6s for 60 pages)
- Add _extract_lines_from_rawdict() helper to avoid rawdict re-parse
- Keep extract_pdf_lines_normalized() as backward-compatible API
- First-rebuild: 14.2s -> 10.9s (60p), 8.8s -> 8.0s (81p)
- Group raw_blocks by page, call get_text('rawdict') once per page instead of once per block
(1433 calls → 60 calls, 68s → 16.6s for a 60-page paper)
- Add _spans_from_rawdict() helper to filter cached rawdict by char bbox center overlap
- Keep extract_pdf_spans_for_block() as single-block compatibility API unchanged
- Add incremental backfill: skip pages where all blocks already have span_metadata
- Remove figure_title from _is_text_like_raw_block() coverage count (figure titles are
typically image-rendered, not selectable PDF text)
- Add regression test: fast path output matches single-block API
Fix 1 (units.py): object unit_id now includes the entry's block_id
instead of the node's block_span, fixing PRIMARY KEY collisions when
a section has multiple objects (e.g. two figure_captions).
Fix 2 (builder.py): retrieval units are now persisted during
build_from_index(). Scans each paper's OCR output directory for
structure-tree.json, builds body+object units and manifest, then
upserts into the DB tables and meta. Also clears body_units and
object_units at rebuild start alongside the other tables.
1. show_in_base — separate hidden from maintenance vs hidden from both
2. Unified is_degraded check — done_degraded and health yellow/red
share one entry point
3. Hash error_summary in manifest instead of raw string
4. Fix Promise.withResolvers for ES compat
1. compute_use_cases output uses status/gates/reasons (not recommended/gate_results)
2. write_feedback validates every mark has result_hash and fulltext_hash
3. load_readiness_policy(policy=...) bypasses user override (reproducible)
4. confidence_and_fallbacks yellow for degraded=True or weak span coverage
Minor: append_mark doesn't mutate caller dict; has_figure_evidence includes
unmatched_legend_count and held_count.
Creates new ocr_quality.py module as pure function layer between
build_ocr_health() and evaluate_readiness(). All 5 indicators:
- rendered_text_integrity
- body_reference_structure
- figure_table_integrity
- metadata_frontmatter_quality
- confidence_and_fallbacks
10 unit tests covering shape, thresholds, applicability,
health_profile spelling, inventory precedence, and run_integrity.
