- #13: Critical — swap delete/write order in _complete_one (write new
vectors before deleting old ones) to prevent data loss on write failure
- #12: Add logger.warning() to 3 resume-skip except blocks so silent
re-embed fallbacks are observable
- #14: Add lightweight HNSW query probe to _assert_collections_healthy
- #11: Log collection query failures in merge_retrieve instead of silent
continue
- #16: Reuse provider across payloads in encode_paper_job — one creation
per worker per paper instead of per payload
P0-1: FTS per-paper insert — move SELECT INTO body_units_fts inside the
per-paper loop with WHERE paper_id = ? to avoid re-inserting other papers'
rows during full rebuild. Fix sqlite3.DatabaseError on empty FTS table.
P0-2: object_units real role_index keys — read from 'captions'/'tables'
(what build_role_indexes actually outputs) with fallback to old key names.
P1-1: unit_id collision for mixed body/backmatter — include unit_kind
suffix (:backmatter_body) in unit_id. Also fix duplicate tree node_ids
(block_id reused on different pages) by appending order{emitted_order}.
P1-2: Embed resume body_units_hash — add compute_body_units_hash() to
manifest.py, write hash + retrieval_policy_version to Chroma metadata,
resume compares both before skipping.
- get_collection(name=) supports paperforge_fulltext or paperforge_body
- delete_paper_vectors deletes from both collections
- embed_body_units + get_body_units_for_embedding (reads from memory DB)
- Status reports chunk_count, body_chunk_count, total_chunks
- Embed build routes to body_units path when DB has body_units
- merge_retrieve queries both collections with unit-level dedup
