#27: write_encoded_payload and delete_paper_vectors use sqlite-vec
- builder.py: writes to vec0 tables + companion meta tables
- _chroma.py: delete_paper_vectors deletes from vec0 + meta by paper_id
- Stores body_units_hash, object_units_hash, retrieval_policy_version in meta tables
#28: merge_retrieve and retrieve_chunks use vec0 k-NN search
- search.py: vec0 k-NN queries with companion meta JOINs
- Same dedup/per-paper-cap logic, same result shapes
#29: build_state migrated from JSON file to SQLite build_state table
#30: E2E embed+retrieve test with sqlite-vec + FixedProvider
- 3 tests: body roundtrip, source correctness, per-paper cap
#32: E2E OCR pipeline test with fixture PDFs
Additional changes:
- schema.py: bump to v6, add hash/policy columns to vec companion meta tables
- embed.py: _assert_collections_healthy uses sqlite-vec; resume hash checks read from meta tables
- status.py: chunk counts from SQL COUNT on companion tables
- Removed unused get_collection/embed_paper imports from embed.py
- All tests updated: 76 pass (was 72, +4 new E2E tests, +4 for migration)
- FixedFixedProvider to generate 1536-dim vectors (matching vec0 schema)
- Fixed health check tests for sqlite-vec path
Replace two-phase (prepare all, then encode+write) with sliding-window
pipeline. Maintains bounded in-flight window (max_workers * 4 = 16),
emits EMBED_PROGRESS from the first paper completion (~2s vs ~30s).
Key changes:
- producer/consumer loop: prepare + submit one paper at a time
- wait(FIRST_COMPLETED) in main thread for encode results
- processed_count = skip + embedded, monotonic EMBED_PROGRESS
- resume skip and no-payload paths also advance processed_count
- encode failure fails closed (return 1, no silent skip)
- removed PR9B_BATCH_SIZE and _batched() (replaced by sliding window)
- 4 integration tests for progress counting, failure, and write cycle
