lllin000_PaperForge/paperforge/embedding
LLLin000 6ff0f6d38b PR9B: Embed Parallel Encode
- Four new dataclasses: EmbeddingPayload, EncodedPayload, PaperEmbeddingJob, PaperEncodedBundle
- prepare_legacy/body/object_payload: extract prepare phase from legacy embed functions
- encode_payload / encode_paper_job: worker-thread-safe encode functions
- write_encoded_payload: ChromaDB serial write
- prepare_payloads_for_entry: routing to correct prepare function
- Existing embed_body_units/embed_paper/embed_object_units refactored as wrappers
- Embed main loop: three-phase pipeline (PREPARE -> ENCODE -> WRITE) with ThreadPoolExecutor
- Bounded batch processing (BATCH_SIZE=8, MAX_WORKERS=4)
- Encode-success-gated delete: old vectors only removed after encode succeeds
- 23 regression tests for new functions
2026-07-06 22:41:25 +08:00
..
backends feat: add LanceDB evaluation backend scaffold 2026-07-05 14:19:37 +08:00
providers feat(embedding): add OpenAI-compatible embedding provider 2026-05-18 17:14:18 +08:00
__init__.py feat: PR 8 — Object Vector Retrieval (paperforge_objects collection) 2026-07-06 19:58:46 +08:00
_chroma.py feat: PR 8 — Object Vector Retrieval (paperforge_objects collection) 2026-07-06 19:58:46 +08:00
_config.py chore: commit session work before hotfix switch 2026-05-18 22:47:30 +08:00
build_state.py feat(embedding): add build state persistence 2026-05-18 17:15:40 +08:00
builder.py PR9B: Embed Parallel Encode 2026-07-06 22:41:25 +08:00
preflight.py feat(embedding): add API-only preflight check 2026-05-18 17:18:04 +08:00
search.py feat: PR 8 — Object Vector Retrieval (paperforge_objects collection) 2026-07-06 19:58:46 +08:00
status.py fix: PR8.1 — status healthy init + object_units rebuild + version bump 2026-07-06 20:08:51 +08:00