mirror of
https://github.com/lllin000/PaperForge.git
synced 2026-07-22 17:00:23 +00:00
- Four new dataclasses: EmbeddingPayload, EncodedPayload, PaperEmbeddingJob, PaperEncodedBundle - prepare_legacy/body/object_payload: extract prepare phase from legacy embed functions - encode_payload / encode_paper_job: worker-thread-safe encode functions - write_encoded_payload: ChromaDB serial write - prepare_payloads_for_entry: routing to correct prepare function - Existing embed_body_units/embed_paper/embed_object_units refactored as wrappers - Embed main loop: three-phase pipeline (PREPARE -> ENCODE -> WRITE) with ThreadPoolExecutor - Bounded batch processing (BATCH_SIZE=8, MAX_WORKERS=4) - Encode-success-gated delete: old vectors only removed after encode succeeds - 23 regression tests for new functions |
||
|---|---|---|
| .. | ||
| backends | ||
| providers | ||
| __init__.py | ||
| _chroma.py | ||
| _config.py | ||
| build_state.py | ||
| builder.py | ||
| preflight.py | ||
| search.py | ||
| status.py | ||