lllin000_PaperForge/paperforge
LLLin000 db2988d8f0 PR9C: Streaming embed pipeline with continuous progress
Replace two-phase (prepare all, then encode+write) with sliding-window
pipeline. Maintains bounded in-flight window (max_workers * 4 = 16),
emits EMBED_PROGRESS from the first paper completion (~2s vs ~30s).

Key changes:
- producer/consumer loop: prepare + submit one paper at a time
- wait(FIRST_COMPLETED) in main thread for encode results
- processed_count = skip + embedded, monotonic EMBED_PROGRESS
- resume skip and no-payload paths also advance processed_count
- encode failure fails closed (return 1, no silent skip)
- removed PR9B_BATCH_SIZE and _batched() (replaced by sliding window)
- 4 integration tests for progress counting, failure, and write cycle
2026-07-07 02:04:12 +08:00
..
adapters feat: add ocr_redo frontmatter field + sync preservation 2026-06-01 11:23:32 +08:00
command_files feat(skills): add /pf-log-reading and /pf-log-session slash commands 2026-05-12 20:55:32 +08:00
commands PR9C: Streaming embed pipeline with continuous progress 2026-07-07 02:04:12 +08:00
core feat: PR 1 — nested structure tree + body_units correctness 2026-07-06 16:42:08 +08:00
doctor chore: commit session work before hotfix switch 2026-05-18 22:47:30 +08:00
embedding fix: OpenAICompatibleProvider uses requests instead of openai client 2026-07-06 23:15:58 +08:00
memory feat: PR 8 — Object Vector Retrieval (paperforge_objects collection) 2026-07-06 19:58:46 +08:00
plugin feat: surface drift state and invalidate maintenance cache 2026-07-05 16:52:17 +08:00
policies feat(ocr-quality): add evaluate_readiness() with YAML policy evaluator 2026-07-04 23:28:06 +08:00
retrieval fix: PR8.1 — status healthy init + object_units rebuild + version bump 2026-07-06 20:08:51 +08:00
schema feat: P0 fixes (ref sort, caption insert) + P1 figure containment render hygiene 2026-06-27 22:56:22 +08:00
services feat: P0 fixes (ref sort, caption insert) + P1 figure containment render hygiene 2026-06-27 22:56:22 +08:00
setup fix: remove v prefix from git tag references in install URLs; update versions.json 2026-05-25 11:45:11 +08:00
skills feat: classify OCR errors and degraded completions 2026-06-08 12:34:44 +08:00
worker PR9A: Resume & Rebuild Correctness 2026-07-06 22:33:08 +08:00
__init__.py bump: 1.5.14 -> 1.5.15 2026-06-01 19:46:46 +08:00
__main__.py fix: force UTF-8 encoding for all subprocess and stdout on Windows, fix pre-existing logger bug 2026-05-11 19:16:18 +08:00
cli.py feat: rebuild speed-up with parallel rebuild, threaded asset cropping, maintenance refactor, embed progress, CLI docs 2026-07-05 19:43:07 +08:00
config.py feat: runtime contract hardening + skill/command truth alignment (Package A+B) 2026-05-16 22:38:43 +08:00
logging_config.py chore: rename PaperForge Lite -> PaperForge across 27 files 2026-04-28 01:25:51 +08:00
ocr_diagnostics.py feat: bump PaddleOCR model from VL-1.5 to VL-1.6 2026-06-01 00:32:46 +08:00
pdf_resolver.py fix: add creators/extra to load_export_rows row dict, fix journal key lookup for first_author/journal/IF in library-records; add first_author, journal, impact_factor columns to Base views; fix PDF storage/ subdirectory resolution 2026-04-27 20:25:35 +08:00
query_planning.py style: fix lint issues in new Phase4.1/Phase5 code 2026-06-05 12:23:55 +08:00
setup_wizard.py feat: add Gemini CLI agent platform support (.gemini/skills) 2026-06-01 19:46:17 +08:00