Commit graph

88 commits

Author SHA1 Message Date
Research Assistant
08ddebbc0b docs(state): mark Phase 5 complete 2026-04-23 20:11:20 +08:00
Research Assistant
e79ff8cbc6 docs(05-01): complete plan 05-01 with OCR state machine tests and command docs verification 2026-04-23 20:10:03 +08:00
Research Assistant
935d948ede feat(05-01): add OCR state machine tests covering job transitions
- Test pending->queued transition on job submission
- Test processing->done transition when polling returns success
- Test processing->error transition on API error response
- Test processing->blocked transition via HTTPError 401 (classify_error maps to blocked)
- Test sync_ocr_queue skips done/blocked items from existing queue
- Test cleanup_blocked_ocr_dirs removes empty dirs, preserves dirs with payload
- Test all 7 OCR states (pending, queued, running, done, error, blocked, nopdf) don't crash

Key: patch requests.post with side_effect=HTTPError 401 to trigger
classify_error path, since registry token is always present in test env.
2026-04-23 19:37:15 +08:00
Research Assistant
b6df4ae510 docs(05-02): complete 05-02 plan metadata
- 05-02-SUMMARY.md created with 13 smoke tests documented
- STATE.md: Phase 5 progress table updated with 05-01/05-02 done
- ROADMAP.md: Phase 5 plans checked off, phase status updated to COMPLETE
- REQUIREMENTS.md: REL-02 marked complete (05-02)
2026-04-23 18:48:45 +08:00
Research Assistant
405fc3f127 test(05-02): add end-to-end smoke test suite covering full pipeline
Smoke tests (13 total):
- Setup validation (run_doctor returns 0/1, output checks)
- Selection sync (returns 0, scans exports, creates records)
- Index refresh (returns 0, generates formal Literature notes)
- OCR doctor L1-L3 (returns int code, detects missing token)
- Deep-reading queue (returns 0, produces 3-state output)
- CLI main entry (doctor/status return int, not sys.exit)

All tests use tmp_path fixtures, no real vault, no live network calls.
2026-04-23 18:46:28 +08:00
Research Assistant
1704395d34 test(05-02): add fixture vault factory for smoke testing
- fixture_vault: creates complete vault structure with paperforge.json
- fixture_library_records: 3 minimal library-record .md files
- fixture_bbt_json: Better BibTeX JSON export in exports/
- fixture_with_pdf: vault with real temp PDF for PDF-resolver tests
- All use pathlib.Path, Windows-compatible
2026-04-23 18:46:13 +08:00
Research Assistant
dc11e10c37 docs(state): record phase 5 context session 2026-04-23 18:26:49 +08:00
Research Assistant
f8b523baf9 docs(05): capture phase context 2026-04-23 18:26:18 +08:00
Research Assistant
6bbdb3d1d1 docs(02-03): complete OCR Doctor plan with summary, state, and roadmap updates
- Add 02-03-SUMMARY.md with execution summary and deviations
- Update STATE.md: mark 02-03 done, add decisions, update next action
- Update ROADMAP.md: mark 02-03 and 01-04 complete
2026-04-23 13:02:18 +08:00
Research Assistant
2a3295ed38 feat(02-03): add ocr doctor CLI dispatch, tests, and docs
- Convert ocr parser to sub-subcommands (run, doctor)
- Add _cmd_ocr_doctor() with formatted tiered report output
- Update dispatch logic to route ocr doctor subcommand
- Add CLI dispatch test for doctor command
- Update command/lp-ocr.md with doctor documentation
2026-04-23 12:59:04 +08:00
Research Assistant
3bd1c5ea69 feat(02-03): implement ocr_doctor() with L1-L4 diagnostics
- Add paperforge_lite/ocr_diagnostics.py with tiered L1-L4 checks
- L1: token presence, L2: URL reachability, L3: API schema validation
- L4: optional live PDF round-trip test with polling
- Add 7 mocked unit tests for all levels
- Include blank.pdf test fixture
2026-04-23 12:58:54 +08:00
Research Assistant
24e3d9fb30 test(02-01): add failing tests for pdf_resolver path resolution
- 8 test cases covering absolute, vault-relative, junction, storage-relative
- Missing file, has_pdf=False, empty path, whitespace path scenarios
- is_valid_pdf and resolve_junction helper coverage
2026-04-23 12:54:38 +08:00
Research Assistant
97a8fd6772 docs(01-03): complete plan 01-03 2026-04-23 12:07:00 +08:00
Research Assistant
500d268db7 feat(01-03): wire validate_setup.py to shared resolver
- load_config now tries paperforge_lite.config.load_vault_config first,
  falls back to legacy JSON parsing for pre-01-03 installs
- resolve_vault_for_validate checks PAPERFORGE_VAULT before VAULT_PATH
  before cwd (matching the shared resolver's precedence)
- Preserves all existing validation checks
2026-04-23 12:03:08 +08:00
Research Assistant
e0acb64e8d feat(01-03): deploy paperforge_lite package through setup wizard
- setup_wizard.py now copies the paperforge_lite/ package alongside
  the worker and ld_deep scripts during deployment
- Two copy targets: <pf_path>/worker/paperforge_lite/ for
  literature_pipeline.py and <skill_dir>/literature-qa/paperforge_lite/
  for ld_deep.py
- Existing copy targets for literature_pipeline.py and ld_deep.py
  remain unchanged
- Warning emitted if paperforge_lite source not found
2026-04-23 12:01:36 +08:00
Research Assistant
1c1e6d7c0d feat(01-03): replace duplicated resolver logic with shared wrappers
- literature_pipeline: load_vault_config and pipeline_paths now
  delegate to paperforge_lite.config, preserving public names
- ld_deep: _load_vault_config and _paperforge_paths now delegate
  to shared resolver, preserving public names
- Both scripts require paperforge_lite package to be available
  (via pip install or copied install in deployed vaults)
- Existing load_simple_env preserved for .env loading before dispatch
2026-04-23 12:00:24 +08:00
Research Assistant
fec25c2d3e feat(01-03): wire worker and /LD-deep to shared resolver
- Replace literature_pipeline.load_vault_config body with
  paperforge_lite.config.load_vault_config wrapper
- Rebuild pipeline_paths() from shared paperforge_paths() plus
  10 worker-only keys (pipeline, candidates, search_*, harvest_root,
  records, review, config, queue, log, bridge_config*, index, ocr_queue)
- Replace ld_deep._load_vault_config and _paperforge_paths with
  shared resolver wrappers returning ocr, records, literature
- Fix subprocess test to set PYTHONPATH so paperforge_lite is
  importable when worker runs as a standalone script
- 34 tests pass (22 config + 8 worker compat + 4 ld_deep compat)
2026-04-23 12:00:11 +08:00
Research Assistant
618227eb9b test(01-03): add failing tests for worker and /LD-deep resolver integration
- Test literature_pipeline.load_vault_config matches paperforge_lite.config
- Test PAPERFORGE_SYSTEM_DIR env var override in worker
- Test pipeline_paths returns all 17 expected keys (shared + worker-only)
- Test ld_deep._load_vault_config and _paperforge_paths match shared resolver
- Test direct worker status subprocess smoke test (CMD-02)
- Tests fail against current code: worker has 5 config keys vs shared's 7,
  env overrides not implemented, and pipeline_paths lacks shared resolver keys
2026-04-23 11:55:24 +08:00
Research Assistant
767548d2fe docs(01-02): complete 01-02 plan
- paperforge CLI launcher with argparse subcommands
- pyproject.toml packaging entry point
- load_simple_env added to config.py
- 9 CLI tests (3 path + 6 dispatch) passing
- 31 total tests passing
- CONF-02, CMD-01, CMD-03 requirements completed
2026-04-23 11:48:26 +08:00
Research Assistant
9496382e81 feat(01-02): implement CLI launcher, packaging entry point, and shared config resolver
- paperforge_lite/cli.py: argparse CLI with global --vault, subcommands paths/status/selection-sync/index-refresh/deep-reading/ocr (ocr defaults to run action)
- paperforge_lite/__main__.py: python -m paperforge_lite fallback entry point
- paperforge_lite/config.py: shared resolver (load_vault_config, resolve_vault, paperforge_paths, paths_as_strings, load_simple_env) matching Plan 01 contract
- pyproject.toml: setuptools build, paperforge-lite package, Python >=3.10, dependencies from requirements.txt, [project.scripts] paperforge=paperforge_lite.cli:main
- paperforge_lite/__init__.py: package init (version 1.0.0)

CLI loads .env from vault root and <system_dir>/PaperForge/.env before worker dispatch, matching legacy pipeline behavior.
2026-04-23 11:44:26 +08:00
Research Assistant
216ea4e8ac test(01-02): add failing test for CLI paths output and worker dispatch
- Tests main() with --vault vault paths --json emits valid JSON with vault, worker_script, ld_deep_script keys
- Tests main() with --vault vault paths emits text paths without unresolved <system_dir> or <resources_dir> tokens
- Tests ocr and ocr run both dispatch to run_ocr
- Tests status/selection-sync/index-refresh/deep-reading each dispatch to correct worker
- Uses monkeypatch to isolate from real workers and filesystem
2026-04-23 11:44:09 +08:00
Research Assistant
6dc62ac9f1 docs(01-01): complete 01-01 plan
- SUMMARY: 22 passing tests, 3 files created, 2 task commits (test + feat)
- STATE: phase 1 progress updated, decisions logged
- ROADMAP: 01-01 marked complete
2026-04-23 11:43:37 +08:00
Research Assistant
034542bcd2 docs(01-04): complete plan 04 - stable command documentation
- SUMMARY.md: phase 1 plan 4 complete, all tasks committed
- STATE.md: updated with Phase 1 progress (4/4 plans complete)
- ROADMAP.md: marked all 4 Phase 1 plans as done

Commits:
- baa2d77: test(01-04): add failing tests for stable command docs
- 51d35d6: feat(01-04): replace unresolved path-token commands with stable paperforge commands
2026-04-23 11:42:23 +08:00
Research Assistant
a6c0cb0e70 feat(01-01): implement shared config resolver
- DEFAULT_CONFIG: 99_System, 03_Resources, Literature, LiteratureControl, 05_Bases, .opencode/skills, .opencode/command
- ENV_KEYS: all 8 PAPERFORGE_* overrides mapped
- read_paperforge_json: supports both nested vault_config and top-level legacy keys
- resolve_vault: cli > PAPERFORGE_VAULT > json-search > cwd precedence
- load_vault_config: layered merge with locked D-Configuration Hierarchy order
- paperforge_paths: returns exactly 13 required user-facing path keys
- paths_as_strings: JSON-serializable string output
- No os.environ mutation, no OCR secrets, stdlib only
2026-04-23 11:41:10 +08:00
Research Assistant
51d35d6999 feat(01-04): replace unresolved path-token commands with stable paperforge commands
- command/lp-*.md: replace python <system_dir>/... with paperforge status|selection-sync|index-refresh|ocr run; add fallback legacy command section
- command/ld-deep.md: update queue preflight to 'paperforge deep-reading'; add paperforge paths --json for variable resolution
- README.md: update core commands section to show paperforge first, legacy as fallback
- docs/INSTALLATION.md: add paperforge paths/status as primary verification
- docs/setup-guide.md: replace python pipeline/worker/... with paperforge selection-sync/index-refresh
- setup_wizard.py DoneStep: add pip install -e . and paperforge commands as primary next steps

All 15 tests in tests/test_command_docs.py pass.
2026-04-23 11:40:24 +08:00
Research Assistant
baa2d77bbd test(01-04): add failing tests for stable command docs
- Verify lp-* docs contain paperforge status|selection-sync|index-refresh|ocr run
- Verify ld-deep.md mentions paperforge deep-reading and paths --json
- Verify no legacy python <system_dir>/.../literature_pipeline.py in user examples
- Scope exclusion: AGENTS.md frontmatter examples, architecture diagrams

RED phase: all 15 assertions fail before implementation.
2026-04-23 11:37:40 +08:00
Research Assistant
6b0b16b1dd test(01-01): add failing test for config resolver contract
- Tests DEFAULT_CONFIG defaults (99_System, 03_Resources, Literature, etc.)
- Tests ENV_KEYS has all required PAPERFORGE_* overrides
- Tests load_vault_config precedence: overrides > env > JSON > defaults
- Tests nested vault_config honored per CONF-04
- Tests top-level keys override nested for backward compat per CONF-04
- Tests paperforge_paths returns exactly required user-facing keys
- Tests paths are absolute Path objects
- Tests worker_script and ld_deep_script keys present
- Tests paths_as_strings returns JSON-serializable dict[str,str]
- Tests resolve_vault precedence: cli > env > json search > cwd

RED phase: module not yet implemented, all tests fail as expected.
2026-04-23 11:35:52 +08:00
Research Assistant
8097ec6ff3 docs(01): create phase plan 2026-04-23 11:18:05 +08:00
Research Assistant
881457dd92 Fix configurable Lite setup paths 2026-04-23 01:44:13 +08:00
Research Assistant
e0881f7d2c restore ASCII logo in WelcomeStep 2026-04-23 01:17:46 +08:00
Research Assistant
6fcd1132fa feat: read system_dir/resources_dir from paperforge.json instead of hardcoding 99_System/03_Resources 2026-04-23 01:07:50 +08:00
Research Assistant
7517ead1a6 fix: deploy scripts and .env to 99_System/PaperForge/ instead of vault root 2026-04-23 01:05:44 +08:00
Research Assistant
fd7ccb53fc fix: unify PaddleOCR env var names to match pipeline expectations (API_TOKEN/JOB_URL) 2026-04-23 00:57:48 +08:00
Research Assistant
270e1b8dc6 simplify: only check zotero.sqlite and storage exist; remove unnecessary path validation 2026-04-23 00:43:13 +08:00
Research Assistant
b6e59043b2 fix: skip junction creation if Zotero data dir is inside vault 2026-04-23 00:41:59 +08:00
Research Assistant
a4ede4303a fix: add Zotero path validation to prevent vault path confusion; improve instructions 2026-04-23 00:40:08 +08:00
Research Assistant
e7312c1526 fix: support dict-format Better BibTeX JSON exports 2026-04-23 00:32:38 +08:00
Research Assistant
49a817e5c2 fix: add textual to requirements.txt 2026-04-23 00:09:00 +08:00
Research Assistant
5605befcf0 feat: add PaddleOCR API Key input in DeployStep 2026-04-22 23:38:33 +08:00
Research Assistant
71d69b7d89 docs: use descriptive names instead of hardcoded numbers in directory tree 2026-04-22 23:37:00 +08:00
Research Assistant
d72bf76f08 docs: update README and INSTALLATION for wizard-based setup 2026-04-22 23:36:26 +08:00
Research Assistant
47b5661c2d feat: add pre-deploy safety check - block if any prior step incomplete 2026-04-22 23:25:26 +08:00
Research Assistant
27134ba569 fix: Better BibTeX format and emphasize keep-updated requirement 2026-04-22 23:24:31 +08:00
Research Assistant
89374cb35f remove: all 科研读图指南 and Template references from project 2026-04-22 23:20:49 +08:00
Research Assistant
29949395ef prefill Zotero path with actual username and home directory 2026-04-22 23:09:01 +08:00
Research Assistant
6f1f451893 fix: hide absolute paths in wizard; auto-detect Zotero data directory 2026-04-22 23:06:55 +08:00
Research Assistant
7764b86b2e feat: add vault path input with validation; improve directory tree display; hide absolute paths in ZoteroStep 2026-04-22 22:29:36 +08:00
Research Assistant
da80b8c781 fix: remove duplicate lines causing IndentationError 2026-04-22 22:13:59 +08:00
Research Assistant
c607dd0441 rename: LiteraturePipeline -> PaperForge in all paths and references 2026-04-22 22:10:16 +08:00
Research Assistant
19292e1eee refactor: simplify VaultStep (custom dir names) and ZoteroStep (manual path input) 2026-04-22 21:59:32 +08:00