Commit graph

46 commits

Author SHA1 Message Date
Research Assistant
f617054d5a feat: paperforge status --json, plugin panel with OCR progress bar 2026-04-28 19:21:52 +08:00
Research Assistant
248a0b6446 fix: restructure skills — separate pf-deep/pf-paper, add prefix substitution, create config_file on deploy 2026-04-28 18:33:11 +08:00
Research Assistant
0c90e40636 test: add integration smoke tests for multi-agent deploy 2026-04-28 16:27:31 +08:00
Research Assistant
aac31c4a76 test: add AGENT_CONFIGS format/prefix field tests, update expected agents 2026-04-28 16:13:58 +08:00
Research Assistant
ebc4fdeb00 chore: rename PaperForge Lite -> PaperForge across 27 files 2026-04-28 01:25:51 +08:00
Research Assistant
61df09c77f feat: add postprocess-pass2 molecule for validating Pass 2 output 2026-04-28 00:24:31 +08:00
Research Assistant
cc1f696ded feat: add fixed sub-headings to figure/table callout blocks in skeleton 2026-04-28 00:15:48 +08:00
Research Assistant
d82fc52521 feat: persistent OCR poll until completion; fix OpenCode commands for pip install; fix test hangs; add command_files to package 2026-04-27 22:52:03 +08:00
Research Assistant
94f00e990a fix: sync paperforge.json version to 1.4.0; fix update.py missing GITHUB_REPO/GITHUB_ZIP imports; remove PaperForge.base generation (duplicates Literature Hub) 2026-04-27 21:25:00 +08:00
Research Assistant
2bfbec691e fix: add zotero_dir to paperforge_paths() for PDF resolution; fix L3 diagnostic to skip schema check without file upload 2026-04-27 21:12:29 +08:00
Research Assistant
ee260e05f5 fix: L2 diagnostic accepts 400 as reachable (PaddleOCR rejects bare POST without fileUrl) 2026-04-27 20:41:49 +08:00
Research Assistant
13b3e142cc fix: OCR diagnostic L2 probe uses POST instead of GET (PaddleOCR rejects GET) 2026-04-27 20:34:47 +08:00
Research Assistant
a3270dc791 fix: add creators/extra to load_export_rows row dict, fix journal key lookup for first_author/journal/IF in library-records; add first_author, journal, impact_factor columns to Base views; fix PDF storage/ subdirectory resolution 2026-04-27 20:25:35 +08:00
Research Assistant
67a52a898a fix: VaultStep.__init__ missing step_id and checker in super() call 2026-04-27 19:49:36 +08:00
Research Assistant
2edb5b65cc docs(v1.4): commit planning artifacts, test files, and phase summaries for phases 13-19 2026-04-27 19:12:03 +08:00
Research Assistant
7cccf4eefc refactor(17-dead-code-precommit): remove delegation wrappers and dead code, run ruff cleanup
- Remove def load_vault_config delegation wrappers from all 7 worker modules
- Add from paperforge.config import load_vault_config, paperforge_paths at module level
- Remove unused system_dir/resources_dir/control_dir extraction in pipeline_paths
- Fix sync.py intra-function imports (run_selection_sync, run_index_refresh)
- Fix repair.py import path for _resolve_formal_note_path (direct from _utils)
- Add ruff and pre-commit as test dependencies in pyproject.toml
- Configure per-file ruff ignores for pre-existing issues
- Run ruff check --fix + ruff format (353 + 58 issues auto-fixed)
- All 203 tests pass, ruff check zero warnings
2026-04-27 17:40:47 +08:00
Research Assistant
0beb575c30 fix(13-logging-foundation): update test stubs and capsys→caplog for logging migration
- Add verbose=False to all stub function signatures in cli dispatch tests
- Switch test_verbose_output_printed from capsys to caplog since repair
  diagnostics now route through logging (stderr) not print (stdout)
2026-04-27 15:21:10 +08:00
Research Assistant
4d57d17975 fix(v1.3-doctor): update worker_script path and doctor checks for v1.3 architecture
- config.py: worker_script now points to paperforge/worker/__init__.py
  (pipeline/ removed in Phase 12)
- worker/status.py: doctor checks paperforge.worker importability
  instead of old literature_pipeline.py existence
- tests/test_config.py: update assertion for new worker_script name
2026-04-25 00:02:53 +08:00
Research Assistant
f54fe8a3e0 test(phase-12): update test imports for new worker module structure
- Update all pipeline.worker.scripts patches to paperforge.worker.*
- Fix test_base_views.py and test_base_preservation.py imports
- Update test_legacy_worker_compat.py for new module paths
- Fix OCR test patches to target correct module namespaces
2026-04-24 22:36:54 +08:00
Research Assistant
4297dcd39b fix(test): update absolute path test for Phase 11 normalization
Absolute paths now normalized with bsolute: prefix per D-01.
Test updated to expect bsolute:C:\... format.
2026-04-24 20:55:49 +08:00
Research Assistant
72cbdc3250 test(11-01): add test_path_normalization.py with 25 test methods
- TestBBTPathNormalization: 8 tests covering absolute Windows, storage: prefix,
  bare relative, Chinese chars, spaces, backslash normalization, empty path,
  absolute non-storage
- TestMainPdfIdentification: 6 tests covering title=PDF primary, largest file
  fallback, first PDF fallback, no PDFs, single PDF, mixed PDF/non-PDF
- TestWikilinkGeneration: 6 tests covering basic wikilink, junction resolution,
  forward slashes, Chinese filename, empty path, nonexistent file
- TestLoadExportRowsIntegration: 5 tests using fixture JSON files
- Fixtures: bbt_export_absolute.json, bbt_export_storage.json,
  bbt_export_mixed.json
2026-04-24 20:30:32 +08:00
Research Assistant
1de282e51f fix(phase-10): resolve pre-existing test failures and import errors
- Add missing __init__.py files to pipeline package (3 files)
- Fix load_export_rows to normalize bare Zotero paths with storage: prefix
- Update test_generic_exception assertion: doctor -> diagnose

Fixes 3 test failures + 2 collection errors discovered during verification.
Test suite: 178 passed, 2 skipped, 0 failed
2026-04-24 18:17:50 +08:00
Research Assistant
b14039852b test(09-command-unification): update tests for new command names
- Update test_command_docs.py to reference new pf-* command docs
- Add TestUnifiedCommandsInUserDocs to verify no old commands in primary docs
- Update test_smoke.py docstring for sync command
2026-04-24 16:56:47 +08:00
Research Assistant
45de80da5d refactor(phase-9): update all Python imports to paperforge
- Updated 35 files: from paperforge_lite.* → from paperforge.*
- Verified: pytest passes (172 passed, 2 skipped)
2026-04-24 16:15:30 +08:00
Research Assistant
766ed1edfa feat(08-deep-helper-deployment): add rollback to prepare_deep_reading
- Wrap figure-map, chart-type-map, and scaffold generation in try/except
- Track written files for cleanup on failure
- Save original formal note content before modification
- On exception: delete partial files and restore original note
- Add test_prepare_rollback.py with 3 tests:
  - figure-map failure triggers rollback
  - scaffold failure triggers rollback
  - success path leaves all files intact
- Fix zotero_key quote stripping in scan_deep_reading_queue
2026-04-24 14:37:13 +08:00
Research Assistant
802d34bff6 test(08-deep-helper-deployment): add smoke tests and fixtures
- Add conftest.py with test_vault fixture for deterministic test vault
- Add test_smoke.py with 14 regression tests covering:
  - setup wizard pip install step
  - doctor importability and env name checks
  - ld_deep.py import from deployed location
  - prepare produces scaffold with figure-map and chart-type-map
  - queue shows ready papers with correct ocr_status
  - doc commands extractable and executable
  - metadata fields (first_author, journal) populated
  - BBT PDF path resolution
- Fix zotero_key quote stripping in scan_deep_reading_queue
2026-04-24 14:34:10 +08:00
Research Assistant
afa19eb5b2 feat(08-deep-helper-deployment): create OCR-complete fixtures for TSTONE001
- Add synthetic 骨科.json export with TSTONE001 metadata
- Add generate_ocr_fixture.py script for deterministic fixture generation
- Generate fulltext.md, figure-map.json, chart-type-map.json, meta.json
- Fixtures represent a realistic biomechanical paper with 4 figures, 2 tables
2026-04-24 13:13:45 +08:00
Research Assistant
d0ba1da715 fix(run_repair): case 2 repair updates all three + case 3 handles None; add test_fix_case4 2026-04-24 00:42:44 +08:00
Research Assistant
6052c823e6 Add run_repair() function for three-way OCR state divergence repair 2026-04-24 00:02:32 +08:00
Research Assistant
981f5fa8ba fix(run_deep_reading): call validate_ocr_meta() before using ocr_status
- Replace direct meta.get('ocr_status') read with validate_ocr_meta() call
  in run_deep_reading() to properly validate OCR completion status
- Fixes bug where meta.json with ocr_status=done but missing files
  incorrectly marked paper as ready for deep reading
- validate_ocr_meta() checks 7 conditions before returning 'done':
  file existence, sizes, page markers, etc.
- done_incomplete status is correctly treated as blocked (not ready)
  by existing blocked logic at line 2810
- Add regression test: test_smoke_deep_reading_done_incomplete_is_blocked
2026-04-23 23:42:09 +08:00
Research Assistant
1d8478ce9c fix(wizard): auto-run pip install -e . and improve UX flow
- setup_wizard.py: _deploy() now calls pip install -e <repo_root> automatically
- DoneStep: removes manual 'pip install -e .' instruction (wizard handles it)
- README.md: removes separate pip install -r requirements.txt step
- docs/INSTALLATION.md: same pip install simplification
- pyproject.toml: add pytest ignore for nested sandbox vault
- tests/sandbox/: remove old vault/ (replaced by 00_TestVault/)
- tests/sandbox/README.md: update flow, wizard handles pip install
2026-04-23 20:49:10 +08:00
Research Assistant
31cce820e2 docs(sandbox): add pip install step to README 2026-04-23 20:34:13 +08:00
Research Assistant
e3151a6368 feat(tests/sandbox): add generate_sandbox.py with 5 mock papers across 2 domains 2026-04-23 20:27:26 +08:00
Research Assistant
dda7886d41 feat(tests): add sealed sandbox vault for paperforge testing 2026-04-23 20:18:27 +08:00
Research Assistant
b9f06ccad3 feat: complete PaperForge Lite v1.0 release hardening (5 phases, 28 reqs, 145 tests) 2026-04-23 20:15:20 +08:00
Research Assistant
935d948ede feat(05-01): add OCR state machine tests covering job transitions
- Test pending->queued transition on job submission
- Test processing->done transition when polling returns success
- Test processing->error transition on API error response
- Test processing->blocked transition via HTTPError 401 (classify_error maps to blocked)
- Test sync_ocr_queue skips done/blocked items from existing queue
- Test cleanup_blocked_ocr_dirs removes empty dirs, preserves dirs with payload
- Test all 7 OCR states (pending, queued, running, done, error, blocked, nopdf) don't crash

Key: patch requests.post with side_effect=HTTPError 401 to trigger
classify_error path, since registry token is always present in test env.
2026-04-23 19:37:15 +08:00
Research Assistant
405fc3f127 test(05-02): add end-to-end smoke test suite covering full pipeline
Smoke tests (13 total):
- Setup validation (run_doctor returns 0/1, output checks)
- Selection sync (returns 0, scans exports, creates records)
- Index refresh (returns 0, generates formal Literature notes)
- OCR doctor L1-L3 (returns int code, detects missing token)
- Deep-reading queue (returns 0, produces 3-state output)
- CLI main entry (doctor/status return int, not sys.exit)

All tests use tmp_path fixtures, no real vault, no live network calls.
2026-04-23 18:46:28 +08:00
Research Assistant
1704395d34 test(05-02): add fixture vault factory for smoke testing
- fixture_vault: creates complete vault structure with paperforge.json
- fixture_library_records: 3 minimal library-record .md files
- fixture_bbt_json: Better BibTeX JSON export in exports/
- fixture_with_pdf: vault with real temp PDF for PDF-resolver tests
- All use pathlib.Path, Windows-compatible
2026-04-23 18:46:13 +08:00
Research Assistant
2a3295ed38 feat(02-03): add ocr doctor CLI dispatch, tests, and docs
- Convert ocr parser to sub-subcommands (run, doctor)
- Add _cmd_ocr_doctor() with formatted tiered report output
- Update dispatch logic to route ocr doctor subcommand
- Add CLI dispatch test for doctor command
- Update command/lp-ocr.md with doctor documentation
2026-04-23 12:59:04 +08:00
Research Assistant
3bd1c5ea69 feat(02-03): implement ocr_doctor() with L1-L4 diagnostics
- Add paperforge_lite/ocr_diagnostics.py with tiered L1-L4 checks
- L1: token presence, L2: URL reachability, L3: API schema validation
- L4: optional live PDF round-trip test with polling
- Add 7 mocked unit tests for all levels
- Include blank.pdf test fixture
2026-04-23 12:58:54 +08:00
Research Assistant
24e3d9fb30 test(02-01): add failing tests for pdf_resolver path resolution
- 8 test cases covering absolute, vault-relative, junction, storage-relative
- Missing file, has_pdf=False, empty path, whitespace path scenarios
- is_valid_pdf and resolve_junction helper coverage
2026-04-23 12:54:38 +08:00
Research Assistant
fec25c2d3e feat(01-03): wire worker and /LD-deep to shared resolver
- Replace literature_pipeline.load_vault_config body with
  paperforge_lite.config.load_vault_config wrapper
- Rebuild pipeline_paths() from shared paperforge_paths() plus
  10 worker-only keys (pipeline, candidates, search_*, harvest_root,
  records, review, config, queue, log, bridge_config*, index, ocr_queue)
- Replace ld_deep._load_vault_config and _paperforge_paths with
  shared resolver wrappers returning ocr, records, literature
- Fix subprocess test to set PYTHONPATH so paperforge_lite is
  importable when worker runs as a standalone script
- 34 tests pass (22 config + 8 worker compat + 4 ld_deep compat)
2026-04-23 12:00:11 +08:00
Research Assistant
618227eb9b test(01-03): add failing tests for worker and /LD-deep resolver integration
- Test literature_pipeline.load_vault_config matches paperforge_lite.config
- Test PAPERFORGE_SYSTEM_DIR env var override in worker
- Test pipeline_paths returns all 17 expected keys (shared + worker-only)
- Test ld_deep._load_vault_config and _paperforge_paths match shared resolver
- Test direct worker status subprocess smoke test (CMD-02)
- Tests fail against current code: worker has 5 config keys vs shared's 7,
  env overrides not implemented, and pipeline_paths lacks shared resolver keys
2026-04-23 11:55:24 +08:00
Research Assistant
216ea4e8ac test(01-02): add failing test for CLI paths output and worker dispatch
- Tests main() with --vault vault paths --json emits valid JSON with vault, worker_script, ld_deep_script keys
- Tests main() with --vault vault paths emits text paths without unresolved <system_dir> or <resources_dir> tokens
- Tests ocr and ocr run both dispatch to run_ocr
- Tests status/selection-sync/index-refresh/deep-reading each dispatch to correct worker
- Uses monkeypatch to isolate from real workers and filesystem
2026-04-23 11:44:09 +08:00
Research Assistant
baa2d77bbd test(01-04): add failing tests for stable command docs
- Verify lp-* docs contain paperforge status|selection-sync|index-refresh|ocr run
- Verify ld-deep.md mentions paperforge deep-reading and paths --json
- Verify no legacy python <system_dir>/.../literature_pipeline.py in user examples
- Scope exclusion: AGENTS.md frontmatter examples, architecture diagrams

RED phase: all 15 assertions fail before implementation.
2026-04-23 11:37:40 +08:00
Research Assistant
6b0b16b1dd test(01-01): add failing test for config resolver contract
- Tests DEFAULT_CONFIG defaults (99_System, 03_Resources, Literature, etc.)
- Tests ENV_KEYS has all required PAPERFORGE_* overrides
- Tests load_vault_config precedence: overrides > env > JSON > defaults
- Tests nested vault_config honored per CONF-04
- Tests top-level keys override nested for backward compat per CONF-04
- Tests paperforge_paths returns exactly required user-facing keys
- Tests paths are absolute Path objects
- Tests worker_script and ld_deep_script keys present
- Tests paths_as_strings returns JSON-serializable dict[str,str]
- Tests resolve_vault precedence: cli > env > json search > cwd

RED phase: module not yet implemented, all tests fail as expected.
2026-04-23 11:35:52 +08:00