obsidian-plugin/lllin000_PaperForge
1
0
Fork 0
mirror of https://github.com/lllin000/PaperForge.git synced 2026-07-22 06:50:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
216 commits 4 branches 19 tags 51 MiB
Commit graph

216 commits

This branch
This branch
All branches
Author SHA1 Message Date
Research Assistant
0505fd2fda feat: add paperforge setup CLI command, one-click install script, sync __version__ to 1.4.0, detect local version from pip metadata 2026-04-27 21:34:03 +08:00
Research Assistant
94f00e990a fix: sync paperforge.json version to 1.4.0; fix update.py missing GITHUB_REPO/GITHUB_ZIP imports; remove PaperForge.base generation (duplicates Literature Hub) 2026-04-27 21:25:00 +08:00
Research Assistant
f31efec521 chore: remove PaperForge.base generation (duplicates Literature Hub) 2026-04-27 21:20:14 +08:00
Research Assistant
2bfbec691e fix: add zotero_dir to paperforge_paths() for PDF resolution; fix L3 diagnostic to skip schema check without file upload 2026-04-27 21:12:29 +08:00
Research Assistant
ee260e05f5 fix: L2 diagnostic accepts 400 as reachable (PaddleOCR rejects bare POST without fileUrl) 2026-04-27 20:41:49 +08:00
Research Assistant
13b3e142cc fix: OCR diagnostic L2 probe uses POST instead of GET (PaddleOCR rejects GET) 2026-04-27 20:34:47 +08:00
Research Assistant
a3270dc791 fix: add creators/extra to load_export_rows row dict, fix journal key lookup for first_author/journal/IF in library-records; add first_author, journal, impact_factor columns to Base views; fix PDF storage/ subdirectory resolution 2026-04-27 20:25:35 +08:00
Research Assistant
67a52a898a fix: VaultStep.__init__ missing step_id and checker in super() call 2026-04-27 19:49:36 +08:00
Research Assistant
c2a29a27d9 docs: update README for v1.4 — add structured logging, tqdm, retry, pre-commit; fix chart type count 14->19; update docs links 2026-04-27 19:17:35 +08:00
Research Assistant
2edb5b65cc docs(v1.4): commit planning artifacts, test files, and phase summaries for phases 13-19 2026-04-27 19:12:03 +08:00
Research Assistant
c9f31119ff docs: clear [Unreleased] section per Keep a Changelog convention after v1.4 release 2026-04-27 19:08:48 +08:00
Research Assistant
917a3b09cd chore: bump version to 1.4.0 for v1.4 release 2026-04-27 19:06:55 +08:00
Research Assistant
d521d9c50f docs(18-documentation-ux-polish): complete 18-02 plan — SUMMARY, STATE, ROADMAP 2026-04-27 18:07:29 +08:00
Research Assistant
40868620dc docs(18-documentation-ux-polish): create chart-reading INDEX.md and update ROADMAP.md 
- Create chart-reading/INDEX.md listing all 19 guides ordered by biomedical commonness
- Update ROADMAP.md Phase 18 details with 2 plans (1 wave)
- Mark 18-01 as complete, update progress table to 2/1
 2026-04-27 18:06:08 +08:00
Research Assistant
78bf95b818 docs(18-documentation-ux-polish): add ADR-012/013 and update AGENTS.md command table 
- Add ADR-012 (Shared Utilities Extraction) and ADR-013 (Dual-Output Logging)
- Add command mapping table to AGENTS.md Section 1
- Add --verbose and auto_analyze_after_ocr notes to Section 5
- Add --verbose variants to all CLI examples in Section 8
- Add chart-reading INDEX.md cross-reference
- Update Section 11 with v1.4 migration guide link
 2026-04-27 18:06:03 +08:00
Research Assistant
729ce990c7 docs(18-documentation-ux-polish): create MIGRATION-v1.4.md and fix README orphaned lines 
- Create docs/MIGRATION-v1.4.md following v1.2 template structure
- Document all v1.3-to-v1.4 changes: dual-output logging, retry, pre-commit, auto_analyze_after_ocr
- Remove orphaned legacy code lines 102-104 from README.md
- Verify no other rendering issues in doc audit
 2026-04-27 18:05:58 +08:00
Research Assistant
5713620c1d docs(18-documentation-ux-polish): complete 18-01 plan — SUMMARY, STATE, ROADMAP 
Plan 18-01 adds auto_analyze_after_ocr config+OCR hook, creates
CHANGELOG.md (Keep a Changelog, v1.0-v1.4), and creates CONTRIBUTING.md
(dev setup, hooks, test workflow, architecture, conventions).

203 tests pass. 0 regressions.
 2026-04-27 18:01:11 +08:00
Research Assistant
2096039e03 docs(DX-04): create CONTRIBUTING.md with dev setup and conventions 
- Development setup instructions (pip install -e .[test])
- Pre-commit hooks setup and hook descriptions
- Test workflow with common pytest invocations
- Architecture overview (two-layer design, leaf module rule)
- Code conventions (10 rules: logging, imports, commit format, etc.)
- Pull request process and getting help section
 2026-04-27 17:59:18 +08:00
Research Assistant
935a8fe7fb docs(DX-03): create CHANGELOG.md in Keep a Changelog format 
- Add version history from v1.0.0 through v1.4.0 plus [Unreleased]
- Each version has Added, Changed, Fixed subsections with concise bullets
- Follows https://keepachangelog.com/en/1.1.0/ standards
- 162 lines covering initial MVP through current v1.4 code health release
 2026-04-27 17:58:44 +08:00
Research Assistant
93713be21e feat(UX-01): add auto_analyze_after_ocr config and OCR hook 
- Add auto_analyze_after_ocr: false to paperforge.json (opt-in, default off)
- Insert hook in ocr.py after ocr_status=done to set analyze:true when
  config is enabled
- Wrapped in try/except so a single failure does not abort OCR batch
- Uses read_json (already imported) and re.sub (already imported)
- Regex pattern matches existing frontmatter convention
 2026-04-27 17:57:33 +08:00
Research Assistant
a27fada582 docs(18-documentation-ux-polish): create phase plan — 2 plans in 1 wave 
Plan 18-01: Core Config & Foundation Docs (auto_analyze_after_ocr,
  CHANGELOG, CONTRIBUTING) — UX-01, DX-03, DX-04
Plan 18-02: Migration, Architecture & Doc Polish (MIGRATION-v1.4, ADRs,
  AGENTS table, INDEX, README fix, ROADMAP) — DOCS-01/02/03/04, UX-02/03/04
 2026-04-27 17:55:57 +08:00
Research Assistant
e77f41f68c docs(17-dead-code-precommit): complete 17-01 plan — SUMMARY, STATE, ROADMAP 2026-04-27 17:42:33 +08:00
Research Assistant
7cccf4eefc refactor(17-dead-code-precommit): remove delegation wrappers and dead code, run ruff cleanup 
- Remove def load_vault_config delegation wrappers from all 7 worker modules
- Add from paperforge.config import load_vault_config, paperforge_paths at module level
- Remove unused system_dir/resources_dir/control_dir extraction in pipeline_paths
- Fix sync.py intra-function imports (run_selection_sync, run_index_refresh)
- Fix repair.py import path for _resolve_formal_note_path (direct from _utils)
- Add ruff and pre-commit as test dependencies in pyproject.toml
- Configure per-file ruff ignores for pre-existing issues
- Run ruff check --fix + ruff format (353 + 58 issues auto-fixed)
- All 203 tests pass, ruff check zero warnings
 2026-04-27 17:40:47 +08:00
Research Assistant
c08f86de0f feat(17-dead-code-precommit): add ruff config, pre-commit hooks, Check 5, and OCR error context 
- Add [tool.ruff] section to pyproject.toml (target-version py310, rules E/F/I/UP/B/SIM)
- Create .pre-commit-config.yaml with 7 hooks (ruff lint+format, check-yaml, check-toml, end-of-file-fixer, trailing-whitespace, consistency-audit)
- Add Check 5 (check_duplicate_utils) to scripts/consistency_audit.py
- Add library_record field to OCR error meta.json (4 error paths in ocr.py)
 2026-04-27 17:29:13 +08:00
Research Assistant
1d0b666d2c docs(17-dead-code-precommit): create 17-01 plan for dead code removal + pre-commit hooks 
Single plan across 2 waves:
- Wave 1: ruff config, .pre-commit-config.yaml, Check 5, OBS-05
- Wave 2: ruff check --fix, delegation wrapper removal, test verification

Covers requirements CH-04, DX-01, DX-02, OBS-05.
 2026-04-27 17:26:05 +08:00
Research Assistant
a8f0b5b348 docs(15-01): complete deep-reading queue merge plan 
- SUMMARY.md: documents canonical scan_library_records() implementation
- STATE.md: updated position, metrics, and session continuity
- ROADMAP.md: Phase 15 marked complete (1 plan, 1 summary)
- REQUIREMENTS.md: CH-03 marked complete
 2026-04-27 16:00:44 +08:00
Research Assistant
d472c30d6b feat(15-01): refactor deep_reading.py and ld_deep.py to use scan_library_records() 
deep_reading.py:
- Added scan_library_records, _resolve_formal_note_path to _utils imports
- Replaced _resolve_formal_note_path body with re-export comment
- Replaced inline scan loop with scan_library_records() call
- Retained status sync, categorization, and report generation logic

ld_deep.py:
- Added module-level import from paperforge.worker._utils
- Replaced scan_deep_reading_queue() body with thin wrapper
- Wrapper filters deep_reading_status!=done and sorts by OCR/domain/key
- All processing pipeline tests pass (203/203)
 2026-04-27 15:58:23 +08:00
Research Assistant
e24ef4ca6f feat(15-01): add scan_library_records() and _resolve_formal_note_path() to _utils.py 
- Add _resolve_formal_note_path() with function-level paperforge_paths import
- Add scan_library_records() as pure data acquisition function
- Returns all D-03 fields: zotero_key, domain, title, analyze, do_ocr,
  deep_reading_status, ocr_status, note_path
- Uses same regex patterns as ld_deep.py for frontmatter extraction
- No side effects, no categorization, no sorting (caller's responsibility)
- Part of new '# --- Deep-Reading Queue ---' section
 2026-04-27 15:55:07 +08:00
Research Assistant
99022b0948 docs(15-deep-reading-queue-merge): create 15-01 plan for canonical scan_library_records() 
Phase 15 merges two divergent queue-scanning implementations (worker/deep_reading.py
and skills/ld_deep.py) into a single canonical scan_library_records() in _utils.py.
Plan 01: add shared function + refactor both callers. 2 tasks, 1 wave.
 2026-04-27 15:52:57 +08:00
Research Assistant
0beb575c30 fix(13-logging-foundation): update test stubs and capsys→caplog for logging migration 
- Add verbose=False to all stub function signatures in cli dispatch tests
- Switch test_verbose_output_printed from capsys to caplog since repair
  diagnostics now route through logging (stderr) not print (stdout)
 2026-04-27 15:21:10 +08:00
Research Assistant
3a9256af82 docs(13-logging-foundation): complete 03-verbose-wiring plan 
- SUMMARY.md created and self-verified
- STATE.md updated with position and metrics
- ROADMAP.md updated with plan progress (3/3 plans complete)
 2026-04-27 15:18:08 +08:00
Research Assistant
f70ea85f9d feat(13-logging-foundation): wire verbose through sync/ocr/status commands 
- commands/sync.py: pass verbose to run_selection_sync and run_index_refresh
- commands/ocr.py: pass verbose to run_ocr
- commands/status.py: pass verbose to run_status
- Pattern: verbose=getattr(args, 'verbose', False)
- deep.py and repair.py already had this wiring, no changes needed
 2026-04-27 15:16:41 +08:00
Research Assistant
00692ef667 feat(13-logging-foundation): add verbose param to worker function signatures 
- run_selection_sync(vault) -> run_selection_sync(vault, verbose=False)
- run_index_refresh(vault) -> run_index_refresh(vault, verbose=False)
- run_ocr(vault) -> run_ocr(vault, verbose=False)
- run_doctor(vault) -> run_doctor(vault, verbose=False)
- run_status(vault) -> run_status(vault, verbose=False)
- Backward compatible: default False preserves existing call sites
 2026-04-27 15:15:43 +08:00
Research Assistant
8bae45e301 docs(13-logging-foundation): add self-check result to 13-02-SUMMARY.md 2026-04-27 15:13:43 +08:00
Research Assistant
a52049289d docs(13-logging-foundation): complete 13-02 module-level loggers plan 2026-04-27 15:13:16 +08:00
Research Assistant
03aa680d7f docs(13-logging-foundation): complete 01-logging-foundation plan 2026-04-27 15:12:31 +08:00
Research Assistant
492c4c2790 feat(13-logging-foundation): migrate diagnostic print() calls to logger.*() 
- repair.py: 10 [repair]-tagged diagnostic prints migrated to logger.error/info/warning with printf-style formatting
- commands/ocr.py: 4 [INFO]/[WARN] diagnostic prints migrated to logger.info/warning
- worker/update.py: _log() and _color() functions removed; all 36 _log() call sites migrated to logger.*()
- All user-facing stdout print() calls preserved (4 in repair.py, 6 in commands/ocr.py _diagnose(), 2 in update.py for pip/git output)
- All migrated calls use printf-style %%s/%%d formatting for lazy evaluation
 2026-04-27 15:12:04 +08:00
Research Assistant
ca7d665896 feat(13-logging-foundation): add global --verbose flag and wire configure_logging() in cli.py 
- Import configure_logging from paperforge.logging_config
- Add --verbose/-v as root parser argument (global flag for all subcommands)
- Remove per-subcommand --verbose from deep-reading and repair subparsers
  (users now use 'paperforge -v deep-reading' style instead)
- Call configure_logging(verbose=args.verbose) in main() before command dispatch
- validate: root-level --verbose works across all subcommands
- deviation per Rule 2: subparser --verbose removed; backward compat for
  'deep-reading -v' requires root-level style 'paperforge -v deep-reading'
  (argparse does not inherit root flags to subparsers)
 2026-04-27 15:10:26 +08:00
Research Assistant
e0dbfeccec feat(13-logging-foundation): add module-level logger to all 12 worker and command modules 
- Added import logging and logger = logging.getLogger(__name__) to:
  worker: repair, deep_reading, ocr, sync, status, base_views, update
  commands: ocr, repair, deep, sync, status
- 12 of 13 planned files completed (paperforge/update.py does not exist on disk - skipped)
- Verified all logger instances resolve to correct hierarchical names
 2026-04-27 15:07:32 +08:00
Research Assistant
abe9afe3b2 feat(13-logging-foundation): create paperforge/logging_config.py with configure_logging() 
- Single configure_logging(verbose) entry point for Python stdlib logging
- Reads PAPERFORGE_LOG_LEVEL env var (default INFO)
- Invalid env var values silently fall back to WARNING
- verbose=True overrides to DEBUG
- Idempotency guard: if handlers already exist, only adjust level
- StreamHandler writes to stderr with LEVEL:name:message format
- get_paperforge_logger() convenience accessor for early-boot use
 2026-04-27 15:06:23 +08:00
Research Assistant
e4b07b8adf docs: complete v1.4 code health research — stack, features, architecture, pitfalls synthesis 2026-04-25 23:48:59 +08:00
Research Assistant
e5802f1630 docs: clarify dual-mode command invocation in AGENTS.md and README.md 
- AGENTS.md: Add /pf-sync, /pf-ocr, /pf-status to Agent command table
  with explanation of dual-mode invocation (CLI vs Agent wrapper)
- README.md: Update core commands section with all 5 /pf-* commands
  and dual-mode explanation; remove outdated worker script reference
- Remove obsolete pipeline/ directory from README structure diagram
 2026-04-25 00:37:54 +08:00
Research Assistant
05d718af84 chore: remove user data and old files from repo 
Removed:
- 05_Bases/ (Obsidian user data)
- 99_System/ (user config data)
- update.py (old script, superseded by paperforge/worker/update.py)

Updated .gitignore to prevent future commits of user data.
 2026-04-25 00:25:37 +08:00
Research Assistant
ec2e91d4ac feat(update): smart update with auto-detection + Windows one-click script 
paperforge update:
- Auto-detects installation method (pip/pip-editable/git/zip)
- Routes to appropriate update strategy
- pip users: pip install --upgrade
- pip-editable users: git pull + pip install -e .
- git users: git pull
- unknown: zip download fallback

scripts/update-paperforge.ps1:
- Windows one-click update script
- Auto-detects install method
- Supports -Force (skip confirm) and -DryRun
- Clear colored output with error handling

AGENTS.md:
- Added comprehensive update section with 4 methods
- Documented auto-update and PowerShell script
 2026-04-25 00:17:54 +08:00
Research Assistant
660204352f fix(cli): add missing 'update' subcommand 
- worker/update.py had run_update() but CLI never registered it
- Users can now run: paperforge update
 2026-04-25 00:08:46 +08:00
Research Assistant
4d57d17975 fix(v1.3-doctor): update worker_script path and doctor checks for v1.3 architecture 
- config.py: worker_script now points to paperforge/worker/__init__.py
  (pipeline/ removed in Phase 12)
- worker/status.py: doctor checks paperforge.worker importability
  instead of old literature_pipeline.py existence
- tests/test_config.py: update assertion for new worker_script name
 2026-04-25 00:02:53 +08:00
Research Assistant
b99087c457 milestone(v1.3): complete Path Normalization & Architecture Hardening 
- Phase 11: Zotero Path Normalization (8 tasks)
  - _normalize_attachment_path(): 3 BBT format support
  - _identify_main_pdf(): hybrid main PDF selection
  - obsidian_wikilink_for_pdf(): standard [[relative/path]] format
  - Enhanced doctor/repair with path_error handling
  - 25 new tests, all passing

- Phase 12: Architecture Cleanup (11 tasks)
  - Migrated 4041-line literature_pipeline.py to 7 worker modules
  - paperforge/worker/: sync, ocr, repair, status, deep_reading, base_views, update
  - Migrated skills/literature-qa/ to paperforge/skills/
  - Eliminated pipeline/ and skills/ directories
  - 203 tests passed, 2 skipped, 0 failed
  - Consistency audit 4/4 passing

Archive: .planning/milestones/v1.3.md
 2026-04-24 23:47:48 +08:00
Research Assistant
535d6fe927 docs(phase-12): complete architecture cleanup summary and state update 
- Add 12-SUMMARY.md with full execution details
- Update STATE.md: Phase 12 marked COMPLETE
- Document 4 deviations (test patches, circular imports, legacy compat)
- Record 203 passed, 2 skipped, 0 failed
 2026-04-24 22:50:13 +08:00
Research Assistant
87251db0ca chore(phase-12): remove old pipeline/ and skills/ directories 
- Delete pipeline/worker/scripts/literature_pipeline.py (4041 lines)
- Delete pipeline/ directory hierarchy
- Delete skills/literature-qa/ directory hierarchy
- Confirmed no imports reference old locations
 2026-04-24 22:39:09 +08:00
Research Assistant
f54fe8a3e0 test(phase-12): update test imports for new worker module structure 
- Update all pipeline.worker.scripts patches to paperforge.worker.*
- Fix test_base_views.py and test_base_preservation.py imports
- Update test_legacy_worker_compat.py for new module paths
- Fix OCR test patches to target correct module namespaces
 2026-04-24 22:36:54 +08:00