Commit graph

380 commits

Author SHA1 Message Date
Research Assistant
de9c8a0f0d fix: restore ROADMAP.md (was rewritten for v1.7, not deleted) 2026-05-04 13:01:59 +08:00
Research Assistant
cb159397d2 chore: delete archived REQUIREMENTS.md and ROADMAP.md (archived to milestones/) 2026-05-04 13:01:47 +08:00
Research Assistant
b294dec386 chore: add v1.6.0 to plugin versions.json 2026-05-04 13:01:30 +08:00
Research Assistant
0b3776909f chore: bump version to v1.6.0 2026-05-04 13:01:13 +08:00
Research Assistant
ab105b2d1a chore: complete v1.6 AI-ready literature asset foundation milestone 2026-05-04 13:00:43 +08:00
Research Assistant
3c7bcf32a7 docs(26-03): complete plugin Copy Context actions 2026-05-04 12:52:16 +08:00
Research Assistant
ac667bfe3e feat(26-03): add Copy Context Quick Action with key resolution
- Add paperforge-copy-context ACTIONS entry with needsKey flag
- Update _runAction to resolve zotero_key from active file frontmatter
- Add needsFilter support for collection context (--all default)
- Implement clipboard copy with JSON validation for context commands
- Add running guard to prevent simultaneous execution
- Add variable timeout (30s context, 60s collection, 600s other)
- Register command palette entries for both context actions
2026-05-04 12:51:07 +08:00
Research Assistant
0f9c74299b docs(26-02): complete context CLI command plan 2026-05-04 12:47:49 +08:00
Research Assistant
60355d4e97 test(26-01): add migration tests for flat-to-workspace, _build_entry workspace write, and idempotency
- test_migrate_flat_note_to_workspace: D-11/D-12 copy-not-move verification
- test_migrate_extracts_deep_reading: D-13 deep-reading.md extraction
- test_migrate_creates_ai_dir: ai/ directory creation
- test_migrate_idempotent_skips_existing: D-15 idempotency
- test_build_entry_writes_to_workspace_after_migration: workspace-aware _build_entry
- test_build_entry_flat_fallback_for_unmigrated_paper: backward compat fallback
- test_build_entry_new_paper_creates_workspace: new paper behavior
- test_run_index_refresh_calls_migrate: spy on run_index_refresh integration
- bonus: test_migrate_returns_zero_when_no_index
2026-05-04 12:46:31 +08:00
Research Assistant
2e02729f83 test(26-02): write 14 tests for context command modes and provenance output
- TestContextSingleKey: single key JSON object with _provenance and _ai_readiness
- TestContextSingleKeyNotFound: missing key returns 1 with error
- TestContextDomainFilter: --domain filters correctly (AIC-03)
- TestContextCollectionFilter: --collection prefix match (AIC-03)
- TestContextAll: --all returns all entries
- TestContextProvenanceTraceability: all 9 provenance path keys present (AIC-04)
- TestContextAiReadinessBlocking: blocking explanation for pdf_ready/fulltext_ready
- TestContextNoEntriesMatch: empty results and no-mode error
- TestContextIndexMissing: missing index guidance
2026-05-04 12:45:52 +08:00
Research Assistant
26a5e001ef feat(26-01): add migrate_to_workspace() and update _build_entry() for workspace path writing
- Add migrate_to_workspace() function to sync.py that copies flat notes to workspace directories
- Extract ## 🔍 精读 section into deep-reading.md in workspace
- Create ai/ directory in each paper workspace
- Wire migration into run_index_refresh() before build_index()
- Update _build_entry() in asset_index.py to write notes to workspace path when workspace dir exists
- Preserve flat path fallback for backward compatibility when workspace dir does not exist
2026-05-04 12:44:25 +08:00
Research Assistant
80664588ff feat(26-02): wire context command into CLI subparser, registry, and dispatch
- Add 'context' to _COMMAND_REGISTRY for dynamic module loading
- Add context subparser with key (nargs='?'), --domain, --collection, --all
- Dispatch in main() to paperforge.commands.context.run(args)
- D-06: always outputs JSON, no --json flag needed
2026-05-04 12:43:43 +08:00
Research Assistant
91b54306e2 feat(26-02): create context command module with single-key and filter modes
- run() reads canonical index, filters by key/domain/collection/all
- _format_context_entry wraps entries with _provenance and _ai_readiness blocks
- AIC-04: blocking explanation when lifecycle != 'ai_context_ready'
- AIC-02: single key outputs single JSON object with provenance
- AIC-03: --domain and --collection output JSON arrays
- D-01: canonical index entry IS the AI context -- no separate pack format
2026-05-04 12:43:14 +08:00
Research Assistant
4a16c225cb docs(26): add workspace migration scope (flat→folder, preserve deep reading) 2026-05-04 12:33:34 +08:00
Research Assistant
07577730f1 docs(26): capture phase 26 context (AI context packs) 2026-05-04 12:01:30 +08:00
Research Assistant
cc21accd4e docs(25-03): complete Base views lifecycle columns + repair build_index plan 2026-05-04 11:36:44 +08:00
Research Assistant
157fc4d0ff fix(25-03): use double-quote YAML wrapping for lifecycle filter values
- Filters now contain single-quoted lifecycle values (e.g. lifecycle = 'fulltext_ready')
  which broke single-quote YAML wrapping; switched to double-quote wrapper
- Updated _render_views_section(), merge_base_views(), _build_base_yaml()
- Updated filter preservation test to use lifecycle-based filter strings
2026-05-04 11:35:17 +08:00
Research Assistant
9dc90b7a3b docs(25-01): update ROADMAP progress to 1/3 2026-05-04 11:28:58 +08:00
Research Assistant
f18213e651 docs(25-01): complete status --json source migration + doctor Index Health plan 2026-05-04 11:28:27 +08:00
Research Assistant
e2ef4e9f02 feat(25-01): refactor run_status to read canonical index + add doctor Index Health
- run_status() reads canonical index via summarize_index() for lifecycle,
  health, and maturity aggregates; falls back to filesystem when index missing
- JSON output includes lifecycle_level_counts, health_aggregate,
  maturity_distribution (or None when falling back)
- Text output shows lifecycle and health lines when index is present
- run_doctor() shows Index Health section with PDF/OCR/Note/Asset Health
  counts and status per dimension
- Brownfield detection: legacy schema, old Base templates, partial OCR assets
- Fixed status_tag mapping to support 'info' status (existing bug)
- 6 new tests covering index-backed JSON output, fallback, text output,
  Index Health with/without index, and mixed health counts

Phase 25-01, Tasks 2+3
2026-05-04 11:26:39 +08:00
Research Assistant
d79275aadc docs(25-02): complete plugin dashboard direct JSON read + doctor/repair Quick Actions 2026-05-04 11:25:31 +08:00
Research Assistant
1a6d936258 feat(25-03): wire repair to call build_index() after fixing source artifacts
- Add build_index() call at end of run_repair() when fix=True or fix_paths=True
- Add 'rebuilt' key to result dict initialization
- Add user-facing messages about repair completion and recovery path (MIG-04)
- Lazy import inside conditional block to avoid circular dependency
- Add tests: build_index called after fix, not called during dry-run,
  rebuilt in result, error fallback on build_index failure
2026-05-04 11:25:12 +08:00
Research Assistant
972b894b1d feat(25-02): add doctor and repair Quick Action buttons
- Add 'paperforge-doctor' action: runs 'python -m paperforge doctor'
- Add 'paperforge-repair' action: runs 'python -m paperforge repair'
- ACTIONS array now has 4 entries: sync, ocr, doctor, repair
- Existing _runAction() handles both new commands unchanged (thin CLI shell)
2026-05-04 11:23:56 +08:00
Research Assistant
dbeb17f1bc feat(25-03): update Base views with lifecycle-based columns, filters, and sort order
- Replace has_pdf/do_ocr/analyze/ocr_status columns with lifecycle/maturity_level/next_step
- Update filters to use lifecycle states instead of raw status combinations
- Add sort by lifecycle ascending to all views
- Update PROPERTIES_YAML and _build_base_yaml properties
- Add sort YAML rendering in _render_views_section and merge_base_views
- Update existing filter tests to match lifecycle semantics
- Add tests: lifecycle columns, removed old columns, lifecycle filters, sort, properties
2026-05-04 11:23:38 +08:00
Research Assistant
ed1b36703c feat(25-02): refactor _fetchStats to read formal-library.json directly
- Replace Python CLI spawn with fs.readFileSync for primary path
- Single-pass aggregation of lifecycle, health, and OCR counts per D-06
- Fallback to CLI spawn when index file is missing (D-07)
- Add fs and path as top-level requires
- Access system_dir via app.plugins.plugins['paperforge'].settings
2026-05-04 11:23:29 +08:00
Research Assistant
3a695a5269 feat(25-01): add summarize_index() helper to asset_index.py
- New function summarize_index() reads canonical index and returns lifecycle,
  health, and maturity aggregates
- Returns None for missing or legacy bare-list index
- 4 new tests: aggregates, missing, legacy format, empty items

Phase 25-01, Task 1
2026-05-04 11:22:46 +08:00
Research Assistant
74e3a6a698 docs(25): create phase 25 surface convergence plans
3 plans in 1 wave:
- 25-01: status --json source migration + doctor Index Health + brownfield detection
- 25-02: Plugin dashboard direct JSON read + doctor/repair Quick Actions
- 25-03: Base views lifecycle columns + repair source-first rebuild pattern
2026-05-04 11:20:00 +08:00
Research Assistant
3f9ff0e14b test(24-02): add integration tests for derived state fields in index entries
- Add TestDerivedStateFields class with 6 test methods
- Verify lifecycle/health/maturity/next_step present after full build
- Verify same fields present after incremental refresh
- Verify lifecycle is a valid state string
- Verify health dict has four dimension keys
- Verify maturity structure (level, level_name, checks, blocking)
- Verify next_step is a valid action string
2026-05-04 10:43:43 +08:00
Research Assistant
1900e96972 feat(24-02): add lifecycle, health, maturity, next_step to canonical index entries
- Import compute_lifecycle, compute_health, compute_maturity, compute_next_step from asset_state
- Call all four functions after entry dict construction in _build_entry()
- New fields flow automatically through build_index() and refresh_index_entry()
2026-05-04 10:42:06 +08:00
Research Assistant
5090d17507 docs(24-01): complete asset state derivation plan
- SUMMARY.md with TDD cycle documentation and 3 auto-fixes
- STATE.md: advanced to next plan, 78% overall progress
- ROADMAP.md: Phase 24 status updated (1/2 plans complete)
- REQUIREMENTS.md: STATE-01..04 and AIC-01 marked complete
2026-05-04 10:38:49 +08:00
Research Assistant
2ab79283d8 refactor(24-01): polish asset_state docstrings and edge cases
- Fix note_health to use ternary expression (ruff SIM108)
- All docstrings describe inputs, outputs, and derivation rules
- Module-level docstring lists all four exports
- All 26 tests pass, ruff: All checks passed
2026-05-04 10:36:22 +08:00
Research Assistant
5f99a6c92e feat(24-01): implement asset_state derivation functions
- compute_lifecycle: six progressive states (indexed → pdf_ready → fulltext_ready → deep_read_done → ai_context_ready)
- compute_health: four dimensions (pdf/ocr/note/asset) with concrete fix instructions
- compute_maturity: level 1-6 with per-check pass/fail and blocking indicator
- compute_next_step: priority-ordered recommendation (sync/ocr//pf-deep/ready)
- All 26 tests pass, pure functions with no filesystem or config imports
2026-05-04 10:35:27 +08:00
Research Assistant
bc7c1ba1e5 test(24-01): add failing tests for asset_state derivation functions
- Class TestComputeLifecycle: 8 tests covering indexed, pdf_ready, fulltext_ready, deep_read_done, ai_context_ready states
- Class TestComputeHealth: 4 tests covering all four health dimensions
- Class TestComputeMaturity: 6 tests covering levels 1-6 with blocking indicators
- Class TestComputeNextStep: 8 tests covering sync, ocr, /pf-deep, ready recommendations
- All 26 tests fail with ModuleNotFoundError (module not yet created)
2026-05-04 10:32:19 +08:00
Research Assistant
da8beac145 docs(24): create phase plan — derived lifecycle, health & maturity (2 plans, 6 tasks) 2026-05-04 10:25:05 +08:00
Research Assistant
3eb98be095 docs(24): capture phase 24 context (lifecycle health maturity) 2026-05-04 10:10:26 +08:00
Research Assistant
de153d2402 fix(23): use Beijing time (UTC+8) for index timestamp, fix test stub signature 2026-05-04 01:08:13 +08:00
Research Assistant
f67f6786e5 docs(23-03): complete incremental refresh wiring plan
- 4 tasks, 5 files modified/created
- OCR, deep-reading, repair now use incremental index refresh
- sync.py documented with full-rebuild convention
- 9 integration tests added
- Phase 23 complete (3/3 plans)
2026-05-04 00:57:16 +08:00
Research Assistant
e1b54d7319 feat(23-03): add integration tests for incremental refresh across workers
- Add test_asset_index_integration.py with 9 tests:
  - refresh_index_entry preserves unrelated entries
  - refresh_index_entry appends new keys
  - refresh_index_entry falls back to build_index on legacy format
  - refresh_index_entry skips unknown keys gracefully
  - Structural checks for OCR, deep-reading, repair call sites
  - Workspace path fields present and consistent
- Fix indentation bug in repair.py (Rule 1 auto-fix from Task 3)
2026-05-04 00:55:17 +08:00
Research Assistant
31d78c296f feat(23-03): add incremental refresh to repair operations and document sync convention
- Import refresh_index_entry in repair.py
- Call refresh_index_entry after each successful path fix in repair_pdf_paths
- Call refresh_index_entry after each divergence fix in run_repair
- Add docstring to run_index_refresh in sync.py explaining full-rebuild default
  convention vs incremental refresh by key
2026-05-04 00:51:56 +08:00
Research Assistant
0f2b957cc6 feat(23-03): add incremental index refresh to deep-reading status sync
- Import refresh_index_entry from asset_index module
- After status sync report, refresh canonical index for every paper
- Refresh even when synced==0 (formal note content may have changed)
- Failure to refresh logs warning, does not abort the worker
2026-05-04 00:50:41 +08:00
Research Assistant
c412f82390 feat(23-03): switch OCR post-processing to incremental index refresh
- Add refresh_index_entry import from asset_index module
- Replace full rebuild (_sync.run_index_refresh) with incremental refresh per completed OCR key
- Capture done keys before queue filter to preserve them for incremental refresh
- Add ImportError fallback to full rebuild for pre-migration safety
- When no OCR completed, still run full rebuild for sync changes
2026-05-04 00:49:59 +08:00
Research Assistant
88b5240bec docs(23-02): add self-check PASSED to SUMMARY 2026-05-04 00:48:01 +08:00
Research Assistant
00c735e0bf docs(23-02): complete legacy migration, incremental refresh, workspace paths plan
- SUMMARY.md with 3 task commits documented
- STATE.md advanced to Plan 3 of 3
- ROADMAP.md updated for Phase 23 progress (2/3 plans complete)
- REQUIREMENTS.md: ASSET-03 marked complete
2026-05-04 00:47:31 +08:00
Research Assistant
9691d68970 feat(23-02): add workspace path fields, frontmatter mirroring, and --rebuild-index CLI flag
- Add paper_root, main_note_path, fulltext_path, deep_reading_path, ai_path
  to _build_entry() result dict (Phase 22 paper workspace layout, D-12)
- Mirror workspace path fields in formal note frontmatter for Base views (D-14)
- Add --rebuild-index flag to cli.py sync command parser
- Thread rebuild_index parameter through commands/sync.py -> run_index_refresh()
- Existing system paths (ocr_path, meta_path, note_path) preserved (D-13)
- All 14 existing tests pass
2026-05-04 00:46:34 +08:00
Research Assistant
b7f018ea86 feat(23-02): implement incremental refresh (refresh_index_entry) and schema_version check
- Extract _build_entry() from build_index() loop body as shared helper
- Add refresh_index_entry(vault, key) for single-entry incremental update
- Add schema_version mismatch detection in build_index()
- Lazy imports inside _build_entry() avoid circular deps with sync.py
- Legacy format detected in refresh falls back to full rebuild
- All 14 existing tests pass
2026-05-04 00:44:23 +08:00
Research Assistant
4879874392 feat(23-02): add legacy format detection, backup, and auto-migration to build_index
- Add read_index(vault) to read existing index file safely
- Add is_legacy_format(data) to distinguish bare-list from envelope
- Add migrate_legacy_index(vault) to copy legacy index to .bak before rebuild
- Modify build_index() to call migrate_legacy_index at start
- Corrupt/missing files handled gracefully (treated as fresh build)
2026-05-04 00:42:16 +08:00
Research Assistant
54b9a75c5e docs(23-01): complete canonical-asset-index-safe-rebuilds plan 2026-05-04 00:38:09 +08:00
Research Assistant
b8a57eeb2e test(23-01): add tests for asset_index.py core functionality
- TestGetIndexPath: path resolution to indexes/formal-library.json
- TestBuildEnvelope: empty/with-items/count-matching/ISO timestamp
- TestAtomicWriteIndex: file creation, content correctness, atomicity
  on interrupt, parent dir creation, overwrite, lock timeout via threading
- TestBuildIndexEmpty: empty exports dir and absent exports dir return 0
  and write empty envelope
- 14 tests passing
2026-05-04 00:36:15 +08:00
Research Assistant
7c96b8e15a feat(23-01): create asset_index.py module with envelope, atomic writes, and build_index extraction
- New asset_index.py module with build_index(), get_index_path(), atomic_write_index()
- build_envelope() wraps items in versioned envelope (schema_version, generated_at, paper_count, items)
- atomic_write_index() uses tempfile.NamedTemporaryFile + os.replace for Windows-safe atomic writes
- Cross-process locking via filelock.FileLock with 10-second timeout
- build_index() extracted from sync.py run_index_refresh loop (lines 1686-1746)
- sync.py run_index_refresh() delegates to asset_index.build_index()
- Orphaned-record cleanup stays in sync.py
2026-05-04 00:34:54 +08:00
Research Assistant
3486593a4e chore(23-01): add filelock>=3.13.0 dependency
- Added filelock>=3.13.0 to pyproject.toml dependencies
- Required for cross-process locking during index writes
2026-05-04 00:29:08 +08:00