mirror of
https://github.com/lllin000/PaperForge.git
synced 2026-07-22 06:50:53 +00:00
milestone(v1.3): complete Path Normalization & Architecture Hardening
- Phase 11: Zotero Path Normalization (8 tasks) - _normalize_attachment_path(): 3 BBT format support - _identify_main_pdf(): hybrid main PDF selection - obsidian_wikilink_for_pdf(): standard [[relative/path]] format - Enhanced doctor/repair with path_error handling - 25 new tests, all passing - Phase 12: Architecture Cleanup (11 tasks) - Migrated 4041-line literature_pipeline.py to 7 worker modules - paperforge/worker/: sync, ocr, repair, status, deep_reading, base_views, update - Migrated skills/literature-qa/ to paperforge/skills/ - Eliminated pipeline/ and skills/ directories - 203 tests passed, 2 skipped, 0 failed - Consistency audit 4/4 passing Archive: .planning/milestones/v1.3.md
This commit is contained in:
Research Assistant
parent
535d6fe927
commit
b99087c457
4 changed files with 203 additions and 20 deletions
|
|
@ -162,3 +162,42 @@
|
|||
|
||||
---
|
||||
*Milestone v1.2 initiated: 2026-04-24*
|
||||
|
||||
---
|
||||
|
||||
## Milestone v1.3: Path Normalization & Architecture Hardening (2026-04-24)
|
||||
|
||||
**Goal:** Fix real-world Zotero path handling, clean up module architecture, and close gaps discovered during v1.2 execution.
|
||||
|
||||
**Completed:** 2026-04-24
|
||||
|
||||
### What Shipped
|
||||
|
||||
| Phase | Name | Goal |
|
||||
|-------|------|------|
|
||||
| 11 | Zotero Path Normalization | Parse 3 BBT formats, generate wikilinks, multi-attachment support |
|
||||
| 12 | Architecture Cleanup | Eliminate module boundary leaks (pipeline/ → paperforge/worker/) |
|
||||
|
||||
### Key Deliverables
|
||||
|
||||
- `_normalize_attachment_path()` — unified BBT path parsing (absolute Windows, storage: prefix, bare relative)
|
||||
- `_identify_main_pdf()` — hybrid strategy for main PDF vs supplementary
|
||||
- `obsidian_wikilink_for_pdf()` — standard `[[relative/path]]` wikilink generation
|
||||
- `path_error` frontmatter field — granular error states (not_found/invalid/permission_denied)
|
||||
- `paperforge doctor` Path Resolution checks — junction detection, path validation
|
||||
- `paperforge repair --fix-paths` — automatic path error repair
|
||||
- `paperforge/worker/` package — 7 focused modules from 4041-line monolith
|
||||
- `paperforge/skills/literature-qa/` — migrated skills with subdirectory structure
|
||||
|
||||
### Verification
|
||||
|
||||
- 203 tests passed, 2 skipped, 0 failed
|
||||
- Consistency audit 4/4 passing
|
||||
- No `pipeline.worker.scripts` or `skills.literature_qa` imports remain
|
||||
|
||||
### Archive
|
||||
|
||||
`.planning/milestones/v1.3.md`
|
||||
|
||||
---
|
||||
*Milestone v1.3 completed: 2026-04-24*
|
||||
|
|
|
|||
|
|
@ -6,17 +6,25 @@ PaperForge Lite is a local Obsidian + Zotero literature workflow for medical res
|
|||
|
||||
This is a brownfield release-hardening project for `D:\L\Med\Research\99_System\LiteraturePipeline\github-release`, informed by the fuller local implementation under `D:\L\Med\Research\99_System\LiteraturePipeline` and the production Obsidian Base views under `D:\L\Med\Research\05_Bases`.
|
||||
|
||||
## Current Milestone: v1.3 Path Normalization & Architecture Hardening
|
||||
## Completed Milestone: v1.3 Path Normalization & Architecture Hardening
|
||||
|
||||
**Goal:** Fix real-world Zotero path handling, clean up module architecture, and close gaps discovered during v1.2 execution.
|
||||
**Status:** COMPLETE (2026-04-24)
|
||||
**Archive:** `.planning/milestones/v1.3.md`
|
||||
|
||||
**Target features:**
|
||||
- Zotero attachment path normalization (absolute paths → Vault-relative wikilinks)
|
||||
- Multi-attachment support (main PDF + supplementary materials)
|
||||
- Pipeline module boundary cleanup (`pipeline/` → `paperforge/`)
|
||||
**Delivered:**
|
||||
- Zotero attachment path normalization (3 BBT formats → Vault-relative wikilinks)
|
||||
- Multi-attachment support (main PDF + supplementary with hybrid strategy)
|
||||
- Pipeline module boundary cleanup (`pipeline/` → `paperforge/worker/` as 7 modules)
|
||||
- Skill scripts integration (`skills/` → `paperforge/skills/`)
|
||||
- Test dead zone elimination (fix/remove broken tests)
|
||||
- Consistency audit CI integration
|
||||
- Test dead zone elimination (203 passed, 2 skipped, 0 failed)
|
||||
- Enhanced `paperforge doctor` with Path Resolution checks
|
||||
- `paperforge repair --fix-paths` for automatic path error repair
|
||||
|
||||
---
|
||||
|
||||
## Next Milestone: v1.4 (TBD)
|
||||
|
||||
See `.planning/ROADMAP.md` for candidate features.
|
||||
|
||||
## Core Value
|
||||
|
||||
|
|
@ -42,13 +50,13 @@ A new user can install PaperForge, configure their own vault paths and PaddleOCR
|
|||
- ✓ v1.2 unified agent commands under `/pf-*` namespace and simplified CLI (`paperforge sync`, `paperforge ocr`).
|
||||
- ✓ v1.2 documented architecture (10 ADRs), migration guide, and established consistency audit.
|
||||
|
||||
### Active
|
||||
### v1.3 Completed (2026-04-24)
|
||||
|
||||
- [ ] Zotero attachment path normalization (absolute paths, multi-attachments, wikilinks)
|
||||
- [ ] Pipeline module boundary cleanup (`pipeline/` → `paperforge/`)
|
||||
- [ ] Skill scripts integration (`skills/` → `paperforge/skills/`)
|
||||
- [ ] Test dead zone elimination (fix/remove broken tests)
|
||||
- [ ] Consistency audit CI integration (pre-commit / GitHub Action)
|
||||
- ✓ Zotero attachment path normalization (absolute paths, multi-attachments, wikilinks) — Phase 11
|
||||
- ✓ Pipeline module boundary cleanup (`pipeline/` → `paperforge/worker/` as 7 modules) — Phase 12
|
||||
- ✓ Skill scripts integration (`skills/` → `paperforge/skills/`) — Phase 12
|
||||
- ✓ Test dead zone elimination (203 passed, 0 failed) — Phase 12
|
||||
- [ ] Consistency audit CI integration (pre-commit / GitHub Action) — deferred to future
|
||||
|
||||
### Out of Scope
|
||||
|
||||
|
|
|
|||
|
|
@ -6,18 +6,21 @@ See: `.planning/PROJECT.md` (updated 2026-04-24)
|
|||
|
||||
**Core value:** A new user can install PaperForge, configure their own vault paths and PaddleOCR credentials, then run the full literature pipeline with copy-pasteable commands that diagnose failures clearly.
|
||||
|
||||
**Current focus:** Milestone v1.3 — Path Normalization & Architecture Hardening (Phase 12 IN PROGRESS)
|
||||
**Current focus:** Milestone v1.3 — COMPLETE
|
||||
|
||||
## Current Position
|
||||
|
||||
Phase: 12 (COMPLETE)
|
||||
Plan: 12-PLAN.md (11 tasks, 5 waves)
|
||||
Status: All tasks completed, 203 tests passing, 0 failures
|
||||
Last activity: 2026-04-24 — Phase 12 execution complete (498a9ed, ba8a08b, f54fe8a, 87251db)
|
||||
Phase: 12 (COMPLETE)
|
||||
Milestone: v1.3 (COMPLETE)
|
||||
Status: All phases done, 203 tests passing, 0 failures
|
||||
Last activity: 2026-04-24 — Milestone v1.3 completion
|
||||
|
||||
## Next Action
|
||||
|
||||
Phase 13: TBD — Review ROADMAP.md for next phase
|
||||
Milestone v1.3 complete. Next steps:
|
||||
- `/gsd-new-milestone` to start v1.4
|
||||
- Review `.planning/ROADMAP.md` for candidate features
|
||||
- Or end session
|
||||
|
||||
Options:
|
||||
- `/gsd-execute-phase 13` — begin Phase 13 execution
|
||||
|
|
|
|||
133
.planning/milestones/v1.3.md
Normal file
133
.planning/milestones/v1.3.md
Normal file
|
|
@ -0,0 +1,133 @@
|
|||
# Milestone: v1.3 — Path Normalization & Architecture Hardening
|
||||
|
||||
**Completed:** 2026-04-24
|
||||
**Phases:** 11-12 (2 phases)
|
||||
**Commits:** 75 commits since v1.2
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
v1.3 是 **PaperForge** 的第三个里程碑，聚焦两大主题：
|
||||
1. **路径规范化** — 修复 Zotero 附件路径解析问题，支持多附件识别和 Obsidian wikilink 生成
|
||||
2. **架构清理** — 消除模块边界泄漏，将 4041 行的单体脚本拆分为 7 个专注模块
|
||||
|
||||
---
|
||||
|
||||
## Accomplishments
|
||||
|
||||
### Phase 11: Zotero Path Normalization (8 tasks)
|
||||
|
||||
**核心交付：**
|
||||
- `_normalize_attachment_path()` — 统一处理 3 种 BBT 导出格式（absolute Windows, storage: prefix, bare relative）
|
||||
- `_identify_main_pdf()` — 混合策略识别主 PDF vs 补充材料
|
||||
- `obsidian_wikilink_for_pdf()` — 生成标准 `[[relative/path]]` wikilink
|
||||
- `paperforge doctor` 增强 — 检测 junction、验证路径、推荐 mklink 命令
|
||||
- `paperforge repair --fix-paths` — 自动修复路径错误
|
||||
- `paperforge status` — 显示 path_error 统计
|
||||
|
||||
**新 frontmatter 字段：**
|
||||
- `bbt_path_raw` — 原始 BBT 路径（调试）
|
||||
- `zotero_storage_key` — 8 位 Zotero 存储键
|
||||
- `attachment_count` — 附件总数
|
||||
- `supplementary` — 补充材料 wikilink 列表
|
||||
- `path_error` — 路径错误状态
|
||||
|
||||
**测试：** 25 个新测试方法，全部通过
|
||||
|
||||
**提交：** 11 个（包含测试修复）
|
||||
|
||||
### Phase 12: Architecture Cleanup (11 tasks)
|
||||
|
||||
**核心交付：**
|
||||
- `paperforge/worker/sync.py` (~1440 行) — selection-sync, index-refresh, 路径工具
|
||||
- `paperforge/worker/ocr.py` (~1377 行) — OCR 队列、PDF 处理、图表切割
|
||||
- `paperforge/worker/repair.py` — 状态修复、路径修复
|
||||
- `paperforge/worker/status.py` — doctor、status 命令
|
||||
- `paperforge/worker/deep_reading.py` — 深度阅读队列
|
||||
- `paperforge/worker/base_views.py` — Base 视图管理
|
||||
- `paperforge/worker/update.py` — 更新系统
|
||||
- `paperforge/skills/literature-qa/` — 从 skills/ 迁移
|
||||
|
||||
**删除：**
|
||||
- `pipeline/worker/scripts/literature_pipeline.py` (4041 行)
|
||||
- 整个 `pipeline/` 目录
|
||||
- 整个 `skills/` 目录
|
||||
|
||||
**更新：**
|
||||
- 40+ 处导入路径更新
|
||||
- 所有测试文件适配新模块结构
|
||||
- `paperforge/config.py` 更新 `ld_deep_script` 解析
|
||||
|
||||
**测试：** 203 passed, 2 skipped, 0 failed
|
||||
|
||||
**提交：** 5 个
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions
|
||||
|
||||
| ID | Decision | Rationale |
|
||||
|--|---------|-----------|
|
||||
| D-01 | 统一路径转换时机（load_export_rows 阶段） | 避免下游重复处理 |
|
||||
| D-02 | 混合策略识别主 PDF | title==PDF → 最大文件 → 最短标题 |
|
||||
| D-03 | 代码适配所有 BBT 格式 | 不依赖用户配置 |
|
||||
| D-04 | junction 在 absolutize 中解析 | 简化 wikilink 生成逻辑 |
|
||||
| D-05 | 颗粒化错误状态 | path_error 字段记录 not_found/invalid/permission_denied |
|
||||
| D-06 | 完全重构 pipeline → paperforge/worker/ | 与 commands/ 结构对齐 |
|
||||
| D-07 | skills 保留子目录迁移 | 配套资源需要子目录结构 |
|
||||
| D-08 | base_views 提取到独立模块 | 完全模块化标准 |
|
||||
|
||||
---
|
||||
|
||||
## Statistics
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Total commits | 75 |
|
||||
| Phases completed | 2 (11, 12) |
|
||||
| New modules created | 7 |
|
||||
| Old modules removed | 2 (pipeline, skills) |
|
||||
| Lines moved | 4041 → ~2800 (across 7 files) |
|
||||
| Test coverage | 203 passed, 2 skipped |
|
||||
| Consistency audit | 4/4 |
|
||||
|
||||
---
|
||||
|
||||
## Breaking Changes
|
||||
|
||||
None.所有变更都是内部重构，用户界面保持不变：
|
||||
- `paperforge sync` 行为不变
|
||||
- `paperforge ocr` 行为不变
|
||||
- `paperforge doctor` 输出增强（新增 Path Resolution 部分）
|
||||
- Agent 命令 (`/pf-deep`, `/pf-paper`) 不变
|
||||
|
||||
---
|
||||
|
||||
## Deferred to Future
|
||||
|
||||
1. Repair scan O(n*m) 性能优化
|
||||
2. 一致性审计 CI 集成
|
||||
3. Skill 热加载机制
|
||||
4. Supplementary 材料标题确定策略
|
||||
5. 用户可配置路径覆盖
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
```
|
||||
pytest tests/ -v
|
||||
# 203 passed, 2 skipped, 0 failed
|
||||
|
||||
python scripts/consistency_audit.py
|
||||
# 4/4 passing
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Milestone Candidates
|
||||
|
||||
参考 ROADMAP.md:
|
||||
- **v1.4** — Base 视图增强、Obsidian 插件架构准备
|
||||
- **v2.0** — TypeScript Obsidian 插件、UI 界面
|
||||
Loading…
Reference in a new issue