Fix configurable Lite setup paths

This commit is contained in:
Research Assistant 2026-04-23 01:44:13 +08:00
parent e0881f7d2c
commit 881457dd92
18 changed files with 611 additions and 1420 deletions

View file

@ -13,7 +13,7 @@
[ ] Python 依赖已安装 (pip install requests pymupdf pillow)
[ ] PaddleOCR API Key 已配置(在 .env 中)
[ ] 目录结构已创建setup.py 会自动完成)
[ ] Zotero 数据目录已链接到 99_System/Zotero
[ ] Zotero 数据目录已链接到 <system_dir>/Zotero
```
### Better BibTeX 自动导出配置
@ -21,7 +21,7 @@
1. Zotero → Edit → Preferences → Better BibTeX
2. 勾选 **"Keep updated"**(自动导出)
3. 选择导出格式:**Better BibLaTeX** 或 **Better BibTeX**
4. 导出路径设置为:`{你的Vault路径}/99_System/PaperForge/exports/library.json`
4. 导出路径设置为:`{你的Vault路径}/<system_dir>/PaperForge/exports/library.json`
5. 点击 OKJSON 文件会自动生成并保持同步
---
@ -47,13 +47,13 @@ PaperForge Lite 采用 **两层设计**
```
Zotero 添加文献
↓ Better BibTeX 自动导出 JSON
99_System/PaperForge/exports/library.json
<system_dir>/PaperForge/exports/library.json
↓ 运行 selection-sync
03_Resources/LiteratureControl/library-records/<domain>/<key>.md
<resources_dir>/<control_dir>/library-records/<domain>/<key>.md
↓ 运行 index-refresh
03_Resources/Literature/<domain>/<key> - <Title>.md正式笔记
<resources_dir>/<literature_dir>/<domain>/<key> - <Title>.md正式笔记
↓ 用户在 library-record 中设置 do_ocr: true
运行 ocr → 99_System/PaperForge/ocr/<key>/
运行 ocr → <system_dir>/PaperForge/ocr/<key>/
↓ 用户在 library-record 中设置 analyze: true
运行 deep-reading查看队列确认就绪
↓ 用户执行 Agent 命令
@ -68,20 +68,20 @@ Zotero 添加文献
```
{你的Vault根目录}/
├── 03_Resources/
│ ├── Literature/ ← 正式文献笔记index-refresh 生成)
├── <resources_dir>/
│ ├── <literature_dir>/ ← 正式文献笔记index-refresh 生成)
│ │ ├── 骨科/
│ │ ├── 运动医学/
│ │ └── ...(你的分类)
│ └── LiteratureControl/ ← 状态跟踪
│ └── <control_dir>/ ← 状态跟踪
│ └── library-records/ ← selection-sync 输出
│ ├── 骨科/
│ │ └── ABCDEFG.md ← 单条文献状态记录
│ └── 运动医学/
│ └── HIJKLMN.md
├── 99_System/
│ ├── LiteraturePipeline/
├── <system_dir>/
│ ├── PaperForge/
│ │ ├── exports/ ← Better BibTeX 自动导出的 JSON
│ │ │ └── library.json
│ │ ├── ocr/ ← OCR 结果(每个文献一个子目录)
@ -94,7 +94,7 @@ Zotero 添加文献
│ │ └── literature_pipeline.py ← 核心脚本
│ └── Zotero/ ← Junction/Symlink 到 Zotero 数据目录
├── .opencode/ ← OpenCode Agent 配置(自动创建)
├── <agent_config_dir>/ ← OpenCode Agent 配置(自动创建)
│ └── skills/
│ └── literature-qa/ ← 深度阅读 Skill
│ ├── scripts/
@ -110,11 +110,11 @@ Zotero 添加文献
| 目录 | 内容 | 谁生成/修改 |
|------|------|------------|
| `03_Resources/Literature/` | 正式文献笔记(含 frontmatter + 精读内容) | index-refresh 生成Agent 写入精读 |
| `03_Resources/LiteratureControl/library-records/` | 文献状态跟踪analyze, ocr_status 等) | selection-sync 生成,用户修改状态 |
| `99_System/PaperForge/exports/` | Better BibTeX JSON 导出 | Zotero 自动导出 |
| `99_System/PaperForge/ocr/` | OCR 全文 + 图表切割 | ocr worker 生成 |
| `99_System/Zotero/` | Zotero 数据目录的链接 | 安装时手动创建 junction |
| `<resources_dir>/<literature_dir>/` | 正式文献笔记(含 frontmatter + 精读内容) | index-refresh 生成Agent 写入精读 |
| `<resources_dir>/<control_dir>/library-records/` | 文献状态跟踪analyze, ocr_status 等) | selection-sync 生成,用户修改状态 |
| `<system_dir>/PaperForge/exports/` | Better BibTeX JSON 导出 | Zotero 自动导出 |
| `<system_dir>/PaperForge/ocr/` | OCR 全文 + 图表切割 | ocr worker 生成 |
| `<system_dir>/Zotero/` | Zotero 数据目录的链接 | 安装时手动创建 junction |
---
@ -123,23 +123,23 @@ Zotero 添加文献
### selection-sync
- **作用**:检测 Zotero 中的新条目,创建 library-records
- **运行时机**:添加新文献到 Zotero 后
- **输出**`03_Resources/LiteratureControl/library-records/<domain>/<key>.md`
- **输出**`<resources_dir>/<control_dir>/library-records/<domain>/<key>.md`
- **示例**
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py \
--vault "{vault路径}" selection-sync
```
### index-refresh
- **作用**:基于 library-records 生成正式文献笔记
- **运行时机**selection-sync 之后,或需要更新笔记格式时
- **输出**`03_Resources/Literature/<domain>/<key> - <Title>.md`
- **输出**`<resources_dir>/<literature_dir>/<domain>/<key> - <Title>.md`
- **说明**:会读取 Better BibTeX JSON 提取元数据,生成带 frontmatter 的 Obsidian 笔记
### ocr
- **作用**:将 PDF 上传到 PaddleOCR API提取全文文本和图表
- **触发条件**library-record 中 `do_ocr: true`
- **输出**`99_System/PaperForge/ocr/<key>/` 目录
- **输出**`<system_dir>/PaperForge/ocr/<key>/` 目录
- `fulltext.md`:提取的全文(含 `<!-- page N -->` 分页标记)
- `images/`:自动切割的图表图片
- `meta.json`OCR 状态(`ocr_status: done/pending/processing/failed`
@ -198,8 +198,8 @@ year: 2024
doi: "10.xxxx/xxxxx"
collection_path: "子分类" # Zotero 子收藏夹路径
has_pdf: true # 是否有 PDF 附件(自动生成)
pdf_path: "99_System/Zotero/..." # PDF 相对路径(自动生成)
fulltext_md_path: "99_System/PaperForge/ocr/..."
pdf_path: "<system_dir>/Zotero/..." # PDF 相对路径(自动生成)
fulltext_md_path: "<system_dir>/PaperForge/ocr/..."
recommend_analyze: true # 系统推荐精读(有 PDF 时自动设为 true
analyze: false # 【用户控制】是否生成精读?设为 true 触发
do_ocr: true # 【用户控制】是否运行 OCR设为 true 触发
@ -231,7 +231,7 @@ tags:
- 文献阅读
- 子分类
keywords: ["keyword1", "keyword2"]
pdf_link: "99_System/Zotero/..."
pdf_link: "<system_dir>/Zotero/..."
---
```
@ -247,7 +247,7 @@ pdf_link: "99_System/Zotero/..."
```bash
# 在 Vault 根目录执行
python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py \
--vault "你的Vault路径" selection-sync
```
@ -261,21 +261,21 @@ python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
### Step 3: 运行 index-refresh
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py \
--vault "你的Vault路径" index-refresh
```
预期输出:
```
[INFO] Generated 5 formal notes
[INFO] Output: 03_Resources/Literature/骨科/XXXXXXX - Title.md
[INFO] Output: <resources_dir>/<literature_dir>/骨科/XXXXXXX - Title.md
...
```
### Step 4: 标记要精读的文献
在 Obsidian 中:
1. 打开 `03_Resources/LiteratureControl/library-records/骨科/XXXXXXX.md`
1. 打开 `<resources_dir>/<control_dir>/library-records/骨科/XXXXXXX.md`
2. 将 `do_ocr: false` 改为 `do_ocr: true`
3. 将 `analyze: false` 改为 `analyze: true`
4. 保存文件
@ -283,7 +283,7 @@ python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
### Step 5: 运行 OCR
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py \
--vault "你的Vault路径" ocr
```
@ -292,7 +292,7 @@ python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
### Step 6: 检查 OCR 状态
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py \
--vault "你的Vault路径" deep-reading
```
@ -324,23 +324,23 @@ Agent 会自动:
```bash
# 检测 Zotero 新条目
python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py \
--vault "你的Vault路径" selection-sync
# 生成/更新正式笔记
python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py \
--vault "你的Vault路径" index-refresh
# 运行 OCR处理 do_ocr=true 的文献)
python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py \
--vault "你的Vault路径" ocr
# 查看精读队列
python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py \
--vault "你的Vault路径" deep-reading
# 查看整体状态
python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py \
--vault "你的Vault路径" status
```
@ -362,7 +362,7 @@ python 99_System/PaperForge/worker/scripts/literature_pipeline.py \
### Q: OCR 一直显示 pending
- 检查 PaddleOCR API Key 是否配置正确(`.env` 文件)
- 检查网络连接
- 查看 `99_System/PaperForge/ocr/<key>/meta.json` 中的错误信息
- 查看 `<system_dir>/PaperForge/ocr/<key>/meta.json` 中的错误信息
### Q: /LD-deep 提示 OCR 未完成?
- 确认 library-record 中 `ocr_status: done`
@ -387,14 +387,17 @@ cd 你的Vault路径
git pull origin main
# 或手动复制更新文件
cp -r 新下载的scripts/* 99_System/PaperForge/worker/scripts/
cp -r 新下载的scripts/* <system_dir>/PaperForge/worker/scripts/
```
### 备份注意事项
- `03_Resources/` 和 `99_System/PaperForge/ocr/` 包含你的数据,需备份
- `<resources_dir>/` 和 `<system_dir>/PaperForge/ocr/` 包含你的数据,需备份
- `.env` 包含 API Key不要提交到 git
- `99_System/PaperForge/exports/` 可重新生成(由 Zotero 自动导出)
- `<system_dir>/PaperForge/exports/` 可重新生成(由 Zotero 自动导出)
---
*PaperForge Lite | 快速开始指南 | 安装后阅读*

View file

@ -58,17 +58,17 @@ python setup_wizard.py --vault /path/to/your/vault
```
your-vault/
├── [资源目录]/ # 默认: 03_Resources
│ └── LiteratureControl/
├── [资源目录]/ # 安装时可自定义
│ └── [文献索引目录]/
│ └── library-records/ # 文献状态跟踪
├── [系统目录]/ # 默认: 99_System
├── [系统目录]/ # 安装时可自定义
│ ├── PaperForge/
│ │ ├── exports/ # Zotero JSON 导出
│ │ ├── ocr/ # OCR 结果
│ │ └── worker/scripts/
│ │ └── literature_pipeline.py
│ └── Zotero/ # Junction 到 Zotero 数据目录
├── [Agent配置目录]/ # 根据平台: .opencode, .cursor 等
├── [Agent配置目录]/ # 根据平台和安装配置决定
│ └── skills/
│ └── literature-qa/
│ ├── scripts/ld_deep.py
@ -89,10 +89,10 @@ your-vault/
```bash
# Worker 命令
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . selection-sync
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . index-refresh
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . ocr
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . status
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py --vault . selection-sync
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py --vault . index-refresh
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py --vault . ocr
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py --vault . status
# Agent 命令(在 OpenCode 中使用)
/LD-deep <zotero_key> # 完整三阶段精读
@ -102,3 +102,4 @@ python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . stat
## License
MIT License — 允许商业使用,需保留版权声明。

View file

@ -8,9 +8,9 @@
2. 支持 Zotero key、标题片段、DOI、PMID、关键词
3. 优先搜索本地 Zotero 并锁定单篇论文
4. 绑定该论文对应的:
- `99_System/PaperForge/ocr/<KEY>/fulltext.md`
- `99_System/PaperForge/ocr/<KEY>/meta.json`
- `03_Resources/Literature/.../KEY - Title.md`
- `<system_dir>/PaperForge/ocr/<KEY>/fulltext.md`
- `<system_dir>/PaperForge/ocr/<KEY>/meta.json`
- `<resources_dir>/<literature_dir>/.../KEY - Title.md`
5. 在正式文献卡片中检查或创建 `## 🔍 精读`
6. 以“研究思路 + figure-by-figure”方式一次性完成精读写回
@ -189,16 +189,16 @@
| 变量 | 示例值 |
| ----------------- | ----------------------------------------------------------------- |
| `{{ZOTERO_KEY}}` | `Y5KQ4JQ7` |
| `{{FORMAL_NOTE}}` | `D:\L\Med\Research\03_Resources\Literature\骨科\Y5KQ4JQ7 - 3D printed hydrogel for articular cartilage regeneration.md` |
| `{{FULLTEXT_MD}}` | `D:\L\Med\Research\99_System\LiteraturePipeline\ocr\Y5KQ4JQ7\fulltext.md` |
| `{{SCRIPT}}` | `D:\L\Med\Research\.opencode\skills\literature-qa\scripts\ld_deep.py` |
| `{{FORMAL_NOTE}}` | `<Vault>/<resources_dir>/<literature_dir>/骨科/Y5KQ4JQ7 - title.md` |
| `{{FULLTEXT_MD}}` | `<Vault>/<system_dir>/PaperForge/ocr/Y5KQ4JQ7/fulltext.md` |
| `{{SCRIPT}}` | `<Vault>/<skill_dir>/literature-qa/scripts/ld_deep.py` |
### Spawn 命令格式
```
Task(
description="LD-deep {{ZOTERO_KEY}}",
prompt="加载 subagent prompt: D:\L\Med\Research\.opencode\skills\literature-qa\prompt_deep_subagent.md\n\n填入以下变量\n- ZOTERO_KEY: {{ZOTERO_KEY}}\n- FORMAL_NOTE: {{FORMAL_NOTE}}\n- FULLTEXT_MD: {{FULLTEXT_MD}}\n- SCRIPT: {{SCRIPT}}",
prompt="加载 subagent prompt: <Vault>/<skill_dir>/literature-qa/prompt_deep_subagent.md\n\n填入以下变量\n- ZOTERO_KEY: {{ZOTERO_KEY}}\n- FORMAL_NOTE: {{FORMAL_NOTE}}\n- FULLTEXT_MD: {{FULLTEXT_MD}}\n- SCRIPT: {{SCRIPT}}",
subagent_type="general"
)
```

View file

@ -7,7 +7,7 @@
1. 解析 `/LD <query>` 中的查询词
2. 支持 Zotero key、标题片段、DOI、PMID、关键词
3. 优先搜索本地 Zotero解析到单篇目标论文
4. 加载 `D:\L\Med\Research\99_System\LiteraturePipeline\ocr\<KEY>\fulltext.md` 作为主文本
4. 根据 Vault 根目录的 `paperforge.json` 加载 `<system_dir>/PaperForge/ocr/<KEY>/fulltext.md` 作为主文本
5. 读取 `meta.json` 显示论文标题、作者、期刊、年份
6. 进入 Q&A 模式,用中文回答用户关于该论文的问题
7. 在当前论文上下文中,用户可再说“精读这篇文章”切换到 deep 层

View file

@ -1,16 +1,11 @@
# /lp-index-refresh
刷新 LiteraturePipeline 索引,包含孤立记录清理逻辑
根据 library-records 和 Zotero JSON 导出生成正式文献笔记
## Commands
## Command
先读取 Vault 根目录的 `paperforge.json`,用其中的 `system_dir` 拼出 worker 路径,再运行:
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault D:\L\Med\Research index-refresh
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py --vault . index-refresh
```
## 输出示例
```
index-refresh: wrote 684 index rows
index-refresh: cleaned 0 orphaned records in 骨科
```

View file

@ -1,9 +1,11 @@
# /lp-ocr
触发 OCR 处理队列。
处理 library-records 中 `do_ocr: true` 的 PDF OCR 队列。
## Commands
## Command
先读取 Vault 根目录的 `paperforge.json`,用其中的 `system_dir` 拼出 worker 路径,再运行:
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault D:\L\Med\Research ocr
```
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py --vault . ocr
```

View file

@ -1,9 +1,11 @@
# /lp-selection-sync
同步 Zotero 选中项到 library-records。
同步 Zotero Better BibTeX JSON 导出到 library-records。
## Commands
## Command
先读取 Vault 根目录的 `paperforge.json`,用其中的 `system_dir` 拼出 worker 路径,再运行:
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault D:\L\Med\Research selection-sync
```
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py --vault . selection-sync
```

View file

@ -1,27 +1,11 @@
# /lp-status
查看 LiteraturePipeline 状态。
查看 PaperForge Lite 当前安装与运行状态。
## 检查 exports 记录数
## Command
先读取 Vault 根目录的 `paperforge.json`,用其中的 `system_dir` 拼出 worker 路径,再运行:
```bash
python -c "import json; from pathlib import Path; data = json.loads(Path('99_System/PaperForge/exports/骨科.json').read_text('utf-8')); items = [i for i in data.get('items',[]) if i.get('itemType') != 'attachment']; print(f'骨科: {len(items)} records')"
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py --vault . status
```
## 检查 library-records 总数
```bash
python -c "from pathlib import Path; count = sum(1 for _ in Path('03_Resources/LiteratureControl/library-records').rglob('*.md')); print(f'library-records: {count} records')"
```
## 检查 OCR 完成数
```bash
python -c "from pathlib import Path; done = sum(1 for p in Path('99_System/PaperForge/ocr').glob('*/meta.json') if 'done' in Path(p).read_text('utf-8')); print(f'OCR done: {done}')"
```
## 检查索引记录数
```bash
python -c "import json; data = json.load(open('99_System/PaperForge/indexes/formal-library.json', encoding='utf-8')); print(f'Index: {len(data)} records')"
```

View file

@ -50,22 +50,22 @@ pip install requests pymupdf pillow
### Step 2: 创建目录结构
```bash
mkdir -p "{vault_path}/99_System/PaperForge/ocr"
mkdir -p "{vault_path}/99_System/PaperForge/worker/scripts"
mkdir -p "{vault_path}/99_System/Zotero"
mkdir -p "{vault_path}/03_Resources/LiteratureControl/library-records"
mkdir -p "{vault_path}/<system_dir>/PaperForge/ocr"
mkdir -p "{vault_path}/<system_dir>/PaperForge/worker/scripts"
mkdir -p "{vault_path}/<system_dir>/Zotero"
mkdir -p "{vault_path}/<resources_dir>/<control_dir>/library-records"
```
### Step 3: 链接 Zotero 数据目录
**Windows** (管理员终端):
```cmd
mklink /J "{vault_path}\99_System\Zotero" "C:\Users\<User>\Zotero"
mklink /J "{vault_path}\<system_dir>\Zotero" "C:\Users\<User>\Zotero"
```
**macOS/Linux**:
```bash
ln -s "~/Zotero" "{vault_path}/99_System/Zotero"
ln -s "~/Zotero" "{vault_path}/<system_dir>/Zotero"
```
### Step 4: 配置 .env
@ -79,8 +79,8 @@ PADDLEOCR_JOB_URL=https://paddleocr.aistudio-app.com/api/v2/ocr/jobs
### Step 5: 部署脚本
```bash
cp pipeline/worker/scripts/literature_pipeline.py "{vault_path}/99_System/PaperForge/worker/scripts/"
cp -r skills/literature-qa "{vault_path}/.opencode/skills/"
cp pipeline/worker/scripts/literature_pipeline.py "{vault_path}/<system_dir>/PaperForge/worker/scripts/"
cp -r skills/literature-qa "{vault_path}/<skill_dir>/"
cp AGENTS.md "{vault_path}/AGENTS.md"
```
@ -92,7 +92,7 @@ cp AGENTS.md "{vault_path}/AGENTS.md"
```bash
cd "{vault_path}"
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . status
python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py --vault . status
```
预期输出:
@ -132,3 +132,4 @@ Status: OK
5. **开始精读**:使用 `/LD-deep <zotero_key>` 生成结构化阅读笔记
详细用法参见 [AGENTS.md](../AGENTS.md)。

View file

@ -90,9 +90,9 @@ Better BibTeX 导出后:
└── exports/综述.json
PaperForge 自动生成:
├── 05_Bases/骨科.base
├── 05_Bases/运动医学.base
└── 05_Bases/综述.base
├── <resources_dir>/<control_dir>/library-records/骨科/
├── <resources_dir>/<control_dir>/library-records/运动医学/
└── <resources_dir>/<control_dir>/library-records/综述/
```
**规则:一个 JSON 文件 = 一个 Base 视图 = 一个文献分类**
@ -118,7 +118,7 @@ Zotero
├─ 格式(Format): Better BibLaTeX ★ 重要!不是 BibTeX
├─ 勾选 [Keep updated] ★ 必须勾选
├─ 文件名: 骨科.json ★ 建议用中文名,好识别
└─ 保存位置: {你的Vault}/99_System/PaperForge/exports/
└─ 保存位置: {你的Vault}/<system_dir>/PaperForge/exports/
└─ [保存]
```
@ -130,7 +130,7 @@ Zotero
4. 勾选右下角的 `Keep updated`
5. 点击 `...` 选择保存位置,导航到:
```
{你的Vault根目录}/99_System/PaperForge/exports/
{你的Vault根目录}/<system_dir>/PaperForge/exports/
```
6. 文件名填写收藏夹名称,如 `骨科.json`
7. 点击保存
@ -142,7 +142,7 @@ Zotero
导出完成后,检查文件:
```bash
ls 99_System/PaperForge/exports/
ls <system_dir>/PaperForge/exports/
# 应该看到: 骨科.json 运动医学.json ...
```
@ -174,9 +174,9 @@ python pipeline/worker/scripts/literature_pipeline.py --vault . index-refresh
```
此时你应该看到:
- `03_Resources/LiteratureControl/library-records/` 下出现状态记录文件
- `03_Resources/Literature/` 下出现正式笔记
- `05_Bases/` 下出现 Base 视图文件(每个 JSON 对应一个)
- `<resources_dir>/<control_dir>/library-records/` 下出现状态记录文件
- `<resources_dir>/<literature_dir>/` 下出现正式笔记
- `<resources_dir>/<control_dir>/library-records/` 下出现状态记录文件
---
@ -222,3 +222,6 @@ python setup_wizard.py --vault .
---
*PaperForge Lite | 安装指南*

View file

@ -9,12 +9,12 @@
"check_interval_days": 7
},
"protected_paths": [
"03_Resources/",
"05_Bases/",
"99_System/PaperForge/ocr/",
"99_System/PaperForge/exports/",
"99_System/PaperForge/indexes/",
"99_System/PaperForge/candidates/",
"<resources_dir>/",
"<base_dir>/",
"<system_dir>/PaperForge/ocr/",
"<system_dir>/PaperForge/exports/",
"<system_dir>/PaperForge/indexes/",
"<system_dir>/PaperForge/candidates/",
".env",
"AGENTS.md"
],
@ -26,3 +26,6 @@
],
"changelog_url": "https://github.com/LLLin000/PaperForge/releases"
}

View file

@ -45,7 +45,7 @@ def load_journal_db(vault: Path) -> dict[str, dict]:
global _JOURNAL_DB
if _JOURNAL_DB is not None:
return _JOURNAL_DB
zoterostyle_path = vault / '99_System' / 'Zotero' / 'zoterostyle.json'
zoterostyle_path = vault / load_vault_config(vault)['system_dir'] / 'Zotero' / 'zoterostyle.json'
if zoterostyle_path.exists():
try:
_JOURNAL_DB = read_json(zoterostyle_path)
@ -125,27 +125,41 @@ def _extract_year(value: str) -> str:
def load_vault_config(vault: Path) -> dict:
"""Read vault configuration from paperforge.json."""
defaults = {
"system_dir": "99_System",
"resources_dir": "03_Resources",
"literature_dir": "Literature",
"control_dir": "LiteratureControl",
"base_dir": "05_Bases",
}
pf_json = vault / "paperforge.json"
if pf_json.exists():
try:
data = json.loads(pf_json.read_text(encoding="utf-8"))
nested = data.get("vault_config", {}) if isinstance(data.get("vault_config"), dict) else {}
merged = {**defaults, **nested, **{k: v for k, v in data.items() if k in defaults and v}}
return {
"system_dir": data.get("system_dir", "99_System"),
"resources_dir": data.get("resources_dir", "03_Resources"),
"literature_dir": data.get("literature_dir", "Literature"),
"system_dir": merged["system_dir"],
"resources_dir": merged["resources_dir"],
"literature_dir": merged["literature_dir"],
"control_dir": merged["control_dir"],
"base_dir": merged["base_dir"],
}
except (json.JSONDecodeError, IOError):
pass
return {"system_dir": "99_System", "resources_dir": "03_Resources", "literature_dir": "Literature"}
return defaults
def pipeline_paths(vault: Path) -> dict[str, Path]:
cfg = load_vault_config(vault)
system_dir = cfg["system_dir"]
resources_dir = cfg["resources_dir"]
literature_dir = cfg["literature_dir"]
control_dir = cfg["control_dir"]
base_dir = cfg["base_dir"]
root = vault / system_dir / "PaperForge"
control_root = vault / resources_dir / "LiteratureControl"
control_root = vault / resources_dir / control_dir
return {
'pipeline': root,
'candidates': root / 'candidates' / 'candidates.json',
@ -168,8 +182,88 @@ def pipeline_paths(vault: Path) -> dict[str, Path]:
'ocr': root / 'ocr',
'ocr_queue': root / 'ocr' / 'ocr-queue.json',
'resources': vault / resources_dir,
'literature': vault / resources_dir / literature_dir,
'bases': vault / base_dir,
}
def load_domain_config(paths: dict[str, Path]) -> dict:
"""Load or create the Lite domain mapping from export JSON files."""
config_path = paths['config']
if config_path.exists():
config = read_json(config_path)
else:
config = {"domains": []}
domains = config.setdefault("domains", [])
known_exports = {str(entry.get("export_file", "")) for entry in domains}
changed = not config_path.exists()
for export_path in sorted(paths['exports'].glob('*.json')):
if export_path.name in known_exports:
continue
domains.append({"domain": export_path.stem, "export_file": export_path.name, "allowed_collections": []})
known_exports.add(export_path.name)
changed = True
if changed:
config_path.parent.mkdir(parents=True, exist_ok=True)
write_json(config_path, config)
return config
def base_markdown_filter(path: Path, vault: Path) -> str:
try:
return str(path.relative_to(vault)).replace('\\', '/')
except ValueError:
return str(path).replace('\\', '/')
def write_base_file(path: Path, folder_filter: str, name: str) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
content = f"""filters:
and:
- file.inFolder("{folder_filter}")
properties:
zotero_key:
displayName: "Zotero Key"
title:
displayName: "Title"
year:
displayName: "Year"
has_pdf:
displayName: "PDF"
do_ocr:
displayName: "OCR"
analyze:
displayName: "Analyze"
ocr_status:
displayName: "OCR Status"
deep_reading_status:
displayName: "Deep Reading"
views:
- type: table
name: "{name}"
order:
- file.name
- title
- year
- has_pdf
- do_ocr
- analyze
- ocr_status
- deep_reading_status
- pdf_path
- fulltext_md_path
"""
path.write_text(content, encoding='utf-8')
def ensure_base_views(vault: Path, paths: dict[str, Path], config: dict) -> None:
paths['bases'].mkdir(parents=True, exist_ok=True)
records_root = paths['library_records']
write_base_file(paths['bases'] / 'PaperForge.base', base_markdown_filter(records_root, vault), 'All Records')
seen_domains = set()
for entry in config.get('domains', []):
domain = str(entry.get('domain', '') or '').strip()
if not domain or domain in seen_domains:
continue
seen_domains.add(domain)
write_base_file(paths['bases'] / f"{slugify_filename(domain)}.base", base_markdown_filter(records_root / domain, vault), domain)
def build_collection_lookup(collections: dict) -> dict:
path_cache = {}
item_paths = {}
@ -596,7 +690,8 @@ def load_control_actions(paths: dict[str, Path]) -> dict[str, dict]:
def run_selection_sync(vault: Path) -> int:
paths = pipeline_paths(vault)
config = read_json(paths['config'])
config = load_domain_config(paths)
ensure_base_views(vault, paths, config)
domain_lookup = {entry['export_file']: entry['domain'] for entry in config['domains']}
written = 0
updated = 0
@ -617,7 +712,7 @@ def run_selection_sync(vault: Path) -> int:
if validated_error:
meta['error'] = validated_error
write_json(meta_path, meta)
note_path = paths['resources'] / 'Literature' / domain / f"{item['key']} - {slugify_filename(item['title'])}.md"
note_path = paths['literature'] / domain / f"{item['key']} - {slugify_filename(item['title'])}.md"
note_text = note_path.read_text(encoding='utf-8') if note_path.exists() else ''
fulltext_md_path = obsidian_wikilink_for_path(vault, meta.get('fulltext_md_path', '') or meta.get('markdown_path', ''))
ocr_status = meta.get('ocr_status', 'pending')
@ -1119,7 +1214,8 @@ def analyze_selected_keys(paths: dict[str, Path]) -> set[str]:
def run_index_refresh(vault: Path) -> int:
paths = pipeline_paths(vault)
config = read_json(paths['config'])
config = load_domain_config(paths)
ensure_base_views(vault, paths, config)
domain_lookup = {entry['export_file']: entry['domain'] for entry in config['domains']}
exports = {}
for export_path in sorted(paths['exports'].glob('*.json')):
@ -1128,7 +1224,7 @@ def run_index_refresh(vault: Path) -> int:
exports[domain] = {row['key']: row for row in export_rows}
selected_keys = None
index_rows = []
lit_root = paths['resources'] / 'Literature'
lit_root = paths['literature']
for export_path in sorted(paths['exports'].glob('*.json')):
domain = domain_lookup.get(export_path.name, export_path.stem)
export_rows = load_export_rows(export_path)
@ -1159,7 +1255,7 @@ def run_index_refresh(vault: Path) -> int:
index_rows.append(entry)
write_json(paths['index'], index_rows)
print(f'index-refresh: wrote {len(index_rows)} index rows')
control_records_dir = paths['resources'] / 'LiteratureControl' / 'library-records'
control_records_dir = paths['library_records']
if control_records_dir.exists():
for domain_dir in control_records_dir.iterdir():
if not domain_dir.is_dir():
@ -1218,7 +1314,11 @@ def ensure_ocr_meta(vault: Path, row: dict) -> dict:
meta.setdefault('page_count', 0)
meta.setdefault('markdown_path', '')
meta.setdefault('json_path', '')
meta.setdefault('assets_path', f'99_System/PaperForge/ocr/{key}/images')
try:
assets_path = str((paths['ocr'] / key / 'images').relative_to(vault)).replace('\\', '/')
except ValueError:
assets_path = str(paths['ocr'] / key / 'images')
meta.setdefault('assets_path', assets_path)
meta.setdefault('fulltext_md_path', '')
meta.setdefault('error', '')
return meta
@ -2314,9 +2414,17 @@ def run_ocr(vault: Path) -> int:
write_json(paths['ocr'] / key / 'meta.json', meta)
changed += 1
continue
with open(queue_row['pdf_path'], 'rb') as file_handle:
response = requests.post(job_url, headers={'Authorization': f'bearer {token}'}, data={'model': model, 'optionalPayload': json.dumps(optional_payload)}, files={'file': file_handle}, timeout=120)
response.raise_for_status()
try:
with open(queue_row['pdf_path'], 'rb') as file_handle:
response = requests.post(job_url, headers={'Authorization': f'bearer {token}'}, data={'model': model, 'optionalPayload': json.dumps(optional_payload)}, files={'file': file_handle}, timeout=120)
response.raise_for_status()
except requests.RequestException as e:
meta['ocr_status'] = 'error'
meta['error'] = f'PaddleOCR request failed: {e}'
queue_row['queue_status'] = 'error'
write_json(paths['ocr'] / key / 'meta.json', meta)
changed += 1
continue
meta['ocr_job_id'] = response.json()['data']['jobId']
meta['ocr_status'] = 'queued'
meta['ocr_started_at'] = datetime.now(timezone.utc).isoformat()
@ -2334,8 +2442,8 @@ def run_ocr(vault: Path) -> int:
return 0
def _resolve_formal_note_path(vault: Path, zotero_key: str, domain: str) -> Path | None:
"""Resolve formal literature note from 03_Resources/Literature by zotero_key."""
lit_root = vault / '03_Resources' / 'Literature'
"""Resolve formal literature note by zotero_key."""
lit_root = pipeline_paths(vault)['literature']
domain_dir = lit_root / domain
if not domain_dir.exists():
return None
@ -2360,7 +2468,8 @@ def run_deep_reading(vault: Path) -> int:
Actual content filling is done via /LD-deep (agent-driven).
"""
paths = pipeline_paths(vault)
config = read_json(paths['config'])
config = load_domain_config(paths)
ensure_base_views(vault, paths, config)
domain_lookup = {entry['export_file']: entry['domain'] for entry in config['domains']}
synced = 0
pending_queue: list[dict] = []
@ -2419,6 +2528,45 @@ def run_deep_reading(vault: Path) -> int:
print(f'deep-reading: synced {synced} records, {len(pending_queue)} pending')
return 0
def run_status(vault: Path) -> int:
"""Print a compact Lite install/runtime status."""
paths = pipeline_paths(vault)
cfg = load_vault_config(vault)
config = load_domain_config(paths)
ensure_base_views(vault, paths, config)
export_files = sorted(paths['exports'].glob('*.json'))
record_count = sum(1 for _ in paths['library_records'].rglob('*.md')) if paths['library_records'].exists() else 0
note_count = sum(1 for _ in paths['literature'].rglob('*.md')) if paths['literature'].exists() else 0
base_count = sum(1 for _ in paths['bases'].glob('*.base')) if paths['bases'].exists() else 0
ocr_done = 0
ocr_total = 0
if paths['ocr'].exists():
for meta_path in paths['ocr'].glob('*/meta.json'):
ocr_total += 1
try:
meta = read_json(meta_path)
except Exception:
continue
if str(meta.get('ocr_status', '')).strip().lower() == 'done':
ocr_done += 1
env_paths = [vault / '.env', paths['pipeline'] / '.env']
env_found = [str(path.relative_to(vault)).replace('\\', '/') for path in env_paths if path.exists()]
print('PaperForge Lite status')
print(f"- vault: {vault}")
print(f"- system_dir: {cfg['system_dir']}")
print(f"- resources_dir: {cfg['resources_dir']}")
print(f"- literature_dir: {cfg['literature_dir']}")
print(f"- control_dir: {cfg['control_dir']}")
print(f"- exports: {len(export_files)} JSON file(s)")
print(f"- domains: {len(config.get('domains', []))}")
print(f"- library_records: {record_count}")
print(f"- formal_notes: {note_count}")
print(f"- bases: {base_count}")
print(f"- ocr: {ocr_done}/{ocr_total} done")
print(f"- env: {', '.join(env_found) if env_found else 'not configured'}")
return 0
# =============================================================================
# Update 功能
# =============================================================================
@ -2426,17 +2574,24 @@ def run_deep_reading(vault: Path) -> int:
GITHUB_REPO = "LLLin000/PaperForge"
GITHUB_ZIP = f"https://github.com/{GITHUB_REPO}/archive/refs/heads/master.zip"
PROTECTED_PATHS = {
"03_Resources", "05_Bases",
"99_System/PaperForge/ocr",
"99_System/PaperForge/exports",
"99_System/PaperForge/indexes",
"99_System/PaperForge/candidates",
".env", "AGENTS.md",
}
UPDATEABLE_PATHS = ["skills", "pipeline", "command", "scripts"]
def protected_paths(vault: Path) -> set[str]:
cfg = load_vault_config(vault)
pf = f"{cfg['system_dir']}/PaperForge"
return {
cfg["resources_dir"],
cfg["base_dir"],
f"{pf}/ocr",
f"{pf}/exports",
f"{pf}/indexes",
f"{pf}/candidates",
".env",
"AGENTS.md",
}
def _color(text: str, c: str = "") -> str:
colors = {"r": "\033[91m", "g": "\033[92m", "y": "\033[93m", "b": "\033[94m", "c": "\033[96m", "x": "\033[0m"}
if sys.platform == "win32" and not os.environ.get("FORCE_COLOR"):
@ -2463,6 +2618,7 @@ def _remote_version() -> str | None:
def _scan_updates(vault: Path, source: Path) -> list[tuple[Path, Path, str]]:
updates = []
protected = protected_paths(vault)
for name in UPDATEABLE_PATHS:
src_dir = source / name
if not src_dir.exists():
@ -2473,7 +2629,7 @@ def _scan_updates(vault: Path, source: Path) -> list[tuple[Path, Path, str]]:
rel = src.relative_to(source)
dst = vault / rel
rel_str = str(rel).replace("\\", "/")
if any(rel_str.startswith(p) for p in PROTECTED_PATHS):
if any(rel_str.startswith(p) for p in protected):
continue
if dst.exists():
if hashlib.sha256(src.read_bytes()).hexdigest() != hashlib.sha256(dst.read_bytes()).hexdigest():
@ -2628,11 +2784,12 @@ def main() -> int:
parser.add_argument('--limit', type=int, default=8)
parser.add_argument('--sources', nargs='+')
parser.add_argument('--skip-ingest', action='store_true')
parser.add_argument('worker', choices=['selection-sync', 'index-refresh', 'ocr', 'deep-reading', 'update', 'wizard', 'all'])
parser.add_argument('worker', choices=['selection-sync', 'index-refresh', 'ocr', 'deep-reading', 'status', 'update', 'wizard', 'all'])
args = parser.parse_args()
# Load .env from vault root first, then from PaperForge directory
paths = pipeline_paths(args.vault)
# Load .env from vault root first, then from the configured PaperForge directory.
load_simple_env(args.vault / '.env')
load_simple_env(args.vault / '99_System' / 'PaperForge' / '.env')
load_simple_env(paths['pipeline'] / '.env')
if args.worker == 'selection-sync':
return run_selection_sync(args.vault)
if args.worker == 'index-refresh':
@ -2641,6 +2798,8 @@ def main() -> int:
return run_ocr(args.vault)
if args.worker == 'deep-reading':
return run_deep_reading(args.vault)
if args.worker == 'status':
return run_status(args.vault)
if args.worker == 'update':
return run_update(args.vault)
if args.worker == 'wizard':
@ -2661,4 +2820,4 @@ def main() -> int:
return code
return run_deep_reading(args.vault)
if __name__ == '__main__':
raise SystemExit(main())
raise SystemExit(main())

File diff suppressed because it is too large Load diff

View file

@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Validate literature workflow setup."""
"""Validate a PaperForge Lite installation."""
from __future__ import annotations
@ -9,220 +9,108 @@ import sys
from pathlib import Path
def check_path(label: str, path: str | Path, expected_type: str = "file") -> tuple[bool, str]:
"""Check if a path exists and is the expected type."""
p = Path(path)
if not p.exists():
def check_path(label: str, path: Path, expected_type: str = "file") -> tuple[bool, str]:
if not path.exists():
return False, f"[FAIL] {label}: not found at {path}"
if expected_type == "file" and not p.is_file():
if expected_type == "file" and not path.is_file():
return False, f"[FAIL] {label}: exists but is not a file ({path})"
if expected_type == "dir" and not p.is_dir():
if expected_type == "dir" and not path.is_dir():
return False, f"[FAIL] {label}: exists but is not a directory ({path})"
return True, f"[OK] {label}: {path}"
def validate_zotero(zotero_path: str) -> list[tuple[bool, str]]:
"""Validate Zotero installation and data directory."""
results = []
zotero_dir = Path(zotero_path)
# Check main directory
ok, msg = check_path("Zotero data directory", zotero_path, "dir")
results.append((ok, msg))
if not ok:
return results
# Check zotero.sqlite
sqlite_path = zotero_dir / "zotero.sqlite"
ok, msg = check_path("zotero.sqlite", sqlite_path, "file")
results.append((ok, msg))
# Check better-bibtex JSON (optional but recommended)
bbt_path = zotero_dir / "better-bibtex.json"
if bbt_path.exists():
results.append((True, f"[OK] Better BibTeX config: {bbt_path}"))
else:
results.append((False, f"[WARN] Better BibTeX config not found (optional but recommended): {bbt_path}"))
return results
def validate_obsidian(vault_path: str) -> list[tuple[bool, str]]:
"""Validate Obsidian vault structure."""
results = []
vault_dir = Path(vault_path)
ok, msg = check_path("Obsidian vault", vault_path, "dir")
results.append((ok, msg))
if not ok:
return results
# Check expected subdirectories
expected_dirs = [
"00_Inbox",
"01_Projects",
"02_Areas",
"03_Resources",
"04_Archives",
"05_Bases",
"99_System",
]
for subdir in expected_dirs:
subpath = vault_dir / subdir
ok, msg = check_path(f"Vault subdirectory: {subdir}", subpath, "dir")
results.append((ok, msg))
return results
def validate_ocr_pipeline(vault_path: str) -> list[tuple[bool, str]]:
"""Validate OCR pipeline configuration."""
results = []
vault_dir = Path(vault_path)
# Check .env file
env_path = vault_dir / ".env"
ok, msg = check_path("Environment config (.env)", env_path, "file")
results.append((ok, msg))
if ok:
# Check if PADDLEOCR_API_TOKEN is set
env_content = env_path.read_text(encoding='utf-8')
if "PADDLEOCR_API_TOKEN" in env_content:
results.append((True, "[OK] PADDLEOCR_API_TOKEN configured in .env"))
else:
results.append((False, "[FAIL] PADDLEOCR_API_TOKEN not found in .env"))
# Check OCR pipeline directory
ocr_dir = vault_dir / "99_System" / "LiteraturePipeline" / "worker"
ok, msg = check_path("OCR pipeline directory", ocr_dir, "dir")
results.append((ok, msg))
# Check worker script
worker_path = ocr_dir / "scripts" / "literature_pipeline.py"
ok, msg = check_path("OCR worker script", worker_path, "file")
results.append((ok, msg))
return results
def validate_ld_deep(vault_path: str) -> list[tuple[bool, str]]:
"""Validate /LD-deep command setup."""
results = []
vault_dir = Path(vault_path)
# Check ld_deep.py script
ld_deep_path = vault_dir / ".opencode" / "skills" / "literature-qa" / "scripts" / "ld_deep.py"
ok, msg = check_path("LD-deep script", ld_deep_path, "file")
results.append((ok, msg))
# Check prompt
prompt_path = vault_dir / ".opencode" / "skills" / "literature-qa" / "prompt_deep_subagent.md"
ok, msg = check_path("LD-deep subagent prompt", prompt_path, "file")
results.append((ok, msg))
# Check chart reading guides
if ok:
# Count guide files
guide_files = list(chart_guide_dir.glob("*.md"))
if len(guide_files) >= 14:
results.append((True, f"[OK] Found {len(guide_files)} chart reading guides"))
else:
results.append((False, f"[WARN] Only {len(guide_files)} chart guides found (expected 14+)"))
return results
def load_config(vault: Path) -> dict:
defaults = {
"system_dir": "99_System",
"resources_dir": "03_Resources",
"literature_dir": "Literature",
"control_dir": "LiteratureControl",
"base_dir": "05_Bases",
"skill_dir": ".opencode/skills",
}
config_path = vault / "paperforge.json"
if not config_path.exists():
return defaults
try:
data = json.loads(config_path.read_text(encoding="utf-8"))
except Exception:
return defaults
nested = data.get("vault_config", {}) if isinstance(data.get("vault_config"), dict) else {}
return {**defaults, **nested, **{k: v for k, v in data.items() if k in defaults and v}}
def validate_python_deps() -> list[tuple[bool, str]]:
"""Check Python dependencies."""
results = []
required = {
"requests": "requests",
"pymupdf": "fitz",
"pillow": "PIL",
"pytest": "pytest",
}
required = {"requests": "requests", "pymupdf": "fitz", "pillow": "PIL", "textual": "textual"}
for package, import_name in required.items():
try:
__import__(import_name)
results.append((True, f"[OK] Python package installed: {package}"))
except ImportError:
results.append((False, f"[FAIL] Python package missing: {package} (pip install {package})"))
results.append((False, f"[FAIL] Python package missing: {package}"))
return results
def main():
"""Run all validation checks."""
print("=" * 60)
print("Literature Workflow Setup Validation")
print("=" * 60)
# Load config from .env or config.json
vault_path = os.environ.get("VAULT_PATH", ".")
zotero_path = os.environ.get("ZOTERO_PATH", "")
config_path = Path(vault_path) / "config.json"
if config_path.exists():
config = json.loads(config_path.read_text(encoding="utf-8"))
vault_path = config.get("vault_path", vault_path)
zotero_path = config.get("zotero_path", zotero_path)
def validate_vault(vault: Path, cfg: dict) -> list[tuple[bool, str]]:
results = [check_path("Obsidian vault", vault, "dir")]
if not results[0][0]:
return results
pf_root = vault / cfg["system_dir"] / "PaperForge"
control_root = vault / cfg["resources_dir"] / cfg["control_dir"] / "library-records"
literature_root = vault / cfg["resources_dir"] / cfg["literature_dir"]
base_root = vault / cfg["base_dir"]
for label, path, kind in [
("paperforge.json", vault / "paperforge.json", "file"),
("PaperForge root", pf_root, "dir"),
("exports directory", pf_root / "exports", "dir"),
("ocr directory", pf_root / "ocr", "dir"),
("domain config", pf_root / "config" / "domain-collections.json", "file"),
("worker script", pf_root / "worker" / "scripts" / "literature_pipeline.py", "file"),
("library records directory", control_root, "dir"),
("literature notes directory", literature_root, "dir"),
("Base directory", base_root, "dir"),
]:
results.append(check_path(label, path, kind))
env_path = pf_root / ".env"
ok, msg = check_path("PaperForge .env", env_path, "file")
results.append((ok, msg))
if ok:
env_text = env_path.read_text(encoding="utf-8")
for key in ("PADDLEOCR_API_TOKEN", "PADDLEOCR_JOB_URL"):
results.append((key in env_text, f"[{'OK' if key in env_text else 'FAIL'}] {key} configured"))
return results
def validate_agent(vault: Path, cfg: dict) -> list[tuple[bool, str]]:
skill_root = vault / cfg["skill_dir"] / "literature-qa"
results = [
check_path("LD-deep script", skill_root / "scripts" / "ld_deep.py", "file"),
check_path("LD-deep subagent prompt", skill_root / "prompt_deep_subagent.md", "file"),
check_path("chart-reading directory", skill_root / "chart-reading", "dir"),
]
chart_dir = skill_root / "chart-reading"
if chart_dir.exists():
guide_count = len(list(chart_dir.glob("*.md")))
results.append((guide_count >= 14, f"[{'OK' if guide_count >= 14 else 'WARN'}] chart-reading guides: {guide_count}"))
return results
def main() -> int:
vault = Path(os.environ.get("VAULT_PATH", ".")).resolve()
cfg = load_config(vault)
all_results = []
# Python dependencies
print("\n## Python Dependencies")
all_results.extend(validate_python_deps())
# Zotero
if zotero_path:
print("\n## Zotero Integration")
all_results.extend(validate_zotero(zotero_path))
else:
all_results.append((False, "[SKIP] Zotero path not configured"))
# Obsidian vault
print("\n## Obsidian Vault")
all_results.extend(validate_obsidian(vault_path))
# OCR pipeline
print("\n## OCR Pipeline")
all_results.extend(validate_ocr_pipeline(vault_path))
# LD-deep
print("\n## LD-deep Commands")
all_results.extend(validate_ld_deep(vault_path))
# Summary
print("\n" + "=" * 60)
all_results.extend(validate_vault(vault, cfg))
all_results.extend(validate_agent(vault, cfg))
passed = sum(1 for ok, _ in all_results if ok)
failed = sum(1 for ok, _ in all_results if not ok)
warnings = sum(1 for ok, msg in all_results if not ok and msg.startswith("[WARN]"))
print(f"Results: {passed} passed, {failed} failed ({warnings} warnings)")
print("=" * 60)
# Print failures
failures = [msg for ok, msg in all_results if not ok and not msg.startswith("[WARN]")]
if failures:
print("\n### Failures to fix:")
for msg in failures:
print(f" - {msg}")
warnings_list = [msg for ok, msg in all_results if not ok and msg.startswith("[WARN]")]
if warnings_list:
print("\n### Warnings (optional):")
for msg in warnings_list:
print(f" - {msg}")
return 0 if not failures else 1
failed = [msg for ok, msg in all_results if not ok and not msg.startswith("[WARN]")]
warnings = [msg for ok, msg in all_results if not ok and msg.startswith("[WARN]")]
print(f"PaperForge Lite validation: {passed} passed, {len(failed)} failed, {len(warnings)} warnings")
for _, msg in all_results:
print(msg)
return 0 if not failed else 1
if __name__ == "__main__":
sys.exit(main())
raise SystemExit(main())

View file

@ -16,10 +16,14 @@ import platform
import subprocess
import sys
import webbrowser
import winreg
from pathlib import Path
from typing import Optional
if sys.platform == "win32":
import winreg
else:
winreg = None
from textual.app import App, ComposeResult
from textual.containers import Container, Grid, Horizontal, Vertical
from textual.message import Message
@ -43,7 +47,7 @@ from textual.widgets import (
# =============================================================================
AGENT_CONFIGS = {
"opencode": {"name": "OpenCode", "skill_dir": ".opencode/skills", "config_file": None},
"opencode": {"name": "OpenCode", "skill_dir": ".opencode/skills", "command_dir": ".opencode/command", "config_file": None},
"cursor": {"name": "Cursor", "skill_dir": ".cursor/skills", "config_file": ".cursor/settings.json"},
"claude": {"name": "Claude Code", "skill_dir": ".claude/skills", "config_file": ".claude/skills.json"},
"windsurf": {"name": "Windsurf", "skill_dir": ".windsurf/skills", "config_file": None},
@ -370,7 +374,7 @@ class WelcomeStep(StepScreen):
def compose(self) -> ComposeResult:
yield from super().compose()
yield Static("""
yield Static(r"""
______ ___ ______ _________________ ___________ _____ _____
| ___ \/ _ \ | ___ \ ___| ___ \ ___| _ | ___ \ __ \| ___|
| |_/ / /_\ \| |_/ / |__ | |_/ / |_ | | | | |_/ / | \/| |__
@ -516,6 +520,8 @@ Obsidian Vault/
yield Input(value="Literature", id="input-literature-dir")
yield Static("文献索引文件夹名称:", classes="step-title")
yield Input(value="LiteratureControl", id="input-control-dir")
yield Static("Obsidian Base 文件夹名称:", classes="step-title")
yield Input(value="05_Bases", id="input-base-dir")
yield Horizontal(
Button("✓ 确认并创建目录", id="btn-setup-vault", variant="primary"),
id="btn-row",
@ -545,6 +551,7 @@ Obsidian Vault/
resources_dir = self.query_one("#input-resources-dir", Input).value.strip() or "03_Resources"
literature_dir = self.query_one("#input-literature-dir", Input).value.strip() or "Literature"
control_dir = self.query_one("#input-control-dir", Input).value.strip() or "LiteratureControl"
base_dir = self.query_one("#input-base-dir", Input).value.strip() or "05_Bases"
# 解析路径(支持名称、相对路径、绝对路径)
def resolve_path(base: Path, path_str: str) -> Path:
@ -556,6 +563,7 @@ Obsidian Vault/
system_path = resolve_path(vault_path, system_dir)
resources_path = resolve_path(vault_path, resources_dir)
base_path = resolve_path(vault_path, base_dir)
# 保存配置
self.app.vault_config = {
@ -564,13 +572,16 @@ Obsidian Vault/
"resources_dir": resources_dir,
"literature_dir": literature_dir,
"control_dir": control_dir,
"base_dir": base_dir,
"paperforge_path": str(system_path / "PaperForge"),
"literature_path": str(resources_path / literature_dir),
"base_path": str(base_path),
}
# 创建目录
dirs_to_create = [
resources_path / control_dir / "library-records",
base_path,
system_path / "PaperForge" / "exports",
system_path / "PaperForge" / "ocr",
]
@ -671,6 +682,7 @@ class ZoteroStep(StepScreen):
if is_inside_vault:
# 在 Vault 内部,直接通过
self.app.zotero_data_dir = str(zotero_data)
self.set_status("Zotero 数据目录已确认", True)
self.app.post_message(StepPassed(self.step_idx))
return
@ -698,6 +710,8 @@ class ZoteroStep(StepScreen):
)
else:
junction_path.symlink_to(zotero_data, target_is_directory=True)
self.app.zotero_data_dir = str(zotero_data)
self.app.zotero_link = str(junction_path)
self.set_status("链接已创建", True)
self.app.post_message(StepPassed(self.step_idx))
except Exception as e:
@ -818,8 +832,8 @@ class DeployStep(StepScreen):
yield Input(placeholder="粘贴你的 PaddleOCR API Key", id="input-api-key")
yield Static("PaddleOCR API URL:", classes="step-title")
yield Input(
value="https://api.paddleocr.com/ocr",
placeholder="https://api.paddleocr.com/ocr",
value="https://paddleocr.aistudio-app.com/api/v2/ocr/jobs",
placeholder="https://paddleocr.aistudio-app.com/api/v2/ocr/jobs",
id="input-api-url",
)
yield Horizontal(
@ -866,6 +880,34 @@ class DeployStep(StepScreen):
vault_config = getattr(self.app, 'vault_config', {})
system_dir = vault_config.get('system_dir', '99_System')
resources_dir = vault_config.get('resources_dir', '03_Resources')
literature_dir = vault_config.get('literature_dir', 'Literature')
control_dir = vault_config.get('control_dir', 'LiteratureControl')
base_dir = vault_config.get('base_dir', '05_Bases')
def apply_user_paths(text: str, skill_dir_value: str = "") -> str:
agent_config_dir = str(Path(skill_dir_value or ".opencode/skills").parent).replace("\\", "/")
replacements = {
"<system_dir>": system_dir,
"<resources_dir>": resources_dir,
"<literature_dir>": literature_dir,
"<control_dir>": control_dir,
"<base_dir>": base_dir,
"<skill_dir>": skill_dir_value,
"<agent_config_dir>": agent_config_dir,
"99_System/PaperForge": f"{system_dir}/PaperForge",
"99_System\\PaperForge": f"{system_dir}\\PaperForge",
"99_System/Zotero": f"{system_dir}/Zotero",
"99_System\\Zotero": f"{system_dir}\\Zotero",
"03_Resources/LiteratureControl": f"{resources_dir}/{control_dir}",
"03_Resources\\LiteratureControl": f"{resources_dir}\\{control_dir}",
"03_Resources/Literature": f"{resources_dir}/{literature_dir}",
"03_Resources\\Literature": f"{resources_dir}\\{literature_dir}",
".opencode/skills": skill_dir_value or ".opencode/skills",
".opencode\\skills": (skill_dir_value or ".opencode/skills").replace("/", "\\"),
}
for old, new in replacements.items():
text = text.replace(old, new)
return text
# 1. 获取 agent 配置
agent_config = getattr(self.app, 'agent_config', None)
@ -892,8 +934,11 @@ class DeployStep(StepScreen):
dirs = [
pf_path / "exports",
pf_path / "ocr",
pf_path / "config",
pf_path / "worker/scripts",
vault / resources_dir / "LiteratureControl" / "library-records",
vault / resources_dir / literature_dir,
vault / resources_dir / control_dir / "library-records",
vault / base_dir,
vault / skill_dir / "literature-qa/scripts",
vault / skill_dir / "literature-qa/chart-reading",
]
@ -920,6 +965,15 @@ class DeployStep(StepScreen):
else:
self.set_status(f"错误:找不到 ld_deep.py: {ld_src}", False)
return False
# Copy subagent prompt
prompt_src = repo_root / "skills/literature-qa/prompt_deep_subagent.md"
prompt_dst = vault / skill_dir / "literature-qa/prompt_deep_subagent.md"
if prompt_src.exists():
shutil.copy2(prompt_src, prompt_dst)
else:
self.set_status(f"错误:找不到 prompt_deep_subagent.md: {prompt_src}", False)
return False
# Copy chart-reading guides
chart_src = repo_root / "skills/literature-qa/chart-reading"
@ -927,11 +981,29 @@ class DeployStep(StepScreen):
if chart_src.exists() and chart_src.is_dir():
for f in chart_src.glob("*.md"):
shutil.copy2(f, chart_dst / f.name)
# Copy OpenCode command files when the target platform supports them.
if getattr(self.app, 'agent_key', '') == 'opencode':
command_src = repo_root / "command"
command_dst = vault / agent_config.get("command_dir", ".opencode/command")
if command_src.exists() and command_src.is_dir():
command_dst.mkdir(parents=True, exist_ok=True)
for f in command_src.glob("*.md"):
text = apply_user_paths(f.read_text(encoding="utf-8"), skill_dir)
(command_dst / f.name).write_text(text, encoding="utf-8")
# Copy user-facing docs. AGENTS.md is regenerated below with the chosen paths.
docs_src = repo_root / "docs"
docs_dst = vault / "docs"
if docs_src.exists() and docs_src.is_dir():
shutil.copytree(docs_src, docs_dst, dirs_exist_ok=True)
for doc in docs_dst.rglob("*.md"):
doc.write_text(apply_user_paths(doc.read_text(encoding="utf-8"), skill_dir), encoding="utf-8")
# 5. 创建 .env放到 PaperForge 目录下)
from textual.widgets import Input
api_key = self.query_one("#input-api-key", Input).value.strip()
api_url = self.query_one("#input-api-url", Input).value.strip() or "https://api.paddleocr.com/ocr"
api_url = self.query_one("#input-api-url", Input).value.strip() or "https://paddleocr.aistudio-app.com/api/v2/ocr/jobs"
if not api_key:
self.set_status("请填写 PaddleOCR API Key", False)
@ -948,28 +1020,72 @@ PADDLEOCR_JOB_URL={api_url}
# PaddleOCR 模型(通常不需要修改)
PADDLEOCR_MODEL=PaddleOCR-VL-1.5
# Zotero data directory selected during setup
ZOTERO_DATA_DIR={getattr(self.app, 'zotero_data_dir', '')}
"""
env_path.write_text(env_content, encoding="utf-8")
# Create a minimal domain mapping. The worker will keep this usable even
# before JSON exports exist, and will infer domains from export filenames.
domain_config = pf_path / "config" / "domain-collections.json"
if not domain_config.exists():
export_domains = [
{"domain": f.stem, "export_file": f.name, "allowed_collections": []}
for f in sorted((pf_path / "exports").glob("*.json"))
]
domain_config.write_text(
json.dumps({"domains": export_domains}, indent=2, ensure_ascii=False),
encoding="utf-8",
)
# 6. 创建 paperforge.json包含用户自定义路径
pf_json = vault / "paperforge.json"
if not pf_json.exists():
import json
pf_json.write_text(json.dumps({
"version": "1.0.0",
"agent_platform": agent_config.get('name', 'OpenCode'),
"skill_dir": skill_dir,
existing_config = {}
if pf_json.exists():
try:
existing_config = json.loads(pf_json.read_text(encoding="utf-8"))
except Exception:
existing_config = {}
existing_config.update({
"version": existing_config.get("version", "1.2.0"),
"agent_platform": agent_config.get('name', 'OpenCode'),
"agent_key": getattr(self.app, 'agent_key', 'opencode'),
"skill_dir": skill_dir,
"command_dir": agent_config.get("command_dir", ""),
"system_dir": system_dir,
"resources_dir": resources_dir,
"literature_dir": literature_dir,
"control_dir": control_dir,
"base_dir": base_dir,
"paperforge_path": f"{system_dir}/PaperForge",
"zotero_data_dir": getattr(self.app, 'zotero_data_dir', ''),
"zotero_link": getattr(self.app, 'zotero_link', f"{system_dir}/Zotero"),
"vault_config": {
"system_dir": system_dir,
"resources_dir": resources_dir,
"paperforge_path": f"{system_dir}/PaperForge",
}, indent=2, ensure_ascii=False), encoding="utf-8")
"literature_dir": literature_dir,
"control_dir": control_dir,
"base_dir": base_dir,
},
})
pf_json.write_text(json.dumps(existing_config, indent=2, ensure_ascii=False), encoding="utf-8")
agents_src = repo_root / "AGENTS.md"
agents_dst = vault / "AGENTS.md"
if agents_src.exists():
agents_text = apply_user_paths(agents_src.read_text(encoding="utf-8"), skill_dir)
agents_dst.write_text(agents_text, encoding="utf-8")
# 7. 验证文件完整性
self.set_status("验证文件完整性...", None)
checks = {
"Worker 脚本": worker_dst.exists(),
"精读脚本": ld_dst.exists(),
"目录结构": (vault / resources_dir / "LiteratureControl" / "library-records").exists(),
"精读提示词": prompt_dst.exists(),
"目录结构": (vault / resources_dir / control_dir / "library-records").exists(),
"Base 目录": (vault / base_dir).exists(),
"分类配置": domain_config.exists(),
"导出目录": (pf_path / "exports").exists(),
"OCR 目录": (pf_path / "ocr").exists(),
}
@ -1003,7 +1119,10 @@ class DoneStep(StepScreen):
def compose(self) -> ComposeResult:
yield from super().compose()
yield Markdown("""
vault_config = getattr(self.app, 'vault_config', {})
system_dir = vault_config.get('system_dir', '99_System')
worker_cmd = f"python {system_dir}/PaperForge/worker/scripts/literature_pipeline.py --vault ."
yield Markdown(f"""
## 安装完成!
所有前置条件已满足你现在可以开始使用 PaperForge Lite
@ -1012,12 +1131,12 @@ class DoneStep(StepScreen):
**1. 同步 Zotero 文献**
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . selection-sync
{worker_cmd} selection-sync
```
**2. 生成正式笔记**
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . index-refresh
{worker_cmd} index-refresh
```
**3. 标记精读文献**
@ -1027,7 +1146,7 @@ python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . inde
**4. 运行 OCR**
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . ocr
{worker_cmd} ocr
```
**5. 执行精读**
@ -1056,7 +1175,7 @@ python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . ocr
**Python 脚本命令备用**
```bash
python 99_System/PaperForge/worker/scripts/literature_pipeline.py --vault . <command>
{worker_cmd} <command>
```
| 命令 | 作用 |
|------|------|

View file

@ -7,8 +7,8 @@
## 输入变量(由主 agent 填入)
- `{{ZOTERO_KEY}}` — Zotero key`Y5KQ4JQ7`
- `{{VAULT}}` — Vault 根路径,如 `D:\L\Med\Research`
- `{{SCRIPT}}``D:\L\Med\Research\.opencode\skills\literature-qa\scripts\ld_deep.py`
- `{{VAULT}}` — Vault 根路径
- `{{SCRIPT}}``<Vault>/<skill_dir>/literature-qa/scripts/ld_deep.py`
## 正确流程(必须按顺序执行)
@ -36,8 +36,8 @@ python {{SCRIPT}} prepare {{ZOTERO_KEY}} --vault "{{VAULT}}" --format text
- 这说明该论文已经精读过。如果用户要求重新精读,通知主 agent 确认是否覆盖。
**prepare 成功后会自动生成以下文件**
- `99_System/PaperForge/ocr/{{ZOTERO_KEY}}/figure-map.json` — 图表清单
- `99_System/PaperForge/ocr/{{ZOTERO_KEY}}/chart-type-map.json` — 图表类型与推荐指南
- `<system_dir>/PaperForge/ocr/{{ZOTERO_KEY}}/figure-map.json` — 图表清单
- `<system_dir>/PaperForge/ocr/{{ZOTERO_KEY}}/chart-type-map.json` — 图表类型与推荐指南
**Agent 需要读取 chart-type-map.json**,为每张 figure 建立 `chart_types` 备忘列表。在 Pass 2 解析该 figure 时,根据其 chart_types 读取 `{{CHART_READING_DIR}}` 下对应的 chart-reading 指南,将关键审查问题整合进"图表质量审查"段落。
@ -295,3 +295,4 @@ python {{SCRIPT}} validate-note "<note_path>" --fulltext "<fulltext_path>"
# 列出待精读队列(信息用)
python {{SCRIPT}} queue --vault "{{VAULT}}" --format table
```

View file

@ -11,6 +11,41 @@ from pathlib import Path
from typing import Iterable
def _load_vault_config(vault: Path) -> dict:
"""Load vault directory configuration from paperforge.json."""
defaults = {
"system_dir": "99_System",
"resources_dir": "03_Resources",
"literature_dir": "Literature",
"control_dir": "LiteratureControl",
"base_dir": "05_Bases",
}
config_path = vault / "paperforge.json"
if config_path.exists():
try:
data = json.loads(config_path.read_text(encoding="utf-8"))
nested = data.get("vault_config", {}) if isinstance(data.get("vault_config"), dict) else {}
return {**defaults, **nested, **{k: v for k, v in data.items() if k in defaults and v}}
except Exception:
pass
return defaults
def _paperforge_paths(vault: Path) -> dict[str, Path]:
config = _load_vault_config(vault)
resources = vault / config["resources_dir"]
return {
"ocr": vault / config["system_dir"] / "PaperForge" / "ocr",
"records": resources / config["control_dir"] / "library-records",
"literature": resources / config["literature_dir"],
}
def _get_ocr_root(vault: Path) -> Path:
"""Get OCR root path dynamically from config."""
return _paperforge_paths(vault)["ocr"]
STUDY_HEADER = "## 🔍 精读"
FIGURE_SECTION_HEADER = "#### Figure-by-Figure 解析"
TABLE_SECTION_HEADER = "#### Table-by-Table 解析"
@ -542,8 +577,8 @@ def build_figure_map(fulltext: str, zotero_key: str = "") -> dict:
def find_note_by_zotero_key(workspace_root: Path, zotero_key: str) -> Path | None:
"""Resolve the formal literature note from 03_Resources/Literature."""
literature_root = workspace_root / "03_Resources" / "Literature"
"""Resolve the formal literature note from the configured literature directory."""
literature_root = _paperforge_paths(workspace_root)["literature"]
if not literature_root.exists():
return None
@ -953,9 +988,10 @@ def prepare_deep_reading(vault: Path, zotero_key: str, force: bool = False) -> d
"chart_recommendations": [],
}
records_root = vault / "03_Resources" / "LiteratureControl" / "library-records"
literature_root = vault / "03_Resources" / "Literature"
ocr_root = vault / "99_System" / "LiteraturePipeline" / "ocr"
paths = _paperforge_paths(vault)
records_root = paths["records"]
literature_root = paths["literature"]
ocr_root = paths["ocr"]
# 1. Find library-record
record_path: Path | None = None
@ -1116,8 +1152,9 @@ def scan_deep_reading_queue(vault: Path) -> list[dict]:
Returns a list of dicts with keys:
- zotero_key, title, domain, analyze, deep_reading_status, ocr_status
"""
records_root = vault / "03_Resources" / "LiteratureControl" / "library-records"
ocr_root = vault / "99_System" / "LiteraturePipeline" / "ocr"
paths = _paperforge_paths(vault)
records_root = paths["records"]
ocr_root = paths["ocr"]
queue: list[dict] = []
if not records_root.exists():
return queue

View file

@ -47,16 +47,6 @@ if sys.platform == "win32":
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8")
sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding="utf-8")
# 用户数据保护清单(绝不动这些路径)
PROTECTED_PATHS = {
"03_Resources", "05_Bases",
"99_System/PaperForge/ocr",
"99_System/PaperForge/exports",
"99_System/PaperForge/indexes",
"99_System/PaperForge/candidates",
".env", "AGENTS.md",
}
# 可更新路径(代码文件)
UPDATEABLE_PATHS = ["skills", "pipeline", "command", "scripts"]
@ -83,6 +73,34 @@ def load_json(path: Path) -> dict:
return json.loads(path.read_text(encoding="utf-8")) if path.exists() else {}
def load_vault_config(vault: Path) -> dict:
defaults = {
"system_dir": "99_System",
"resources_dir": "03_Resources",
"literature_dir": "Literature",
"control_dir": "LiteratureControl",
"base_dir": "05_Bases",
}
data = load_json(vault / "paperforge.json")
nested = data.get("vault_config", {}) if isinstance(data.get("vault_config"), dict) else {}
return {**defaults, **nested, **{k: v for k, v in data.items() if k in defaults and v}}
def protected_paths(vault: Path) -> set[str]:
cfg = load_vault_config(vault)
pf = f"{cfg['system_dir']}/PaperForge"
return {
cfg["resources_dir"],
cfg["base_dir"],
f"{pf}/ocr",
f"{pf}/exports",
f"{pf}/indexes",
f"{pf}/candidates",
".env",
"AGENTS.md",
}
def sha256(path: Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest() if path.exists() else ""
@ -137,6 +155,7 @@ def newer(a: str, b: str) -> bool:
def scan_updates(vault: Path, source: Path) -> list[tuple[Path, Path, str]]:
"""扫描需要更新的文件,返回 (src, dst, action) 列表"""
updates = []
protected = protected_paths(vault)
for name in UPDATEABLE_PATHS:
src_dir = source / name
if not src_dir.exists():
@ -147,7 +166,7 @@ def scan_updates(vault: Path, source: Path) -> list[tuple[Path, Path, str]]:
rel = src.relative_to(source)
dst = vault / rel
rel_str = str(rel).replace("\\", "/")
if any(rel_str.startswith(p) for p in PROTECTED_PATHS):
if any(rel_str.startswith(p) for p in protected):
continue
if dst.exists():
if sha256(src) != sha256(dst):