mirror of
https://github.com/lllin000/PaperForge.git
synced 2026-07-22 06:50:53 +00:00
feat: ocr reading-order fixes, single-source fulltext, redo closed-loop, release notes & manual
This commit is contained in:
Research Assistant
parent
b9d51ee40b
commit
3fdc056886
23 changed files with 1757 additions and 114 deletions
59
docs/superpowers/plans/2026-06-01-ocr-redo-single-source.md
Normal file
59
docs/superpowers/plans/2026-06-01-ocr-redo-single-source.md
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
# OCR Redo Single-Source Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Convert OCR redo into an immediate rerun workflow and remove workspace `fulltext.md` as a second truth source.
|
||||
|
||||
**Architecture:** Reuse the existing OCR worker as the execution engine, but add a redo preflight that clears stale outputs, targets selected keys only, and writes `ocr_redo` back to `false` only on success. Route all `fulltext_path` consumers to canonical OCR storage.
|
||||
|
||||
**Tech Stack:** Python workers/CLI, pytest, Obsidian frontmatter paths.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Remove Workspace Fulltext As Truth Source
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/worker/asset_index.py`
|
||||
- Modify: `paperforge/worker/sync.py`
|
||||
- Test: `tests/test_sync.py`
|
||||
|
||||
- [ ] Delete stale workspace `fulltext.md` during migration/index refresh.
|
||||
- [ ] Stop bridging OCR fulltext into workspace.
|
||||
- [ ] Point `entry["fulltext_path"]` at canonical OCR fulltext.
|
||||
- [ ] Update sync test to expect deletion instead of refresh.
|
||||
|
||||
### Task 2: Make OCR Redo Closed-Loop
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/worker/ocr.py`
|
||||
- Modify: `paperforge/commands/ocr.py`
|
||||
- Test: `tests/test_ocr.py`
|
||||
|
||||
- [ ] Add redo preflight to clear old OCR output and stale workspace fulltext.
|
||||
- [ ] Force selected notes to `do_ocr: true`, `ocr_status: pending`, `ocr_redo: true` before rerun.
|
||||
- [ ] Extend `run_ocr()` to accept selected keys.
|
||||
- [ ] Call OCR directly from `ocr redo`.
|
||||
- [ ] After rerun, set `ocr_redo: false` only for successful keys.
|
||||
- [ ] Leave failed/pending keys at `ocr_redo: true`.
|
||||
|
||||
### Task 3: Verify End-to-End Behavior
|
||||
|
||||
**Files:**
|
||||
- Test: `tests/test_ocr.py`
|
||||
|
||||
- [ ] Cover dry-run no-op behavior.
|
||||
- [ ] Cover invalid key rejection.
|
||||
- [ ] Cover success path: delete workspace fulltext, rerun selected key, write back `ocr_redo: false`.
|
||||
- [ ] Cover incomplete rerun path: preserve `ocr_redo: true`.
|
||||
|
||||
### Task 4: Run Regression Suite
|
||||
|
||||
**Files:**
|
||||
- Test: `tests/test_ocr_rendering.py`
|
||||
- Test: `tests/test_ocr_layout_zones.py`
|
||||
- Test: `tests/test_ocr_body_spine.py`
|
||||
- Test: `tests/test_sync.py`
|
||||
- Test: `tests/test_asset_index.py`
|
||||
|
||||
- [ ] Run targeted OCR redo tests first.
|
||||
- [ ] Run full regression suite for OCR/sync/asset index.
|
||||
|
|
@ -0,0 +1,420 @@
|
|||
# Plugin Release Notes & Manual — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add a `更新与手册` settings tab with versioned Chinese release notes and external manual links, plus an auto-popup on version change.
|
||||
|
||||
**Architecture:** Release notes stored as JSON imported by esbuild into settings.ts. New tab renders collapsible version cards. `main.ts` onload compares `manifest.version` against `settings.last_seen_version` and shows a Modal on mismatch.
|
||||
|
||||
**Tech Stack:** TypeScript (Obsidian plugin API), JSON, esbuild bundling
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Release Notes Data File
|
||||
|
||||
**Files:**
|
||||
- Create: `paperforge/plugin/src/release-notes.json`
|
||||
- Modify: `paperforge/plugin/src/constants.ts:70-92`
|
||||
|
||||
- [ ] **Step 1: Create `release-notes.json` with version 1.5.15 entry**
|
||||
|
||||
Write `paperforge/plugin/src/release-notes.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"versions": [
|
||||
{
|
||||
"version": "1.5.15",
|
||||
"date": "2026-06-01",
|
||||
"title": "OCR 重新设计 — 全文单一真相源 + Redo 闭环 + 阅读顺序修复",
|
||||
"breaking_or_migration": [
|
||||
"fulltext 正文现在统一位于 System/PaperForge/ocr/<key>/fulltext.md，工作区 Resources/.../fulltext.md 不再保留为正文副本",
|
||||
"OCR Redo 现在会立即重跑 OCR（闭环执行），不再只是标记状态后需要手动再跑一次"
|
||||
],
|
||||
"new_features": [
|
||||
"新增 OCR Redo 闭环执行：勾选重做后自动删除旧产物 → 强制 do_ocr → 立即重跑 OCR → 成功后写回 ocr_redo:false",
|
||||
"设置页新增「更新与手册」标签页，可随时查看版本更新记录与使用手册链接",
|
||||
"首次更新时自动弹出更新说明"
|
||||
],
|
||||
"fixes": [
|
||||
"修复 OCR 双栏阅读顺序混乱：2.5 / Discussion / Introduction 等章节标题和正文错位",
|
||||
"修复 Kwon et al. 等叙述句被误判为参考文献，导致正文段落被提前截断并插入 Discussion",
|
||||
"修复 Fig.3 / Fig.4 等页首图注和图块被重排逻辑整体后移，导致图文断裂",
|
||||
"修复首页（page 1）Abstract / Introduction 排序错乱，METHODS 行跑到 Introduction 后",
|
||||
"修复并排 panel（如 Fig.2 的左右两块）未合并为单张 composite figure，被拆成两张独立图片",
|
||||
"修复 redo OCR 后工作区 fulltext 未被同步覆盖，导致用户看到的仍是旧乱序版本",
|
||||
"Dashboard 现在可以识别 System/PaperForge/ocr/<key>/fulltext.md 路径，打开全文后能切到对应 paper 视图"
|
||||
],
|
||||
"recommended_actions": [
|
||||
"如果论文是此版本前完成的 OCR，建议对重要论文执行一次「重做OCR」（在 Base 视图勾选 ocr_redo 后点 ribbon Redo 按钮），以修复阅读顺序问题",
|
||||
"打开全文请直接使用 Dashboard 的「打开全文」按钮，正文已经统一迁至 System/PaperForge/ocr/ 下"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add `last_seen_version` to `PaperForgeSettings` interface**
|
||||
|
||||
In `paperforge/plugin/src/constants.ts`, after line 90:
|
||||
|
||||
```typescript
|
||||
last_seen_version: string;
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add default value to `DEFAULT_SETTINGS`**
|
||||
|
||||
In `paperforge/plugin/src/constants.ts`, after line 116:
|
||||
|
||||
```typescript
|
||||
last_seen_version: "",
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add paperforge/plugin/src/release-notes.json paperforge/plugin/src/constants.ts
|
||||
git commit -m "feat: add release-notes data and last_seen_version setting field"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Settings Tab – 更新与手册
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/plugin/src/settings.ts`
|
||||
|
||||
- [ ] **Step 1: Import release notes and add tab to tab bar**
|
||||
|
||||
At top of `settings.ts`, add import:
|
||||
|
||||
```typescript
|
||||
import releaseNotesData from "./release-notes.json";
|
||||
```
|
||||
|
||||
In `display()` method (around line 92-94), add the third tab:
|
||||
|
||||
```typescript
|
||||
const tabs = [
|
||||
{ id: "setup", label: t("tab_setup") || "安装" },
|
||||
{ id: "features", label: t("tab_features") || "功能" },
|
||||
{ id: "release-notes", label: "更新与手册" },
|
||||
];
|
||||
```
|
||||
|
||||
In the render switch (around line 117-121):
|
||||
|
||||
```typescript
|
||||
if (this.activeTab === "setup") {
|
||||
this._renderSetupTab(tabContents.setup);
|
||||
} else if (this.activeTab === "features") {
|
||||
this._renderFeaturesTab(tabContents.features);
|
||||
} else {
|
||||
this._renderReleaseNotesTab(tabContents["release-notes"]);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement `_renderReleaseNotesTab()`**
|
||||
|
||||
Add new method after `_renderFeaturesTab()`:
|
||||
|
||||
```typescript
|
||||
_renderReleaseNotesTab(containerEl: HTMLElement) {
|
||||
containerEl.createEl("h2", { text: "更新与手册" });
|
||||
|
||||
// --- Version Log ---
|
||||
containerEl.createEl("h3", { text: "版本更新记录" });
|
||||
|
||||
const versions = (releaseNotesData as any).versions || [];
|
||||
for (const ver of versions) {
|
||||
const card = containerEl.createEl("div", { cls: "paperforge-release-card" });
|
||||
|
||||
// Header row
|
||||
const header = card.createEl("div", { cls: "paperforge-release-header" });
|
||||
header.createEl("strong", { text: `v${ver.version} — ${ver.title}` });
|
||||
header.createEl("span", { cls: "paperforge-release-date", text: ` (${ver.date})` });
|
||||
|
||||
// Breaking / Migration
|
||||
if (ver.breaking_or_migration && ver.breaking_or_migration.length > 0) {
|
||||
const section = card.createEl("div", { cls: "paperforge-release-section" });
|
||||
section.createEl("div", { cls: "paperforge-release-label", text: "行为变更 / 迁移注意" });
|
||||
for (const item of ver.breaking_or_migration) {
|
||||
section.createEl("div", { cls: "paperforge-release-item", text: `• ${item}` });
|
||||
}
|
||||
}
|
||||
|
||||
// New Features
|
||||
if (ver.new_features && ver.new_features.length > 0) {
|
||||
const section = card.createEl("div", { cls: "paperforge-release-section" });
|
||||
section.createEl("div", { cls: "paperforge-release-label", text: "新功能" });
|
||||
for (const item of ver.new_features) {
|
||||
section.createEl("div", { cls: "paperforge-release-item", text: `• ${item}` });
|
||||
}
|
||||
}
|
||||
|
||||
// Fixes
|
||||
if (ver.fixes && ver.fixes.length > 0) {
|
||||
const section = card.createEl("div", { cls: "paperforge-release-section" });
|
||||
section.createEl("div", { cls: "paperforge-release-label", text: "修复" });
|
||||
for (const item of ver.fixes) {
|
||||
section.createEl("div", { cls: "paperforge-release-item", text: `• ${item}` });
|
||||
}
|
||||
}
|
||||
|
||||
// Recommended Actions
|
||||
if (ver.recommended_actions && ver.recommended_actions.length > 0) {
|
||||
const section = card.createEl("div", { cls: "paperforge-release-section" });
|
||||
section.createEl("div", { cls: "paperforge-release-label", text: "建议操作" });
|
||||
for (const item of ver.recommended_actions) {
|
||||
section.createEl("div", { cls: "paperforge-release-item", text: `• ${item}` });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- Manual Link ---
|
||||
containerEl.createEl("h3", { text: "使用手册" });
|
||||
const manualSection = containerEl.createEl("div", { cls: "paperforge-manual-links" });
|
||||
const manualLink = manualSection.createEl("a", {
|
||||
text: "→ 查看完整使用手册（GitHub）",
|
||||
href: "https://github.com/LLLin000/PaperForge/blob/main/docs/user-manual.md",
|
||||
});
|
||||
manualLink.setAttr("target", "_blank");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add inline CSS for release notes**
|
||||
|
||||
In the CSS block in `display()` (around line 73-87), add:
|
||||
|
||||
```css
|
||||
.paperforge-release-card { border: 1px solid var(--background-modifier-border); border-radius: 6px; padding: 12px; margin-bottom: 12px; }
|
||||
.paperforge-release-header { margin-bottom: 8px; }
|
||||
.paperforge-release-date { color: var(--text-muted); font-size: 12px; }
|
||||
.paperforge-release-section { margin-bottom: 6px; }
|
||||
.paperforge-release-label { font-weight: 600; color: var(--text-accent); margin-bottom: 2px; font-size: 13px; }
|
||||
.paperforge-release-item { font-size: 13px; margin-left: 8px; color: var(--text-muted); }
|
||||
.paperforge-manual-links { margin-top: 8px; }
|
||||
.paperforge-manual-links a { color: var(--text-accent); }
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Build and verify**
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
Check that `main.js` contains the release notes JSON inlined.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add paperforge/plugin/src/settings.ts paperforge/plugin/src/release-notes.json
|
||||
git commit -m "feat: add 更新与手册 settings tab with release notes and manual links"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Auto-Popup on Version Change
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/plugin/src/main.ts`
|
||||
|
||||
- [ ] **Step 1: Add `_checkReleaseNotes()` method in main.ts**
|
||||
|
||||
After `_firstLaunchSnapshotMigration()` method, add:
|
||||
|
||||
```typescript
|
||||
private _checkReleaseNotes() {
|
||||
const currentVersion = this.manifest.version;
|
||||
const seen = this.settings.last_seen_version;
|
||||
if (seen === currentVersion) return;
|
||||
|
||||
const releaseNotesData = require("./release-notes.json");
|
||||
const versions = releaseNotesData.versions || [];
|
||||
const currentEntry = versions.find((v: any) => v.version === currentVersion);
|
||||
|
||||
const { Modal } = require("obsidian");
|
||||
class ReleaseNotesModal extends Modal {
|
||||
constructor(app: any, entry: any) {
|
||||
super(app);
|
||||
this._entry = entry;
|
||||
}
|
||||
private _entry: any;
|
||||
onOpen() {
|
||||
const { contentEl } = this;
|
||||
contentEl.createEl("h2", { text: `PaperForge v${currentVersion} 更新说明` });
|
||||
if (this._entry) {
|
||||
contentEl.createEl("p", { text: this._entry.title, cls: "paperforge-modal-subtitle" });
|
||||
if (this._entry.breaking_or_migration && this._entry.breaking_or_migration.length > 0) {
|
||||
contentEl.createEl("h4", { text: "行为变更 / 迁移注意" });
|
||||
for (const item of this._entry.breaking_or_migration) {
|
||||
contentEl.createEl("p", { text: `• ${item}`, cls: "paperforge-modal-item" });
|
||||
}
|
||||
}
|
||||
if (this._entry.recommended_actions && this._entry.recommended_actions.length > 0) {
|
||||
contentEl.createEl("h4", { text: "建议操作" });
|
||||
for (const item of this._entry.recommended_actions) {
|
||||
contentEl.createEl("p", { text: `• ${item}`, cls: "paperforge-modal-item" });
|
||||
}
|
||||
}
|
||||
} else {
|
||||
contentEl.createEl("p", { text: "请前往设置 → 更新与手册 查看完整更新记录。" });
|
||||
}
|
||||
new (require("obsidian").Setting)(contentEl)
|
||||
.addButton(btn => btn.setButtonText("知道了").setCta().onClick(() => {
|
||||
this.close();
|
||||
}));
|
||||
}
|
||||
onClose() {
|
||||
const { contentEl } = this;
|
||||
contentEl.empty();
|
||||
}
|
||||
}
|
||||
|
||||
new ReleaseNotesModal(this.app, currentEntry).open();
|
||||
this.settings.last_seen_version = currentVersion;
|
||||
this.saveSettings();
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Call `_checkReleaseNotes()` in `onload()`**
|
||||
|
||||
In `onload()`, after `this._firstLaunchSnapshotMigration()`, add:
|
||||
|
||||
```typescript
|
||||
this._checkReleaseNotes();
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Build, deploy, and verify**
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
Copy-Item main.js manifest.json styles.css -Destination ".obsidian/plugins/paperforge/" -Force
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add paperforge/plugin/src/main.ts
|
||||
git commit -m "feat: auto-popup release notes modal on plugin version change"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: User Manual Stub
|
||||
|
||||
**Files:**
|
||||
- Create: `docs/user-manual.md`
|
||||
|
||||
- [ ] **Step 1: Write minimal Chinese user manual**
|
||||
|
||||
Create `docs/user-manual.md`:
|
||||
|
||||
```markdown
|
||||
# PaperForge 使用手册
|
||||
|
||||
> 版本：1.5.15+
|
||||
|
||||
## 快速开始
|
||||
|
||||
1. 安装 PaperForge 插件并完成设置向导
|
||||
2. 在 Zotero 中配置 Better BibTeX 自动导出
|
||||
3. 打开 PaperForge Dashboard，点击 Sync 同步文献
|
||||
4. 对需要全文的论文，确保 do_ocr 为 true，点击 Run OCR
|
||||
5. OCR 完成后即可在 Dashboard 中打开全文、进行精读
|
||||
|
||||
## 核心概念
|
||||
|
||||
### fulltext（全文）
|
||||
|
||||
每篇论文的 OCR 正文统一存放在：
|
||||
`System/PaperForge/ocr/<ZoteroKey>/fulltext.md`
|
||||
|
||||
这是唯一的正文真相源。Dashboard 的「打开全文」按钮会直接定位到此文件。
|
||||
|
||||
### OCR Redo（重做 OCR）
|
||||
|
||||
当你需要重新提取某篇论文的全文时：
|
||||
|
||||
1. 在领域 Base 的「重做OCR」视图中勾选 `ocr_redo`
|
||||
2. 点击 ribbon 上的 Redo OCR 按钮
|
||||
3. 系统会自动：删除旧产物 → 重新上传 PDF → 等待 OCR → 生成新全文
|
||||
|
||||
完成后 `ocr_redo` 自动变为 `false`。如果失败，标志会保持 `true`，可以再次重试。
|
||||
|
||||
### Dashboard 识别
|
||||
|
||||
Dashboard 会自动识别你当前打开的文件所属的论文。支持识别：
|
||||
- 工作区笔记（`Resources/Literature/<domain>/<key> - Title/<key>.md`）
|
||||
- OCR 全文（`System/PaperForge/ocr/<key>/fulltext.md`）
|
||||
- PDF 文件（通过 wikilink 匹配）
|
||||
|
||||
## 常用工作流
|
||||
|
||||
### 新增论文
|
||||
|
||||
1. Zotero 中添加文献 → 自动导出
|
||||
2. Dashboard → Sync
|
||||
3. 在 Base 视图中确认 do_ocr 为 true
|
||||
4. Dashboard → Run OCR
|
||||
5. OCR 完成后 Dashboard 自动显示「打开全文」按钮
|
||||
|
||||
### 精读论文
|
||||
|
||||
1. Dashboard 中找到目标论文 → 点击打开全文
|
||||
2. 在 Agent 中输入 `/pf-deep <key>` 开始精读
|
||||
3. 精读完成后 Dashboard 状态自动更新
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
Vault/
|
||||
├── System/PaperForge/
|
||||
│ ├── ocr/<key>/ ← OCR 全文、图片、原始 JSON
|
||||
│ ├── exports/ ← Zotero 导出
|
||||
│ └── indexes/ ← 文献索引
|
||||
├── Resources/Literature/
|
||||
│ └── <domain>/<key> - Title/ ← 论文工作区
|
||||
│ ├── <key>.md ← 主笔记
|
||||
│ └── ai/ ← AI 产物
|
||||
├── Bases/ ← Base 视图
|
||||
└── .obsidian/plugins/paperforge/ ← 插件
|
||||
```
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/user-manual.md
|
||||
git commit -m "docs: add Chinese user manual stub"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Run Verification
|
||||
|
||||
- [ ] **Step 1: Build plugin**
|
||||
|
||||
```bash
|
||||
cd paperforge/plugin && npm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run Python regression**
|
||||
|
||||
```bash
|
||||
pytest tests/test_sync.py tests/test_ocr.py tests/test_ocr_rendering.py tests/test_ocr_layout_zones.py tests/test_ocr_body_spine.py tests/test_asset_index.py
|
||||
```
|
||||
Expected: all pass
|
||||
|
||||
- [ ] **Step 3: Run plugin vitest**
|
||||
|
||||
```bash
|
||||
cd paperforge/plugin && npx vitest run tests/zotero-path.test.ts
|
||||
```
|
||||
Expected: 2 passed
|
||||
|
||||
- [ ] **Step 4: Commit any fixes from verification**
|
||||
|
||||
```bash
|
||||
git commit -m "chore: post-implementation verification"
|
||||
```
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
# OCR Redo Single-Source Fulltext Design
|
||||
|
||||
> **Status:** Approved
|
||||
> **Date:** 2026-06-01
|
||||
|
||||
## Goal
|
||||
|
||||
Make OCR redo a one-command closed loop and remove the duplicated workspace `fulltext.md` as a truth source.
|
||||
|
||||
## Decisions
|
||||
|
||||
1. Canonical OCR fulltext lives only at `System/PaperForge/ocr/<key>/fulltext.md`.
|
||||
2. Workspace `Resources/.../fulltext.md` is deleted and no longer regenerated.
|
||||
3. `paperforge ocr redo` immediately reruns OCR instead of only resetting state.
|
||||
4. `ocr_redo` is a one-shot trigger:
|
||||
- stays `true` while redo is still pending/failed
|
||||
- flips to `false` only after successful OCR completion
|
||||
|
||||
## Workflow
|
||||
|
||||
For each note with `ocr_redo: true`:
|
||||
|
||||
1. Delete stale workspace `fulltext.md` if present.
|
||||
2. Delete old OCR output directory under `System/PaperForge/ocr/<key>/`.
|
||||
3. Force note frontmatter to:
|
||||
- `do_ocr: true`
|
||||
- `ocr_status: pending`
|
||||
- `fulltext_md_path: ""`
|
||||
- `ocr_redo: true`
|
||||
4. Invoke OCR code immediately for the selected keys only.
|
||||
5. After OCR returns:
|
||||
- success -> `ocr_status: done`, `ocr_redo: false`
|
||||
- pending/failed -> keep `ocr_redo: true`
|
||||
|
||||
## Single-Source Fulltext Contract
|
||||
|
||||
- `asset_index` advertises canonical OCR `fulltext.md` as `fulltext_path`
|
||||
- sync/migration no longer bridges OCR fulltext into workspace
|
||||
- stale workspace copies should be deleted opportunistically when encountered
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No soft links
|
||||
- No workspace proxy `fulltext.md`
|
||||
- No background redo queue separate from the command
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
# Plugin Release Notes & Manual — Design Spec
|
||||
|
||||
> **Status:** Approved
|
||||
> **Date:** 2026-06-01
|
||||
|
||||
## Goal
|
||||
|
||||
Add a settings tab and auto-popup for versioned release notes and user manual access, in Chinese first.
|
||||
|
||||
## Design
|
||||
|
||||
### 1. Release Notes Data
|
||||
|
||||
- `paperforge/plugin/src/release-notes.json`
|
||||
- Array of versioned entries, each with:
|
||||
- `version`, `date`, `title`
|
||||
- `breaking_or_migration` (string[])
|
||||
- `new_features` (string[])
|
||||
- `fixes` (string[])
|
||||
- `recommended_actions` (string[])
|
||||
|
||||
### 2. Settings Tab
|
||||
|
||||
- New tab `更新与手册` alongside existing `安装` and `功能`
|
||||
- Upper section: version log cards, collapsible, newest first
|
||||
- Lower section: links to external docs (`docs/user-manual.md`)
|
||||
|
||||
### 3. Auto-Popup on Version Change
|
||||
|
||||
- Trigger: `plugin.manifest.version !== settings.last_seen_version`
|
||||
- Modal with version title + migration items + recommended actions
|
||||
- `知道了` button writes `last_seen_version` to persistent settings
|
||||
|
||||
### 4. Version Tracking
|
||||
|
||||
- `last_seen_version: string` field added to `PaperForgeSettings`
|
||||
- Compared on plugin `onload()`
|
||||
- Persisted through `saveSettings()`
|
||||
|
||||
### 5. Deployment
|
||||
|
||||
- `release-notes.json` copied to plugin output dir alongside `main.js`
|
||||
- esbuild config updated with a copy step
|
||||
|
||||
### 6. Manual
|
||||
|
||||
- `docs/user-manual.md` in repo
|
||||
- Plugin links to it externally; no inline rendering
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No English release notes in this phase
|
||||
- No inline Markdown rendering for manual
|
||||
- No server-side release notes fetching
|
||||
403
docs/user-manual.md
Normal file
403
docs/user-manual.md
Normal file
|
|
@ -0,0 +1,403 @@
|
|||
# PaperForge 使用手册
|
||||
|
||||
> 版本：1.5.15+
|
||||
|
||||
---
|
||||
|
||||
## 一、PaperForge 是什么
|
||||
|
||||
PaperForge 是一个 Obsidian 插件，把你 Vault 变成文献研究中心。它可以：
|
||||
|
||||
1. 从 Zotero 自动同步文献目录，为每篇论文生成规范的 Obsidian 笔记
|
||||
2. 把 PDF 上传到 PaddleOCR，提取全文 Markdown 和图片
|
||||
3. 在 Dashboard 中查看全文、跟踪论文处理状态、重做 OCR
|
||||
4. 与 AI Agent（OpenCode / Claude Code / Cursor 等）协作，进行深度阅读、文献搜索、方法论提取
|
||||
5. 基于全文建立记忆层（FTS 全文搜索 + 向量语义搜索 + 结构化索引）
|
||||
|
||||
---
|
||||
|
||||
## 二、安装与初始化
|
||||
|
||||
### 2.1 前置条件
|
||||
|
||||
你需要先确保系统中有以下工具：
|
||||
|
||||
- **Zotero 7**（文献管理软件）
|
||||
- **Better BibTeX 插件**（Zotero 的导出插件）
|
||||
- **Python 3.10+**
|
||||
- **PaddleOCR API Token**（百度飞桨 OCR 服务）
|
||||
|
||||
### 2.2 安装 Guide
|
||||
|
||||
1. 下载 PaperForge 插件，将 `main.js`、`manifest.json`、`styles.css` 放入 Vault 的 `.obsidian/plugins/paperforge/` 目录
|
||||
2. 在 Obsidian 的 Community Plugins 中搜索并启用 PaperForge
|
||||
3. 打开 PaperForge 设置：
|
||||
|
||||
在 Obsidian 左侧 Ribbon 点击 PaperForge 图标，或使用命令面板搜索 `PaperForge: Dashboard`。
|
||||
|
||||
#### 步骤 1：安装标签页中配置
|
||||
|
||||
打开设置后默认在「安装」标签页。你会看到：
|
||||
|
||||
- **Runtime 状态**：显示插件版本和 Python 包的版本是否匹配
|
||||
- **Python 解释器**：系统会自动检测。你也可以手动指定路径（如 `C:\Python310\python.exe`）
|
||||
- **PaddleOCR API Key**：填写你从百度飞桨申请的 API Token
|
||||
- **安装向导**：首次安装点击 `开始安装`，会自动检测环境并引导你完成配置
|
||||
|
||||
安装向导会自动帮你：
|
||||
|
||||
- 在 Vault 中创建 PaperForge 目录结构（`System/PaperForge/`、`Resources/Literature/`、`Bases/` 等）
|
||||
- 检测 Python 环境
|
||||
- 检测 Zotero 和 Better BibTeX
|
||||
- 配置 Zotero 数据目录
|
||||
- 选择你的 AI Agent 平台（OpenCode / Claude Code / Cursor 等）
|
||||
- **部署 Agent Skill 文件**：向导会根据你选择的平台，自动将 PaperForge Skill 部署到对应的 Vault 目录
|
||||
|
||||
#### 步骤 2：Skills 查看
|
||||
|
||||
在「功能」标签页的 Skills 区域，你可以看到哪些 Skill 已经部署到你的 Vault 中。每个 Skill 对应一个 SKILL.md 文件。你可以在这里关闭你不希望 Agent 使用的 Skill。这一页只是查看和管理，实际部署是在安装向导中完成的。
|
||||
|
||||
### 2.3 配置 Better BibTeX 自动导出
|
||||
|
||||
在 Zotero 中安装 Better BibTeX 后，你需要为每个领域配置一个自动导出。
|
||||
|
||||
**核心概念**：一个 Zotero Collection（子分类文件夹）对应一个 Base 视图文件。比如你在 Zotero 中有一个名为"骨科"的子分类文件夹，导出这个 Collection 后，Obsidian 中会自动生成 `骨科.base`。这个 Base 文件包含的就是该 Collection 下所有子文件夹里的论文，并且会随 Zotero 的修改实时同步。
|
||||
|
||||
**具体操作**：
|
||||
|
||||
1. 在 Zotero 左侧面板，右键点击你想导出的 Collection（如"骨科"）
|
||||
2. 选择 `导出 Collection...`（Export Collection...）
|
||||
3. 格式选择：**Better BibTeX JSON**（这是唯一支持的格式）
|
||||
4. 勾选 `保持自动更新`（Keep updated）
|
||||
5. 导出路径：选择 Vault 中的 `System/PaperForge/exports/`
|
||||
|
||||
**实际效果**：
|
||||
- 导出的 JSON 文件名自动使用 Collection 名称
|
||||
- Obsidian 的 `Bases/` 目录下会自动生成同名的 `.base` 文件
|
||||
- Base 文件中的论文列表是该 Collection 下所有子文件夹中的文献
|
||||
- 一次配置，永久自动同步：每次 Zotero 中新增/删除/修改文献，JSON 和 Base 都会刷新
|
||||
|
||||
**不需要每加一篇论文就导一次**：自动导出会持续保持同步。
|
||||
|
||||
### 2.4 第一次 Sync
|
||||
|
||||
打开 Dashboard，点击左侧的 `Sync Library` 按钮。
|
||||
|
||||
Sync 做了什么：
|
||||
|
||||
1. 读取 `System/PaperForge/exports/` 下的 JSON 导出文件
|
||||
2. 为每篇论文生成 Obsidian 笔记（`Resources/Literature/<domain>/<key> - Title/<key>.md`）
|
||||
3. 建立文献索引（`System/PaperForge/indexes/formal-library.json`）
|
||||
4. 重建 FTS 记忆数据库
|
||||
|
||||
---
|
||||
|
||||
## 三、Dashboard 完全指南
|
||||
|
||||
Dashboard 是 PaperForge 的核心界面。它有 **三种模式**，根据你当前在 Obsidian 中打开的文件自动切换。
|
||||
|
||||
### 3.1 全局模式（Global Mode）
|
||||
|
||||
当你没有打开任何特定论文时，Dashboard 显示全局面板。
|
||||
|
||||
**Library Snapshot**：显示你的文献库总览
|
||||
|
||||
- `papers`：总论文数
|
||||
- `PDFs ready`：有 PDF 的论文数
|
||||
- `OCR done`：已完成 OCR 的论文数
|
||||
- `deep-read done`：已完成深度阅读的论文数
|
||||
|
||||
**System Status**：系统健康状态指示灯
|
||||
|
||||
| 组件 | 含义 |
|
||||
|------|------|
|
||||
| Runtime | 插件版本和 Python CLI 版本是否匹配 |
|
||||
| Index | 文献索引是否存在且有效 |
|
||||
| Zotero Export | 是否有 Zotero 导出文件 |
|
||||
| OCR Token | PaddleOCR API Key 是否已配置 |
|
||||
| Memory Layer | 记忆层（FTS + 索引）是否健康 |
|
||||
|
||||
**需要处理**：如果有任何状态异常，Dashboard 会列出问题并提供 `Run Doctor` 和 `Repair Issues` 按钮。
|
||||
|
||||
**Start Working**：提供快捷操作按钮
|
||||
|
||||
- `Sync Library`：同步 Zotero → PaperForge
|
||||
- `Run OCR`：开始 OCR 队列处理
|
||||
- `Redo OCR`：执行一次重做 OCR
|
||||
- `Run Doctor`：运行诊断
|
||||
- `Repair Issues`：修复状态不一致
|
||||
|
||||
### 3.2 论文模式（Paper Mode）
|
||||
|
||||
当你打开一篇论文的工作区笔记（`<key>.md`）或 PDF 时，Dashboard 自动切换到此模式。
|
||||
|
||||
**论文元数据**：标题、作者、年份
|
||||
|
||||
**状态条**：三个圆形指示器
|
||||
|
||||
| 指示器 | 含义 | 状态 |
|
||||
|--------|------|------|
|
||||
| PDF | 论文是否有 PDF | ✓（有） / ○（无） |
|
||||
| OCR | OCR 处理状态 | ✓ done / ○ pending / ✗ failed |
|
||||
| 精读 | 深度阅读是否完成 | ✓ done / ○ pending |
|
||||
|
||||
**快捷按钮**
|
||||
|
||||
| 按钮 | 功能 |
|
||||
|------|------|
|
||||
| 打开 PDF | 在 Obsidian 中打开 PDF |
|
||||
| 打开全文 | 打开 OCR 提取的全文 Markdown |
|
||||
|
||||
**文章概览（Paper Overview）**：从笔记的 `## 精读` 区域中自动提取摘要。如果还没有精读，显示提示信息。
|
||||
|
||||
**下一步推荐（Next Step）**：根据当前论文的处理状态，智能推荐下一步操作：
|
||||
|
||||
- `Sync Needed`：论文尚未同步 → 点击 Sync
|
||||
- `OCR Needed`：PDF 已就绪，全文未提取 → 点击 OCR
|
||||
- `Ready for Deep Reading`：全文就绪 → 复制 `/pf-deep <key>` 命令到 Agent
|
||||
- `All Set`：所有流程完成
|
||||
|
||||
**最近讨论**：显示 `ai/discussion.md` 中最新的问答记录。
|
||||
|
||||
**技术详情**（折叠区）：显示更多内部状态和两个重要的勾选框：
|
||||
|
||||
- `do_ocr`：将此论文加入 OCR 队列
|
||||
- `analyze`：标记此论文需要深度阅读
|
||||
- `Fulltext Path`：OCR 全文本的完整 Vault 路径
|
||||
|
||||
### 3.3 领域模式（Collection Mode）
|
||||
|
||||
当你打开某个领域的 `.base` 文件时，Dashboard 切换到此模式。
|
||||
|
||||
**Workflow Overview**：该领域的漏斗图，显示 Total → PDF Ready → OCR Done → Deep Read 的数量。
|
||||
|
||||
**OCR Pipeline**：彩色进度条，一眼看清该领域 OCR 的 `Pending / Processing / Done / Attention` 分布。
|
||||
|
||||
**快捷操作**：`Run OCR`、`Sync Library`、`Redo OCR`（仅影响该领域的论文）。
|
||||
|
||||
---
|
||||
|
||||
## 四、Base 视图完全指南
|
||||
|
||||
每个领域都有对应的 Base 文件（`Bases/<domain>.base`）。Base 是 Obsidian 的数据库视图，可以像表格一样浏览、排序、筛选文献。
|
||||
|
||||
### 4.1 四个标准面板
|
||||
|
||||
| 面板 | 筛选条件 | 用途 |
|
||||
|------|---------|------|
|
||||
| **控制面板** | 无筛选 | 显示该领域全部论文的完整信息。包括 `has_pdf`、`do_ocr`、`analyze`、`ocr_status`、`deep_reading_status` 等所有 workflow 列 |
|
||||
| **待 OCR** | `do_ocr == true && ocr_status == "pending"` | 列出需要 OCR 处理的论文。勾选这里的 `do_ocr`，下次 Run OCR 就会处理 |
|
||||
| **待深度阅读** | `analyze == true && ocr_status == "done" && deep_reading_status == "pending"` | 列出 OCR 已完成但未精读的论文 |
|
||||
| **Redo OCR** | `ocr_status == "done"` | 列出所有 OCR 已完成的论文。勾选 `ocr_redo`，点 Redo 按钮即可重新提取全文 |
|
||||
|
||||
### 4.2 各列的含义
|
||||
|
||||
| 列名 | 含义 | 谁控制 |
|
||||
|------|------|--------|
|
||||
| `title` | 论文标题 | Zotero 自动 |
|
||||
| `year` | 发表年份 | Zotero 自动 |
|
||||
| `first_author` | 第一作者 | Zotero 自动 |
|
||||
| `journal` | 期刊名 | Zotero 自动 |
|
||||
| `impact_factor` | 影响因子 | 自动推算 |
|
||||
| `has_pdf` | 是否有 PDF | 自动检测 Zotero 附件 |
|
||||
| `do_ocr` | 是否提交 OCR | **用户可修改**（勾选 = 下次 Run OCR 时会处理这篇） |
|
||||
| `analyze` | 是否标记精读 | **用户可修改**（勾选 = 准备好让 Agent 精读这篇） |
|
||||
| `ocr_status` | OCR 处理状态 | 系统自动：`pending` → `processing` → `done` / `failed` |
|
||||
| `deep_reading_status` | 深度阅读状态 | 系统自动：`pending` → `done` |
|
||||
| `ocr_redo` | 是否标记重做 | **用户可修改**（勾选 = 下次 Redo OCR 时处理这篇） |
|
||||
| `ocr_time` | OCR 完成时间 | 系统自动 |
|
||||
| `pdf_path` | PDF 链接 | 系统自动 |
|
||||
| `fulltext_md_path` | 全文路径 | 系统自动 |
|
||||
| `collection_path` | Zotero 中的 Collection 路径 | Zotero 自动 |
|
||||
| `collection_tags` | Collection 标签 | Zotero 自动 |
|
||||
|
||||
### 4.3 用户可手动修改的字段
|
||||
|
||||
在整个 frontmatter 中，**以下字段是用户可以手动修改的**，其余全部由系统自动维护：
|
||||
|
||||
- `do_ocr`：勾选后，论文进入 OCR 队列
|
||||
- `analyze`：勾选后，论文标记为"待深度阅读"
|
||||
- `ocr_redo`：勾选后，论文标记为"需要重做 OCR"
|
||||
- `tags`：标签列表，系统默认写入 `文献阅读` 和领域名，你可以随意增删，系统不会覆盖你的修改
|
||||
|
||||
> **警告**：不要手动改 `ocr_status`、`deep_reading_status`、`fulltext_md_path`、`pdf_path` 等系统字段。这些会被 sync 覆盖。
|
||||
|
||||
### 4.4 `tags` 和 `collection_path` 的来源
|
||||
|
||||
- `tags`：从 Zotero 的标签同步，同时系统默认写入 `文献阅读` 和领域名。**你可以在 Obsidian 中增删 tags，系统不会覆盖你加的 tag**
|
||||
- `collection_path`：来自 Zotero 的 Collection 层级。例如 Zotero 中 `骨科 → 软骨修复 → 生物反应器` 的文章，其 `collection_path` 就是 `骨科|软骨修复|生物反应器`
|
||||
- `collection_tags`：Collection 路径中每一级作为标签存入
|
||||
|
||||
---
|
||||
|
||||
## 五、完整工作流
|
||||
|
||||
### 5.1 新增论文的标准流程
|
||||
|
||||
```
|
||||
Zotero 添加文献 → 自动导出 JSON → Sync → 勾选 do_ocr → Run OCR → 打开全文 → Agent 精读
|
||||
```
|
||||
|
||||
1. **在 Zotero 中添加文献**：通过 DOI、PMID 或网页抓取
|
||||
2. **Better BibTeX 自动导出**：JSON 文件自动更新（因为你已经配置了自动导出）
|
||||
3. **Dashboard → Sync**：读取最新的 JSON 导出，生成/更新 Obsidian 笔记
|
||||
4. **勾选 do_ocr**：在 Base 的「待 OCR」面板中勾选 `do_ocr`，或者直接在论文笔记的 frontmatter 中手动设为 `true`
|
||||
5. **Dashboard → Run OCR**：提交 OCR 任务，等待完成
|
||||
6. **打开全文**：OCR 完成后，Dashboard 显示 `打开全文` 按钮
|
||||
7. **Agent 精读**：在 Agent 中调用 PaperForge Skill，让 Agent 读取全文
|
||||
|
||||
### 5.2 重建全文（Redo OCR）的标准流程
|
||||
|
||||
当你需要重新提取某篇论文的全文时：
|
||||
|
||||
1. 在 Base 的「Redo OCR」面板中找到目标论文
|
||||
2. 勾选 `ocr_redo`
|
||||
3. 在 Dashboard 或 Ribbon 点击 `Redo OCR`
|
||||
4. 系统自动：
|
||||
- 删除旧的 OCR 产物
|
||||
- 删除工作区旧 `fulltext.md`
|
||||
- 强制 `do_ocr: true`
|
||||
- 立即重跑 OCR
|
||||
- 成功后写回 `ocr_redo: false`
|
||||
|
||||
**注意**：如果 OCR 失败，`ocr_redo` 会保持 `true`，方便你再次尝试。
|
||||
|
||||
### 5.3 搜索与查找论文
|
||||
|
||||
有三种搜索方式，覆盖面不同：
|
||||
|
||||
| 方式 | 搜索范围 | 在哪里用 |
|
||||
|------|---------|---------|
|
||||
| Base 面板的搜索框 | 标题、作者 | Obsidian 中直接搜 |
|
||||
| 调用 PaperForge Skill | 标题、摘要、作者、全文 | Agent 中调用 |
|
||||
| `paperforge search` CLI | 标题、摘要、作者、期刊 | 命令行 |
|
||||
|
||||
---
|
||||
|
||||
## 六、记忆层（Memory Layer）
|
||||
|
||||
记忆层是 PaperForge 的"大脑"，由两部分组成：
|
||||
|
||||
### 6.1 结构化记忆（Memory DB）
|
||||
|
||||
- 基于 **SQLite + FTS5**（全文搜索引擎）
|
||||
- 存储每篇论文的元数据（标题、作者、摘要、期刊、领域、Collection 等）
|
||||
- 支持精确的字段查询（按作者、年份、领域、OCR 状态等）
|
||||
- 每次 `Sync` 后自动重建
|
||||
|
||||
### 6.2 向量语义搜索（Vector DB）【可选功能】
|
||||
|
||||
> 向量数据库是**可选择性开启**的。即使没有开启，Agent 仍然可以使用 FTS 元数据搜索来查找论文。
|
||||
|
||||
- 基于 **ChromaDB**（向量数据库）
|
||||
- 将所有已完成 OCR 的论文全文切分成段落，生成向量
|
||||
- 支持**语义搜索**：即使用自然语言描述一个概念（如 "75 Hz 电刺激对软骨细胞的分化影<E58C96>"），ChromaDB 也能在正文中找到匹配的段落
|
||||
- 这是检索论文最强大的方式——它搜索的不只是标题和摘要，而是正文的所有内容
|
||||
|
||||
**开启方式**：
|
||||
|
||||
1. 在插件设置页的「功能」标签页中，找到"向量数据库"区域
|
||||
2. 勾选开启，可选配置 embedding 模型（如 OpenAI embedding API、本地 sentence-transformers 等）
|
||||
3. 点击"构建向量数据库"
|
||||
4. 构建完成后，Agent 就可以使用语义搜索在正文 Methods/Results 中匹配概念
|
||||
|
||||
---
|
||||
|
||||
## 七、与 Agent 协作
|
||||
|
||||
### 7.1 PaperForge Skill 是什么
|
||||
|
||||
PaperForge Skill 是部署到 Vault 中的 Agent 技能文件（`SKILL.md`）。它做的事情是：
|
||||
|
||||
1. **让 Agent 知道你的文献库存在**：Agent 会通过 bootstrap 脚本获取 Vault 路径、Python 位置、文献目录
|
||||
2. **给 Agent 提供搜索工具**：Agent 会使用 `paperforge search`（元数据搜索）和 `paperforge retrieve`（语义搜索）来查找论文
|
||||
3. **规范 Agent 的文献操作行为**：Agent 不会用 `grep` 乱搜文件、不会自己拼路径、不会只看摘要就回答事实性问题
|
||||
4. **定义 Agent 的研究工作流**：精读论文、搜索文献、文献问答、记录问答、提取方法论等
|
||||
|
||||
### 7.2 Agent 读了这个 Skill 会做什么
|
||||
|
||||
当 Agent 加载 PaperForge Skill 后，它会：
|
||||
|
||||
1. 执行 `pf_bootstrap.py` 获取路径和配置
|
||||
2. 执行 `agent-context` 获取文献库概览
|
||||
3. 检查 `runtime-health` 确认系统健康
|
||||
4. 根据你的输入判断意图（搜文献、读论文、找证据、保存笔记、提取方法论）
|
||||
5. 路由到对应的 workflow 执行
|
||||
|
||||
### 7.3 所有 Agent 能力详解
|
||||
|
||||
当你安装了 PaperForge Skill 后，Agent 可以直接响应如下自然语言意图：
|
||||
|
||||
| 你说什么 | Agent 会做什么 |
|
||||
|---------|---------------|
|
||||
| "找文献"、"搜文献"、"找一下"、"搜一下" | 搜索论文目录，返回候选列表 |
|
||||
| "库里有什么"、"collection 里" | 列举领域/Collection 中的论文 |
|
||||
| "读一下这篇"、"看看这篇"、"这篇论文" | 打开指定论文进行阅读 |
|
||||
| "找证据"、"找支持"、"找依据"、"找参数" | 检索全文中的具体证据和方法 |
|
||||
| "精读"、"/pf-deep" | 三阶段深度阅读 |
|
||||
| "记一下"、"保存这次"、"记录一下" | 保存当前会话的阅读笔记 |
|
||||
| "提取方法论" | 从论文中提取可复用的研究方法卡片 |
|
||||
| "/pf-paper <key/DOI>" | 快速定位并阅读已知论文 |
|
||||
|
||||
### 7.4 如果 Agent 没有自动读取 Skill 怎么办
|
||||
|
||||
- **通用方法**：在对话中直接写 `调用 paperforge skill`，或告诉 Agent "读取 vault 中 .opencode/skills/paperforge/SKILL.md"（路径根据你的平台调整）
|
||||
- 如果 Agent 仍然无法加载，检查 Vault 的对应 skills 文件夹（如 `.opencode/skills/`）下是否存在 `paperforge/` 目录
|
||||
- 如果不存在，在插件设置页重新运行一次安装向导，或者手动将 Skill 文件复制进去
|
||||
- 重启你的 CLI / Agent 客户端后重试
|
||||
|
||||
**确认 Agent 已加载 Skill 的标志**：Agent 会先运行 `pf_bootstrap.py` 并输出路径变量，而不是直接用 `grep` 搜文件。
|
||||
|
||||
### 7.5 Agent 使用的搜索工具
|
||||
|
||||
Agent 有三种搜索武器，不会只用一种：
|
||||
|
||||
| 工具 | 命令 | 搜索范围 |
|
||||
|------|------|---------|
|
||||
| 元数据搜索 | `paperforge search "query"` | 标题、摘要、作者、期刊、领域、Collection 路径 |
|
||||
| 语义全文搜索 | `paperforge retrieve "query"` | OCR 正文全部内容（需开启向量 DB） |
|
||||
| Collection/领域列举 | `paperforge context --collection/--domain` | 指定 Collection 下的完整论文列表 |
|
||||
|
||||
**Agent 禁止手动 `grep` / `rg` / `glob` 搜文件**。这是 Skill 的硬性规则。如果 Agent 试图这样做，说明它没有正确加载 Skill，你需要提醒它。
|
||||
|
||||
### 7.6 Agent 的意图路由顺序
|
||||
|
||||
Agent 按以下优先级判断你要做什么：
|
||||
|
||||
1. 机械命令（`/pf-sync`、`/pf-ocr`、`/pf-status`） → 直接执行
|
||||
2. 命令别名（`/pf-deep` → 深度分析，`/pf-paper` → 快速阅读）
|
||||
3. 你要保存/归档 → 执行 capture
|
||||
4. 你给定了明确论文 → 执行阅读
|
||||
5. 你要找论文列表 → 执行检索
|
||||
6. 你要找具体证据 → 执行证据查找
|
||||
7. 意图不清 → Agent 会问你两个问题澄清
|
||||
|
||||
---
|
||||
|
||||
## 八、目录结构参考
|
||||
|
||||
```
|
||||
Vault/
|
||||
├── System/
|
||||
│ └── PaperForge/
|
||||
│ ├── ocr/<ZoteroKey>/ ← OCR 产物
|
||||
│ │ ├── fulltext.md ← 唯一的正文真相源
|
||||
│ │ ├── json/result.json ← OCR 原始结果
|
||||
│ │ ├── images/ ← 提取的图片
|
||||
│ │ ├── pages/ ← 页缓存
|
||||
│ │ └── meta.json ← OCR 进度
|
||||
│ ├── exports/ ← Better BibTeX JSON 导出文件
|
||||
│ └── indexes/ ← 文献索引
|
||||
│ └── formal-library.json
|
||||
├── Resources/
|
||||
│ └── Literature/ ← 论文工作区
|
||||
│ └── <domain>/<ZoteroKey> - <Title>/
|
||||
│ ├── <ZoteroKey>.md ← 主笔记（frontmatter + 精读内容）
|
||||
│ └── ai/ ← AI 产物
|
||||
│ └── discussion.md ← Agent 对话记录
|
||||
├── Bases/ ← Obsidian Base 视图
|
||||
│ └── <domain>.base
|
||||
├── <平台相关>skills/paperforge/ ← Agent Skill 文件
|
||||
│ └── SKILL.md
|
||||
└── .obsidian/plugins/paperforge/ ← 插件本体
|
||||
```
|
||||
|
|
@ -127,10 +127,10 @@ def _get_run_ocr():
|
|||
|
||||
|
||||
def _run_ocr_redo(vault: Path, dry_run: bool = False, verbose: bool = False, no_progress: bool = False) -> int:
|
||||
"""Scan for papers with ocr_redo: true, reset their OCR state."""
|
||||
"""Scan for papers with ocr_redo: true, reset and immediately rerun OCR."""
|
||||
from paperforge.worker.ocr import ocr_redo_papers
|
||||
|
||||
return ocr_redo_papers(vault, dry_run=dry_run, verbose=verbose)
|
||||
return ocr_redo_papers(vault, dry_run=dry_run, verbose=verbose, no_progress=no_progress)
|
||||
|
||||
|
||||
def run(args: argparse.Namespace) -> int:
|
||||
|
|
@ -171,7 +171,13 @@ def run(args: argparse.Namespace) -> int:
|
|||
logger.info("Processing specific key: %s", key)
|
||||
|
||||
run_ocr = _get_run_ocr()
|
||||
exit_code = run_ocr(vault, verbose=getattr(args, "verbose", False), no_progress=getattr(args, "no_progress", False))
|
||||
selected_keys = {key} if key else None
|
||||
exit_code = run_ocr(
|
||||
vault,
|
||||
verbose=getattr(args, "verbose", False),
|
||||
no_progress=getattr(args, "no_progress", False),
|
||||
selected_keys=selected_keys,
|
||||
)
|
||||
|
||||
if json_output:
|
||||
queue_data = _collect_ocr_queue_data(vault)
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -56,7 +56,7 @@ export const ACTIONS: ActionDef[] = [
|
|||
},
|
||||
{
|
||||
id: "paperforge-ocr-redo",
|
||||
title: "重做OCR",
|
||||
title: "Redo OCR",
|
||||
desc: "Re-run OCR for papers marked ocr_redo: true",
|
||||
icon: "\u21BA",
|
||||
cmd: "ocr",
|
||||
|
|
@ -87,6 +87,7 @@ export interface PaperForgeSettings {
|
|||
vector_db_api_key: string;
|
||||
vector_db_api_base: string;
|
||||
vector_db_api_model: string;
|
||||
last_seen_version: string;
|
||||
_python_path_stale?: boolean;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
|
@ -114,6 +115,7 @@ export const DEFAULT_SETTINGS: PaperForgeSettings = {
|
|||
resources_dir: "",
|
||||
literature_dir: "",
|
||||
base_dir: "",
|
||||
last_seen_version: "",
|
||||
};
|
||||
|
||||
// ── Workflow state helpers ──
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
import { Plugin, addIcon, Notice } from "obsidian";
|
||||
import { Plugin, addIcon, Notice, Modal, Setting } from "obsidian";
|
||||
import * as fs from "fs";
|
||||
import * as path from "path";
|
||||
import { execFile, exec, spawn } from "child_process";
|
||||
|
|
@ -32,13 +32,13 @@ export default class PaperForgePlugin extends Plugin {
|
|||
|
||||
const redoAction = ACTIONS.find(a => a.id === "paperforge-ocr-redo");
|
||||
if (redoAction) {
|
||||
this.addRibbonIcon("reset", "PaperForge: 重做OCR", () => {
|
||||
this.addRibbonIcon("reset", "PaperForge: Redo OCR", () => {
|
||||
const vp = (this.app.vault.adapter as any).basePath as string;
|
||||
new Notice(`PaperForge: 重做OCR starting...`);
|
||||
new Notice(`PaperForge: Redo OCR starting...`);
|
||||
const { path: py, extraArgs: ex } = resolvePythonExecutable(vp, this.settings, undefined, undefined);
|
||||
execFile(py, [...ex, "-m", "paperforge", "ocr", "redo"], { cwd: vp, timeout: 600000 }, (err, stdout, stderr) => {
|
||||
if (err) { new Notice(`PaperForge: 重做OCR failed`); return; }
|
||||
new Notice(`PaperForge: 重做OCR done`);
|
||||
if (err) { new Notice(`PaperForge: Redo OCR failed`); return; }
|
||||
new Notice(`PaperForge: Redo OCR done`);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
|
@ -80,6 +80,7 @@ export default class PaperForgePlugin extends Plugin {
|
|||
}
|
||||
this._startFilePolling();
|
||||
this._firstLaunchSnapshotMigration();
|
||||
this._checkReleaseNotes();
|
||||
}
|
||||
|
||||
private _firstLaunchSnapshotMigration() {
|
||||
|
|
@ -335,4 +336,67 @@ export default class PaperForgePlugin extends Plugin {
|
|||
}
|
||||
await this.saveData(dataToSave);
|
||||
}
|
||||
|
||||
private _checkReleaseNotes() {
|
||||
const currentVersion = this.manifest.version;
|
||||
const seen = this.settings.last_seen_version;
|
||||
if (seen === currentVersion) return;
|
||||
|
||||
const releaseNotesData = require("./release-notes.json");
|
||||
const versions = releaseNotesData.versions || [];
|
||||
const currentEntry = versions.find((v: any) => v.version === currentVersion);
|
||||
|
||||
class ReleaseNotesModal extends Modal {
|
||||
private _entry: any;
|
||||
constructor(app: any, entry: any) {
|
||||
super(app);
|
||||
this._entry = entry;
|
||||
}
|
||||
onOpen() {
|
||||
const { contentEl } = this;
|
||||
contentEl.createEl("h2", { text: `PaperForge v${currentVersion} \u66F4\u65B0\u8BF4\u660E` });
|
||||
if (this._entry) {
|
||||
contentEl.createEl("p", { text: this._entry.title, cls: "paperforge-modal-subtitle" });
|
||||
if (this._entry.breaking_or_migration && this._entry.breaking_or_migration.length > 0) {
|
||||
contentEl.createEl("h4", { text: "\u884C\u4E3A\u53D8\u66F4 / \u8FC1\u79FB\u6CE8\u610F" });
|
||||
for (const item of this._entry.breaking_or_migration) {
|
||||
contentEl.createEl("p", { text: `\u2022 ${item}`, cls: "paperforge-modal-item" });
|
||||
}
|
||||
}
|
||||
if (this._entry.new_features && this._entry.new_features.length > 0) {
|
||||
contentEl.createEl("h4", { text: "\u65B0\u529F\u80FD" });
|
||||
for (const item of this._entry.new_features) {
|
||||
contentEl.createEl("p", { text: `\u2022 ${item}`, cls: "paperforge-modal-item" });
|
||||
}
|
||||
}
|
||||
if (this._entry.fixes && this._entry.fixes.length > 0) {
|
||||
contentEl.createEl("h4", { text: "\u4FEE\u590D" });
|
||||
for (const item of this._entry.fixes) {
|
||||
contentEl.createEl("p", { text: `\u2022 ${item}`, cls: "paperforge-modal-item" });
|
||||
}
|
||||
}
|
||||
if (this._entry.recommended_actions && this._entry.recommended_actions.length > 0) {
|
||||
contentEl.createEl("h4", { text: "\u5EFA\u8BAE\u64CD\u4F5C" });
|
||||
for (const item of this._entry.recommended_actions) {
|
||||
contentEl.createEl("p", { text: `\u2022 ${item}`, cls: "paperforge-modal-item" });
|
||||
}
|
||||
}
|
||||
} else {
|
||||
contentEl.createEl("p", { text: "\u7248\u672C\u5DF2\u66F4\u65B0\u81F3 v" + currentVersion + "\uFF0C\u8BF7\u524D\u5F80\u8BBE\u7F6E \u2192 \u66F4\u65B0\u4E0E\u624B\u518C \u67E5\u770B\u5B8C\u6574\u66F4\u65B0\u8BB0\u5F55\u3002" });
|
||||
}
|
||||
new Setting(contentEl)
|
||||
.addButton(btn => btn.setButtonText("\u77E5\u9053\u4E86").setCta().onClick(() => {
|
||||
this.close();
|
||||
}));
|
||||
}
|
||||
onClose() {
|
||||
const { contentEl } = this;
|
||||
contentEl.empty();
|
||||
}
|
||||
}
|
||||
|
||||
new ReleaseNotesModal(this.app, currentEntry).open();
|
||||
this.settings.last_seen_version = currentVersion;
|
||||
this.saveSettings();
|
||||
}
|
||||
}
|
||||
|
|
|
|||
31
paperforge/plugin/src/release-notes.json
Normal file
31
paperforge/plugin/src/release-notes.json
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
{
|
||||
"versions": [
|
||||
{
|
||||
"version": "1.5.15",
|
||||
"date": "2026-06-01",
|
||||
"title": "OCR 重新设计 — 全文单一真相源 + Redo 闭环 + 阅读顺序修复",
|
||||
"breaking_or_migration": [
|
||||
"fulltext 正文现在统一位于 System/PaperForge/ocr/<key>/fulltext.md，工作区 Resources/.../fulltext.md 不再保留为正文副本",
|
||||
"OCR Redo 现在会立即重跑 OCR（闭环执行），不再只是标记状态后需要手动再跑一次"
|
||||
],
|
||||
"new_features": [
|
||||
"新增 OCR Redo 闭环执行：勾选重做后自动删除旧产物 → 强制 do_ocr → 立即重跑 OCR → 成功后写回 ocr_redo:false",
|
||||
"设置页新增「更新与手册」标签页，可随时查看版本更新记录与使用手册链接",
|
||||
"首次更新时自动弹出更新说明"
|
||||
],
|
||||
"fixes": [
|
||||
"修复 OCR 双栏阅读顺序混乱：2.5 / Discussion / Introduction 等章节标题和正文错位",
|
||||
"修复 Kwon et al. 等叙述句被误判为参考文献，导致正文段落被提前截断并插入 Discussion",
|
||||
"修复 Fig.3 / Fig.4 等页首图注和图块被重排逻辑整体后移，导致图文断裂",
|
||||
"修复首页（page 1）Abstract / Introduction 排序错乱，METHODS 行跑到 Introduction 后",
|
||||
"修复并排 panel（如 Fig.2 的左右两块）未合并为单张 composite figure，被拆成两张独立图片",
|
||||
"修复 redo OCR 后工作区 fulltext 未被同步覆盖，导致用户看到的仍是旧乱序版本",
|
||||
"Dashboard 现在可以识别 System/PaperForge/ocr/<key>/fulltext.md 路径，打开全文后能切到对应 paper 视图"
|
||||
],
|
||||
"recommended_actions": [
|
||||
"如果论文是此版本前完成的 OCR，建议对重要论文执行一次「重做OCR」（在 Base 视图勾选 ocr_redo 后点 ribbon Redo 按钮），以修复阅读顺序问题",
|
||||
"打开全文请直接使用 Dashboard 的「打开全文」按钮，正文已经统一迁至 System/PaperForge/ocr/ 下"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -5,6 +5,7 @@ import * as os from "os";
|
|||
import { execFile, execFileSync, spawn, exec } from "child_process";
|
||||
import { t, setLanguage } from "./i18n";
|
||||
import { PaperForgeSettings } from "./constants";
|
||||
import releaseNotesData from "./release-notes.json";
|
||||
import {
|
||||
resolvePythonExecutable, buildRuntimeInstallCommand,
|
||||
paperforgeEnrichedEnv, scanBbtUnderProfiles, scanBbtDirectChildren,
|
||||
|
|
@ -83,6 +84,16 @@ export class PaperForgeSettingTab extends PluginSettingTab {
|
|||
.paperforge-skills-group { margin-bottom: 10px; }
|
||||
.paperforge-skills-group:last-child { margin-bottom: 0; }
|
||||
.vertical-tab-content-container { overflow-y: scroll !important; }
|
||||
.paperforge-release-card { border: 1px solid var(--background-modifier-border); border-radius: 6px; padding: 12px; margin-bottom: 12px; }
|
||||
.paperforge-release-header { margin-bottom: 8px; }
|
||||
.paperforge-release-date { color: var(--text-muted); font-size: 12px; }
|
||||
.paperforge-release-section { margin-bottom: 6px; }
|
||||
.paperforge-release-label { font-weight: 600; color: var(--text-accent); margin-bottom: 2px; font-size: 13px; }
|
||||
.paperforge-release-item { font-size: 13px; margin-left: 8px; color: var(--text-muted); }
|
||||
.paperforge-manual-links { margin-top: 8px; }
|
||||
.paperforge-manual-links a { color: var(--text-accent); }
|
||||
.paperforge-modal-subtitle { color: var(--text-muted); font-size: 13px; margin-bottom: 12px; }
|
||||
.paperforge-modal-item { font-size: 13px; margin-left: 8px; color: var(--text-muted); }
|
||||
`;
|
||||
document.head.appendChild(style);
|
||||
}
|
||||
|
|
@ -90,8 +101,9 @@ export class PaperForgeSettingTab extends PluginSettingTab {
|
|||
// --- Tab bar ---
|
||||
const tabBar = containerEl.createDiv({ cls: "paperforge-settings-tabs" });
|
||||
const tabs = [
|
||||
{ id: "setup", label: t("tab_setup") || "Installation" },
|
||||
{ id: "features", label: t("tab_features") || "Features" },
|
||||
{ id: "setup", label: t("tab_setup") || "\u5B89\u88C5" },
|
||||
{ id: "features", label: t("tab_features") || "\u529F\u80FD" },
|
||||
{ id: "release-notes", label: "\u66F4\u65B0\u4E0E\u624B\u518C" },
|
||||
];
|
||||
const tabContents: Record<string, HTMLDivElement> = {};
|
||||
|
||||
|
|
@ -116,8 +128,10 @@ export class PaperForgeSettingTab extends PluginSettingTab {
|
|||
// --- Render active tab ---
|
||||
if (this.activeTab === "setup") {
|
||||
this._renderSetupTab(tabContents.setup);
|
||||
} else {
|
||||
} else if (this.activeTab === "features") {
|
||||
this._renderFeaturesTab(tabContents.features);
|
||||
} else {
|
||||
this._renderReleaseNotesTab(tabContents["release-notes"]);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1242,4 +1256,59 @@ export class PaperForgeSettingTab extends PluginSettingTab {
|
|||
onPass();
|
||||
});
|
||||
}
|
||||
|
||||
_renderReleaseNotesTab(containerEl: HTMLElement) {
|
||||
containerEl.createEl("h2", { text: "\u66F4\u65B0\u4E0E\u624B\u518C" });
|
||||
|
||||
containerEl.createEl("h3", { text: "\u7248\u672C\u66F4\u65B0\u8BB0\u5F55" });
|
||||
|
||||
const versions = (releaseNotesData as any).versions || [];
|
||||
for (const ver of versions) {
|
||||
const card = containerEl.createEl("div", { cls: "paperforge-release-card" });
|
||||
|
||||
const header = card.createEl("div", { cls: "paperforge-release-header" });
|
||||
header.createEl("strong", { text: `v${ver.version} \u2014 ${ver.title}` });
|
||||
header.createEl("span", { cls: "paperforge-release-date", text: ` (${ver.date})` });
|
||||
|
||||
if (ver.breaking_or_migration && ver.breaking_or_migration.length > 0) {
|
||||
const section = card.createEl("div", { cls: "paperforge-release-section" });
|
||||
section.createEl("div", { cls: "paperforge-release-label", text: "\u884C\u4E3A\u53D8\u66F4 / \u8FC1\u79FB\u6CE8\u610F" });
|
||||
for (const item of ver.breaking_or_migration) {
|
||||
section.createEl("div", { cls: "paperforge-release-item", text: `\u2022 ${item}` });
|
||||
}
|
||||
}
|
||||
|
||||
if (ver.new_features && ver.new_features.length > 0) {
|
||||
const section = card.createEl("div", { cls: "paperforge-release-section" });
|
||||
section.createEl("div", { cls: "paperforge-release-label", text: "\u65B0\u529F\u80FD" });
|
||||
for (const item of ver.new_features) {
|
||||
section.createEl("div", { cls: "paperforge-release-item", text: `\u2022 ${item}` });
|
||||
}
|
||||
}
|
||||
|
||||
if (ver.fixes && ver.fixes.length > 0) {
|
||||
const section = card.createEl("div", { cls: "paperforge-release-section" });
|
||||
section.createEl("div", { cls: "paperforge-release-label", text: "\u4FEE\u590D" });
|
||||
for (const item of ver.fixes) {
|
||||
section.createEl("div", { cls: "paperforge-release-item", text: `\u2022 ${item}` });
|
||||
}
|
||||
}
|
||||
|
||||
if (ver.recommended_actions && ver.recommended_actions.length > 0) {
|
||||
const section = card.createEl("div", { cls: "paperforge-release-section" });
|
||||
section.createEl("div", { cls: "paperforge-release-label", text: "\u5EFA\u8BAE\u64CD\u4F5C" });
|
||||
for (const item of ver.recommended_actions) {
|
||||
section.createEl("div", { cls: "paperforge-release-item", text: `\u2022 ${item}` });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
containerEl.createEl("h3", { text: "\u4F7F\u7528\u624B\u518C" });
|
||||
const manualSection = containerEl.createEl("div", { cls: "paperforge-manual-links" });
|
||||
const manualLink = manualSection.createEl("a", {
|
||||
text: "\u2192 \u67E5\u770B\u5B8C\u6574\u4F7F\u7528\u624B\u518C\uFF08GitHub\uFF09",
|
||||
href: "https://github.com/LLLin000/PaperForge/blob/main/docs/user-manual.md",
|
||||
});
|
||||
manualLink.setAttr("target", "_blank");
|
||||
}
|
||||
}
|
||||
|
|
|
|||
16
paperforge/plugin/src/utils/zotero-path.ts
Normal file
16
paperforge/plugin/src/utils/zotero-path.ts
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
import * as path from 'path';
|
||||
|
||||
export function extractZoteroKeyFromPath(filePath: string): string | null {
|
||||
if (!filePath) return null;
|
||||
let dir = path.dirname(filePath);
|
||||
while (true) {
|
||||
const basename = path.basename(dir);
|
||||
if (!basename || basename === '.') break;
|
||||
const match = basename.match(/^([A-Z0-9]{8})(?:\s*-\s*.*)?$/i);
|
||||
if (match) return match[1];
|
||||
const parent = path.dirname(dir);
|
||||
if (parent === dir) break;
|
||||
dir = parent;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
|
@ -17,6 +17,7 @@ import {
|
|||
checkRuntimeVersion, paperforgeEnrichedEnv
|
||||
} from "../services/python-bridge";
|
||||
import { getDisclosureState, toggleDisclosureState } from "../utils/disclosure";
|
||||
import { extractZoteroKeyFromPath } from "../utils/zotero-path";
|
||||
import { checkOrphanState } from "./modals";
|
||||
|
||||
// ── Interface for plugin ref used by static open ──
|
||||
|
|
@ -557,18 +558,7 @@ export class PaperForgeStatusView extends ItemView {
|
|||
|
||||
/* ── Extract zotero_key from workspace directory name ── */
|
||||
_extractZoteroKeyFromPath(filePath: string): string | null {
|
||||
if (!filePath) return null;
|
||||
let dir = path.dirname(filePath);
|
||||
while (true) {
|
||||
const basename = path.basename(dir);
|
||||
if (!basename || basename === '.') break;
|
||||
const match = basename.match(/^([A-Z0-9]{8})(?:\s*-\s*)/i);
|
||||
if (match) return match[1];
|
||||
const parent = path.dirname(dir);
|
||||
if (parent === dir) break;
|
||||
dir = parent;
|
||||
}
|
||||
return null;
|
||||
return extractZoteroKeyFromPath(filePath);
|
||||
}
|
||||
|
||||
/* ── Pure Mode Resolution (D-07, Phase 32) ── */
|
||||
|
|
@ -800,7 +790,7 @@ export class PaperForgeStatusView extends ItemView {
|
|||
});
|
||||
const globalRedoBtn = btnsRow.createEl('button', { cls: 'paperforge-contextual-btn warn' });
|
||||
globalRedoBtn.createEl('span', { cls: 'paperforge-contextual-btn-icon', text: '\u21BA' });
|
||||
globalRedoBtn.createEl('span', { text: '\u91CD\u505AOCR' });
|
||||
globalRedoBtn.createEl('span', { text: 'Redo OCR' });
|
||||
globalRedoBtn.addEventListener('click', () => {
|
||||
const action = ACTIONS.find(a => a.id === 'paperforge-ocr-redo');
|
||||
if (action) this._runAction(action, globalRedoBtn);
|
||||
|
|
@ -1263,7 +1253,7 @@ export class PaperForgeStatusView extends ItemView {
|
|||
});
|
||||
const redoBtn = actionsRow.createEl('button', { cls: 'paperforge-contextual-btn warn' });
|
||||
redoBtn.createEl('span', { cls: 'paperforge-contextual-btn-icon', text: '\u21BA' });
|
||||
redoBtn.createEl('span', { text: '\u91CD\u505AOCR' });
|
||||
redoBtn.createEl('span', { text: 'Redo OCR' });
|
||||
redoBtn.addEventListener('click', () => {
|
||||
const action = ACTIONS.find(a => a.id === 'paperforge-ocr-redo');
|
||||
if (action) this._runAction(action, redoBtn);
|
||||
|
|
@ -1317,6 +1307,11 @@ export class PaperForgeStatusView extends ItemView {
|
|||
const cache = this.app.metadataCache.getFileCache(activeFile);
|
||||
if (cache && cache.frontmatter && cache.frontmatter.zotero_key) {
|
||||
key = cache.frontmatter.zotero_key;
|
||||
} else {
|
||||
key = this._extractZoteroKeyFromPath(activeFile.path);
|
||||
}
|
||||
if (key) {
|
||||
extraArgs = [...extraArgs, key];
|
||||
} else if (cache && cache.frontmatter) {
|
||||
this._showMessage('[!!] No zotero_key in active note frontmatter', 'error');
|
||||
new Notice('[!!] Open a paper note with a zotero_key in its frontmatter first', 6000);
|
||||
|
|
@ -1334,7 +1329,6 @@ export class PaperForgeStatusView extends ItemView {
|
|||
card.removeClass('running');
|
||||
return;
|
||||
}
|
||||
extraArgs = [...extraArgs, key];
|
||||
}
|
||||
if (a.needsFilter) {
|
||||
extraArgs = [...extraArgs, '--all'];
|
||||
|
|
|
|||
13
paperforge/plugin/tests/zotero-path.test.ts
Normal file
13
paperforge/plugin/tests/zotero-path.test.ts
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import { extractZoteroKeyFromPath } from '../src/utils/zotero-path';
|
||||
|
||||
describe('extractZoteroKeyFromPath', () => {
|
||||
it('extracts zotero key from workspace directory path', () => {
|
||||
expect(extractZoteroKeyFromPath('Resources/Literature/test/2AZ2EGQB - Paper/2AZ2EGQB.md')).toBe('2AZ2EGQB');
|
||||
});
|
||||
|
||||
it('extracts zotero key from canonical OCR fulltext path', () => {
|
||||
expect(extractZoteroKeyFromPath('System/PaperForge/ocr/2AZ2EGQB/fulltext.md')).toBe('2AZ2EGQB');
|
||||
});
|
||||
});
|
||||
|
|
@ -92,6 +92,20 @@ def write_jsonl(path: Path, rows) -> None:
|
|||
path.write_text(text, encoding="utf-8")
|
||||
|
||||
|
||||
def sync_workspace_fulltext(source_fulltext: Path, target_fulltext: Path) -> bool:
|
||||
if not source_fulltext.exists():
|
||||
return False
|
||||
if target_fulltext.exists():
|
||||
try:
|
||||
if source_fulltext.read_text(encoding="utf-8") == target_fulltext.read_text(encoding="utf-8"):
|
||||
return False
|
||||
except Exception:
|
||||
pass
|
||||
target_fulltext.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(str(source_fulltext), str(target_fulltext))
|
||||
return True
|
||||
|
||||
|
||||
# --- YAML Helpers ---
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -341,16 +341,15 @@ def _build_entry(item: dict, vault: Path, paths: dict, domain: str, zotero_dir:
|
|||
except Exception:
|
||||
pass # alias will be injected on next full frontmatter_note pass
|
||||
break # only one old file per key
|
||||
target_fulltext = workspace_dir / "fulltext.md"
|
||||
source_fulltext = paths["ocr"] / key / "fulltext.md"
|
||||
stale_workspace_fulltext = workspace_dir / "fulltext.md"
|
||||
|
||||
workspace_dir.mkdir(parents=True, exist_ok=True)
|
||||
(workspace_dir / "ai").mkdir(parents=True, exist_ok=True)
|
||||
if meta.get("ocr_status") == "done" and source_fulltext.exists() and not target_fulltext.exists():
|
||||
shutil.copy2(str(source_fulltext), str(target_fulltext))
|
||||
logger.info("Bridged fulltext.md to workspace for %s", key)
|
||||
if stale_workspace_fulltext.exists():
|
||||
stale_workspace_fulltext.unlink()
|
||||
|
||||
fulltext_exists = target_fulltext.exists()
|
||||
fulltext_exists = source_fulltext.exists()
|
||||
|
||||
# ---- entry dict -------------------------------------------------------
|
||||
authors = item.get("authors", [])
|
||||
|
|
@ -447,7 +446,7 @@ def _build_entry(item: dict, vault: Path, paths: dict, domain: str, zotero_dir:
|
|||
# Workspace path fields are only advertised when the backing files/dirs exist.
|
||||
"paper_root": str(workspace_dir.relative_to(vault)).replace("\\", "/") + "/",
|
||||
"main_note_path": str(main_note_path.relative_to(vault)).replace("\\", "/"),
|
||||
"fulltext_path": str(target_fulltext.relative_to(vault)).replace("\\", "/") if fulltext_exists else "",
|
||||
"fulltext_path": str(source_fulltext.relative_to(vault)).replace("\\", "/") if fulltext_exists else "",
|
||||
"deep_reading_path": "", # deprecated: deep reading content now lives in main note as ## 🔍 精读
|
||||
"ai_path": str((workspace_dir / "ai").relative_to(vault)).replace("\\", "/") + "/",
|
||||
}
|
||||
|
|
|
|||
|
|
@ -385,15 +385,31 @@ def _col_xc(block: dict) -> float | None:
|
|||
return (bbox[0] + bbox[2]) / 2
|
||||
|
||||
|
||||
def _col_switches(blocks: list[dict], page_width: int) -> int:
|
||||
def _column_kind(block: dict, page_width: int) -> str | None:
|
||||
bbox = block.get("block_bbox", [0, 0, 0, 0])
|
||||
if len(bbox) < 4 or bbox[2] <= bbox[0]:
|
||||
return None
|
||||
mid = page_width / 2 if page_width else 600
|
||||
x1, _, x2, _ = bbox
|
||||
width = x2 - x1
|
||||
# Full-width or center-spanning blocks should keep their original slot.
|
||||
if (x1 < mid and x2 > mid) or (page_width and width >= page_width * 0.55):
|
||||
return "S"
|
||||
xc = (x1 + x2) / 2
|
||||
if xc < mid * 0.95:
|
||||
return "L"
|
||||
if xc > mid * 1.05:
|
||||
return "R"
|
||||
return "S"
|
||||
|
||||
|
||||
def _col_switches(blocks: list[dict], page_width: int) -> int:
|
||||
switches = 0
|
||||
prev_col = None
|
||||
for b in blocks:
|
||||
xc = _col_xc(b)
|
||||
if xc is None:
|
||||
col = _column_kind(b, page_width)
|
||||
if col is None:
|
||||
continue
|
||||
col = "L" if xc < mid * 0.95 else "R" if xc > mid * 1.05 else "S"
|
||||
if prev_col and col != "S" and prev_col != "S" and col != prev_col:
|
||||
switches += 1
|
||||
if col != "S":
|
||||
|
|
@ -404,25 +420,34 @@ def _col_switches(blocks: list[dict], page_width: int) -> int:
|
|||
def validate_block_order(blocks: list[dict], page_width: int) -> list[dict]:
|
||||
if len(blocks) < 4 or not page_width:
|
||||
return blocks
|
||||
mid = page_width / 2
|
||||
|
||||
if _col_switches(blocks, page_width) > 3:
|
||||
left = [b for b in blocks if _col_xc(b) is not None and (_col_xc(b) or 0) < mid]
|
||||
right = [b for b in blocks if _col_xc(b) is not None and (_col_xc(b) or 0) >= mid]
|
||||
other = [b for b in blocks if _col_xc(b) is None]
|
||||
left = [b for b in blocks if _column_kind(b, page_width) == "L"]
|
||||
right = [b for b in blocks if _column_kind(b, page_width) == "R"]
|
||||
centered = [b for b in blocks if _column_kind(b, page_width) not in {"L", "R"}]
|
||||
left.sort(key=lambda b: b.get("block_bbox", [0, 0, 0, 0])[1])
|
||||
right.sort(key=lambda b: b.get("block_bbox", [0, 0, 0, 0])[1])
|
||||
return left + right + other
|
||||
if not centered:
|
||||
return left + right
|
||||
result: list[dict] = []
|
||||
iters = {"L": iter(left), "R": iter(right)}
|
||||
for b in blocks:
|
||||
col_key = _column_kind(b, page_width)
|
||||
if col_key not in {"L", "R"}:
|
||||
result.append(b)
|
||||
continue
|
||||
try:
|
||||
result.append(next(iters[col_key]))
|
||||
except StopIteration:
|
||||
result.append(b)
|
||||
return result
|
||||
|
||||
col_buckets: dict[str, list[dict]] = {"L": [], "R": []}
|
||||
for b in blocks:
|
||||
xc = _col_xc(b)
|
||||
if xc is None:
|
||||
col_key = _column_kind(b, page_width)
|
||||
if col_key not in {"L", "R"}:
|
||||
continue
|
||||
if xc < mid:
|
||||
col_buckets["L"].append(b)
|
||||
else:
|
||||
col_buckets["R"].append(b)
|
||||
col_buckets[col_key].append(b)
|
||||
|
||||
for key in ("L", "R"):
|
||||
sorted_col = sorted(col_buckets[key], key=lambda b: b.get("block_bbox", [0, 0, 0, 0])[1])
|
||||
|
|
@ -432,11 +457,10 @@ def validate_block_order(blocks: list[dict], page_width: int) -> list[dict]:
|
|||
result: list[dict] = []
|
||||
iters = {"L": iter(col_buckets["L"]), "R": iter(col_buckets["R"])}
|
||||
for b in blocks:
|
||||
xc = _col_xc(b)
|
||||
if xc is None:
|
||||
col_key = _column_kind(b, page_width)
|
||||
if col_key not in {"L", "R"}:
|
||||
result.append(b)
|
||||
continue
|
||||
col_key = "L" if xc < mid else "R"
|
||||
try:
|
||||
result.append(next(iters[col_key]))
|
||||
except StopIteration:
|
||||
|
|
@ -794,7 +818,17 @@ def media_clusters(blocks: list[dict]) -> tuple[dict[int, int], list[list[dict]]
|
|||
cy1 = min(item["block_bbox"][1] for item in cluster) - 40
|
||||
cx2 = max(item["block_bbox"][2] for item in cluster) + 40
|
||||
cy2 = max(item["block_bbox"][3] for item in cluster) + 40
|
||||
if not (x2 < cx1 or x1 > cx2 or y2 < cy1 or (y1 > cy2)):
|
||||
vertical_overlap = max(0, min(y2, cy2) - max(y1, cy1))
|
||||
min_height = max(1, min(y2 - y1, cy2 - cy1))
|
||||
if x2 < cx1:
|
||||
horizontal_gap = cx1 - x2
|
||||
elif x1 > cx2:
|
||||
horizontal_gap = x1 - cx2
|
||||
else:
|
||||
horizontal_gap = 0
|
||||
overlaps_cluster = not (x2 < cx1 or x1 > cx2 or y2 < cy1 or y1 > cy2)
|
||||
side_by_side_panel = vertical_overlap >= int(min_height * 0.35) and horizontal_gap <= 60
|
||||
if overlaps_cluster or side_by_side_panel:
|
||||
assigned = idx
|
||||
break
|
||||
if assigned is None:
|
||||
|
|
@ -1373,9 +1407,10 @@ def render_page_blocks(
|
|||
pruned = result.get("prunedResult", {})
|
||||
ocr_width = int(pruned.get("width", 0) or 0)
|
||||
blocks = sorted(pruned.get("parsing_res_list", []), key=block_sort_key)
|
||||
blocks = validate_block_order(blocks, ocr_width)
|
||||
ocr_height = int(pruned.get("height", 0) or 0)
|
||||
blocks = _apply_layered_body_reorder(blocks, ocr_width, ocr_height)
|
||||
if page_index != 1:
|
||||
blocks = validate_block_order(blocks, ocr_width)
|
||||
blocks = _apply_layered_body_reorder(blocks, ocr_width, ocr_height)
|
||||
raw_reference_blocks = [block for block in blocks if block.get("block_label") == "reference_content"]
|
||||
first_reference_y = min(
|
||||
(block.get("block_bbox", [0, 10**9, 0, 0])[1] for block in raw_reference_blocks), default=10**9
|
||||
|
|
@ -1467,11 +1502,11 @@ def render_page_blocks(
|
|||
if page_index == 1 and affiliation_buffer:
|
||||
rendered.extend(affiliation_buffer)
|
||||
affiliation_buffer.clear()
|
||||
rendered.append(clean_block_text(content))
|
||||
if page_index == 1 and deferred_meta:
|
||||
rendered.extend(deferred_meta)
|
||||
deferred_meta.clear()
|
||||
first_page_meta_done = True
|
||||
rendered.append(clean_block_text(content))
|
||||
continue
|
||||
if label == "reference_content":
|
||||
text = clean_block_text(content)
|
||||
|
|
@ -1667,14 +1702,32 @@ def postprocess_ocr_result(vault: Path, key: str, all_results: list[dict]) -> tu
|
|||
return (page_num, markdown_path, json_path, fulltext_md_path)
|
||||
|
||||
|
||||
def ocr_redo_papers(vault: Path, dry_run: bool = False, verbose: bool = False) -> int:
|
||||
"""Scan for papers with ocr_redo: true, reset their OCR state.
|
||||
def _workspace_fulltext_candidates(lit_root: Path, zotero_key: str) -> list[Path]:
|
||||
if not lit_root.exists():
|
||||
return []
|
||||
matches: list[Path] = []
|
||||
for candidate in lit_root.rglob(f"{zotero_key} - *"):
|
||||
if candidate.is_dir():
|
||||
fulltext_path = candidate / "fulltext.md"
|
||||
if fulltext_path.exists():
|
||||
matches.append(fulltext_path)
|
||||
return matches
|
||||
|
||||
|
||||
def _rewrite_note_fields(text: str, **fields: object) -> str:
|
||||
updated = text
|
||||
for field, value in fields.items():
|
||||
updated = _sync.update_frontmatter_field(updated, field, value)
|
||||
return updated
|
||||
|
||||
|
||||
def ocr_redo_papers(vault: Path, dry_run: bool = False, verbose: bool = False, no_progress: bool = False) -> int:
|
||||
"""Scan for papers with ocr_redo: true, reset and immediately rerun OCR.
|
||||
|
||||
Spec 2.6: paperforge ocr redo [--dry-run]
|
||||
"""
|
||||
from paperforge.adapters.obsidian_frontmatter import (
|
||||
extract_preserved_ocr_redo,
|
||||
update_frontmatter_field,
|
||||
)
|
||||
|
||||
paths = pipeline_paths(vault)
|
||||
|
|
@ -1686,7 +1739,7 @@ def ocr_redo_papers(vault: Path, dry_run: bool = False, verbose: bool = False) -
|
|||
logger.info("No literature directory found, nothing to redo")
|
||||
return 0
|
||||
|
||||
redo_entries = []
|
||||
redo_entry_map: dict[str, tuple[Path, str]] = {}
|
||||
for note_file in sorted(lit_root.rglob("*.md")):
|
||||
if note_file.name in ("fulltext.md", "deep-reading.md", "discussion.md"):
|
||||
continue
|
||||
|
|
@ -1704,7 +1757,11 @@ def ocr_redo_papers(vault: Path, dry_run: bool = False, verbose: bool = False) -
|
|||
if verbose:
|
||||
logger.warning("Skipping invalid zotero_key: %s", zotero_key)
|
||||
continue
|
||||
redo_entries.append((zotero_key, note_file, text))
|
||||
existing = redo_entry_map.get(zotero_key)
|
||||
if existing is None or note_file.name == f"{zotero_key}.md":
|
||||
redo_entry_map[zotero_key] = (note_file, text)
|
||||
|
||||
redo_entries = [(key, note_file, text) for key, (note_file, text) in sorted(redo_entry_map.items())]
|
||||
|
||||
if not redo_entries:
|
||||
print("No papers with ocr_redo: true found", flush=True)
|
||||
|
|
@ -1715,29 +1772,61 @@ def ocr_redo_papers(vault: Path, dry_run: bool = False, verbose: bool = False) -
|
|||
print(f" {zotero_key} ({note_file.parent.name})", flush=True)
|
||||
|
||||
if dry_run:
|
||||
print("\nDry-run mode: no changes made. Run without --dry-run to reset.", flush=True)
|
||||
print("\nDry-run mode: no changes made. Run without --dry-run to rerun OCR.", flush=True)
|
||||
return 0
|
||||
|
||||
reset_keys = []
|
||||
for zotero_key, note_file, text in redo_entries:
|
||||
for workspace_fulltext in _workspace_fulltext_candidates(lit_root, zotero_key):
|
||||
workspace_fulltext.unlink(missing_ok=True)
|
||||
if verbose:
|
||||
logger.info("Deleted workspace fulltext for %s", zotero_key)
|
||||
ocr_dir = ocr_root / zotero_key if ocr_root else None
|
||||
if ocr_dir and ocr_dir.exists():
|
||||
shutil.rmtree(ocr_dir)
|
||||
if verbose:
|
||||
logger.info("Deleted OCR directory for %s", zotero_key)
|
||||
|
||||
text = update_frontmatter_field(text, "ocr_status", "pending")
|
||||
text = update_frontmatter_field(text, "ocr_redo", False)
|
||||
text = update_frontmatter_field(text, "fulltext_md_path", "")
|
||||
text = _rewrite_note_fields(
|
||||
text,
|
||||
do_ocr=True,
|
||||
ocr_status="pending",
|
||||
fulltext_md_path="",
|
||||
ocr_redo=True,
|
||||
)
|
||||
note_file.write_text(text, encoding="utf-8")
|
||||
reset_keys.append(zotero_key)
|
||||
|
||||
print(f"Reset {len(reset_keys)} paper(s): {', '.join(reset_keys)}", flush=True)
|
||||
print("Run 'paperforge ocr run' to process the reset papers.", flush=True)
|
||||
return 0
|
||||
print("Launching OCR immediately for reset papers...", flush=True)
|
||||
|
||||
exit_code = run_ocr(vault, verbose=verbose, no_progress=no_progress, selected_keys=set(reset_keys))
|
||||
|
||||
success_keys: list[str] = []
|
||||
failed_keys: list[str] = []
|
||||
for zotero_key, note_file, _original_text in redo_entries:
|
||||
current_text = note_file.read_text(encoding="utf-8")
|
||||
meta_path = ocr_root / zotero_key / "meta.json" if ocr_root else None
|
||||
meta = read_json(meta_path) if meta_path and meta_path.exists() else {}
|
||||
status, _error = validate_ocr_meta(paths, meta) if meta else ("pending", "")
|
||||
current_text = _rewrite_note_fields(current_text, ocr_status=status)
|
||||
if status == "done":
|
||||
current_text = _rewrite_note_fields(current_text, ocr_redo=False)
|
||||
success_keys.append(zotero_key)
|
||||
else:
|
||||
current_text = _rewrite_note_fields(current_text, ocr_redo=True)
|
||||
failed_keys.append(zotero_key)
|
||||
note_file.write_text(current_text, encoding="utf-8")
|
||||
refresh_index_entry(vault, zotero_key)
|
||||
|
||||
if success_keys:
|
||||
print(f"Redo OCR done={len(success_keys)}: {', '.join(success_keys)}", flush=True)
|
||||
if failed_keys:
|
||||
print(f"Redo OCR pending/failed={len(failed_keys)}: {', '.join(failed_keys)}", flush=True)
|
||||
return exit_code
|
||||
|
||||
|
||||
def run_ocr(vault: Path, verbose: bool = False, no_progress: bool = False) -> int:
|
||||
def run_ocr(vault: Path, verbose: bool = False, no_progress: bool = False, selected_keys: set[str] | None = None) -> int:
|
||||
from paperforge.pdf_resolver import resolve_pdf_path
|
||||
|
||||
paths = pipeline_paths(vault)
|
||||
|
|
@ -1774,6 +1863,8 @@ def run_ocr(vault: Path, verbose: bool = False, no_progress: bool = False) -> in
|
|||
|
||||
control_actions = load_control_actions(paths)
|
||||
target_keys = {key for key, action in control_actions.items() if action.get("do_ocr", False)}
|
||||
if selected_keys is not None:
|
||||
target_keys &= set(selected_keys)
|
||||
|
||||
target_rows = []
|
||||
for export_path in sorted(paths["exports"].glob("*.json")):
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass
|
||||
|
|
@ -16,6 +16,14 @@ def _bbox_order_key(block: dict) -> tuple:
|
|||
return (int(bbox[1]), int(bbox[0]))
|
||||
|
||||
|
||||
def _bbox_top(block: dict) -> int:
|
||||
return int(block.get("block_bbox", [0, 0, 0, 0])[1])
|
||||
|
||||
|
||||
def _bbox_bottom(block: dict) -> int:
|
||||
return int(block.get("block_bbox", [0, 0, 0, 0])[3])
|
||||
|
||||
|
||||
def reorder_blocks_layered(
|
||||
blocks: list[dict],
|
||||
page_width: int = 0,
|
||||
|
|
@ -25,9 +33,9 @@ def reorder_blocks_layered(
|
|||
return blocks
|
||||
|
||||
try:
|
||||
from paperforge.worker.ocr_roles import assign_block_role
|
||||
from paperforge.worker.ocr_body_spine import BodySpineNode
|
||||
from paperforge.worker.ocr_layout import detect_layout_zones, order_body_spine
|
||||
from paperforge.worker.ocr_roles import assign_block_role
|
||||
except ImportError:
|
||||
return blocks
|
||||
|
||||
|
|
@ -76,5 +84,27 @@ def reorder_blocks_layered(
|
|||
if idx is not None and idx < len(body_annotated):
|
||||
body_result.append(body_annotated[idx].block)
|
||||
|
||||
non_body_result: list[dict] = [a.block for a in non_body]
|
||||
return body_result + non_body_result
|
||||
body_iter = iter(body_result)
|
||||
result: list[dict] = []
|
||||
for annotated_block in annotated:
|
||||
if annotated_block in body_annotated:
|
||||
try:
|
||||
result.append(next(body_iter))
|
||||
except StopIteration:
|
||||
result.append(annotated_block.block)
|
||||
else:
|
||||
result.append(annotated_block.block)
|
||||
result.extend(body_iter)
|
||||
|
||||
first_body_y = min((_bbox_top(block) for block in body_result), default=None)
|
||||
if first_body_y is not None:
|
||||
leading_roles = {"figure_caption", "table_caption", "media_asset"}
|
||||
leading_blocks = [
|
||||
a.block
|
||||
for a in annotated
|
||||
if a.role in leading_roles and _bbox_bottom(a.block) <= first_body_y
|
||||
]
|
||||
if leading_blocks:
|
||||
leading_ids = {id(block) for block in leading_blocks}
|
||||
result = leading_blocks + [block for block in result if id(block) not in leading_ids]
|
||||
return result
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ _TABLE_PREFIX_PATTERN = re.compile(
|
|||
)
|
||||
|
||||
_REFERENCE_PATTERN = re.compile(
|
||||
r"^\s*(?:\d+\.\s|[A-Z][a-z]+ et al\.|\([A-Z][a-z]+ et al\., \d{4}\))",
|
||||
r"^\s*(?:\d+\.\s|[A-Z][A-Za-z'’\-]+\s+et al\.\s*\(\d{4}[a-z]?\)|\([A-Z][A-Za-z'’\-]+\s+et al\.,\s*\d{4}[a-z]?\))",
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -73,6 +73,12 @@ def assign_block_role(
|
|||
|
||||
# Paddle priors
|
||||
if raw_label == "paragraph_title":
|
||||
if text.strip().lower() == "abstract":
|
||||
return RoleAssignment(
|
||||
role="frontmatter_heading",
|
||||
confidence=0.95,
|
||||
evidence=["abstract heading stays out of body reorder"],
|
||||
)
|
||||
if _has_heading_numbering(text):
|
||||
return RoleAssignment(
|
||||
role="section_heading" if re.match(r"^\d+\s", text) else "subsection_heading",
|
||||
|
|
|
|||
|
|
@ -1158,6 +1158,9 @@ def migrate_to_workspace(vault: Path, paths: dict) -> int:
|
|||
legacy_flags = _legacy_control_flags(paths, key)
|
||||
|
||||
if workspace_dir.exists():
|
||||
stale_workspace_fulltext = workspace_dir / "fulltext.md"
|
||||
if stale_workspace_fulltext.exists():
|
||||
stale_workspace_fulltext.unlink()
|
||||
if main_note_path.exists() and any(value is not None for value in legacy_flags.values()):
|
||||
try:
|
||||
workspace_content = main_note_path.read_text(encoding="utf-8")
|
||||
|
|
@ -1198,21 +1201,6 @@ def migrate_to_workspace(vault: Path, paths: dict) -> int:
|
|||
ai_dir = workspace_dir / "ai"
|
||||
ai_dir.mkdir(exist_ok=True)
|
||||
|
||||
# WS-04: Bridge OCR fulltext to workspace if available
|
||||
meta_path = paths.get("ocr", Path()) / key / "meta.json"
|
||||
if meta_path.exists():
|
||||
try:
|
||||
meta = read_json(meta_path)
|
||||
if meta.get("ocr_status") == "done":
|
||||
source_fulltext = meta_path.parent / "fulltext.md"
|
||||
target_fulltext = workspace_dir / "fulltext.md"
|
||||
if source_fulltext.exists() and not target_fulltext.exists():
|
||||
import shutil
|
||||
|
||||
shutil.copy2(str(source_fulltext), str(target_fulltext))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
migrated += 1
|
||||
|
||||
if migrated > 0:
|
||||
|
|
|
|||
|
|
@ -53,27 +53,99 @@ ocr_redo: true
|
|||
|
||||
|
||||
def test_ocr_redo_papers_updates_frontmatter(tmp_path):
|
||||
from paperforge.worker.ocr import ocr_redo_papers
|
||||
from paperforge.worker import ocr as ocr_worker
|
||||
|
||||
vault = tmp_path / "vault"
|
||||
vault.mkdir()
|
||||
literature = vault / "Resources" / "Literature" / "test_domain"
|
||||
literature.mkdir(parents=True)
|
||||
|
||||
note = literature / "good note.md"
|
||||
workspace = literature / "AB12CD34 - Test"
|
||||
workspace.mkdir()
|
||||
note = workspace / "AB12CD34.md"
|
||||
note.write_text("""---
|
||||
zotero_key: "AB12CD34"
|
||||
title: "Test"
|
||||
ocr_redo: true
|
||||
do_ocr: false
|
||||
ocr_status: done
|
||||
fulltext_md_path: "[[old_path]]"
|
||||
---
|
||||
""", encoding="utf-8")
|
||||
|
||||
rc = ocr_redo_papers(vault, dry_run=False, verbose=True)
|
||||
assert rc == 0
|
||||
(workspace / "fulltext.md").write_text("old workspace copy\n", encoding="utf-8")
|
||||
|
||||
ocr_root = vault / "System" / "PaperForge" / "ocr"
|
||||
ocr_root.mkdir(parents=True)
|
||||
ocr_dir = ocr_root / "AB12CD34"
|
||||
ocr_dir.mkdir()
|
||||
(ocr_dir / "fulltext.md").write_text("old canonical content\n", encoding="utf-8")
|
||||
|
||||
calls = {}
|
||||
|
||||
def fake_run_ocr(vault_arg, verbose=False, no_progress=False, selected_keys=None):
|
||||
calls["selected_keys"] = selected_keys
|
||||
new_dir = ocr_root / "AB12CD34"
|
||||
(new_dir / "json").mkdir(parents=True, exist_ok=True)
|
||||
fulltext = "<!-- page 1 -->\n" + ("A" * 700)
|
||||
(new_dir / "fulltext.md").write_text(fulltext, encoding="utf-8")
|
||||
(new_dir / "json" / "result.json").write_text("{" + ('"x":1,' * 600) + '"done":true}', encoding="utf-8")
|
||||
(new_dir / "meta.json").write_text(
|
||||
'{"zotero_key":"AB12CD34","ocr_status":"done","page_count":1,"fulltext_md_path":"",'
|
||||
'"markdown_path":"System/PaperForge/ocr/AB12CD34/fulltext.md","json_path":"System/PaperForge/ocr/AB12CD34/json/result.json"}',
|
||||
encoding="utf-8",
|
||||
)
|
||||
return 0
|
||||
|
||||
monkeypatch = __import__("pytest").MonkeyPatch()
|
||||
monkeypatch.setattr(ocr_worker, "run_ocr", fake_run_ocr)
|
||||
monkeypatch.setattr(ocr_worker, "refresh_index_entry", lambda _vault, _key: None)
|
||||
|
||||
try:
|
||||
rc = ocr_worker.ocr_redo_papers(vault, dry_run=False, verbose=True)
|
||||
assert rc == 0
|
||||
finally:
|
||||
monkeypatch.undo()
|
||||
|
||||
content = note.read_text(encoding="utf-8")
|
||||
assert 'ocr_redo: false' in content
|
||||
assert 'ocr_status: "pending"' in content
|
||||
assert 'ocr_status: "done"' in content
|
||||
assert 'fulltext_md_path: ""' in content
|
||||
assert 'do_ocr: true' in content
|
||||
assert calls["selected_keys"] == {"AB12CD34"}
|
||||
assert not (workspace / "fulltext.md").exists()
|
||||
|
||||
|
||||
def test_ocr_redo_papers_keeps_flag_true_when_rerun_not_done(tmp_path):
|
||||
from paperforge.worker import ocr as ocr_worker
|
||||
|
||||
vault = tmp_path / "vault"
|
||||
vault.mkdir()
|
||||
literature = vault / "Resources" / "Literature" / "test_domain"
|
||||
literature.mkdir(parents=True)
|
||||
|
||||
workspace = literature / "ZXCV1234 - Test"
|
||||
workspace.mkdir()
|
||||
note = workspace / "ZXCV1234.md"
|
||||
note.write_text("""---
|
||||
zotero_key: "ZXCV1234"
|
||||
title: "Test"
|
||||
ocr_redo: true
|
||||
do_ocr: false
|
||||
ocr_status: done
|
||||
---
|
||||
""", encoding="utf-8")
|
||||
|
||||
monkeypatch = __import__("pytest").MonkeyPatch()
|
||||
monkeypatch.setattr(ocr_worker, "run_ocr", lambda *_args, **_kwargs: 1)
|
||||
monkeypatch.setattr(ocr_worker, "refresh_index_entry", lambda _vault, _key: None)
|
||||
|
||||
try:
|
||||
rc = ocr_worker.ocr_redo_papers(vault, dry_run=False, verbose=True)
|
||||
assert rc == 1
|
||||
finally:
|
||||
monkeypatch.undo()
|
||||
|
||||
content = note.read_text(encoding="utf-8")
|
||||
assert 'ocr_redo: true' in content
|
||||
assert 'ocr_status: "pending"' in content
|
||||
|
|
|
|||
|
|
@ -404,6 +404,225 @@ def test_render_page_blocks_orders_references_within_column(tmp_path: Path) -> N
|
|||
assert ref_items == ["Amin, B. (2019).", "Cai, J. (2018).", "Bagnato, G. L. (2016).", "Bentley, G. (2012)."]
|
||||
|
||||
|
||||
def test_render_page_blocks_keeps_top_figure_before_body_and_keeps_author_et_al_sentence_in_body(tmp_path: Path) -> None:
|
||||
from paperforge.worker.ocr import render_page_blocks
|
||||
|
||||
vault = tmp_path / "vault"
|
||||
images_dir = vault / "System" / "PaperForge" / "ocr" / "KEY" / "images"
|
||||
page_cache_dir = vault / "System" / "PaperForge" / "ocr" / "KEY" / "pages"
|
||||
images_dir.mkdir(parents=True)
|
||||
page_cache_dir.mkdir(parents=True)
|
||||
|
||||
page_image = page_cache_dir / "page_008.png"
|
||||
Image.new("RGB", (1200, 1600), color="white").save(page_image)
|
||||
|
||||
result = {
|
||||
"prunedResult": {
|
||||
"width": 1200,
|
||||
"height": 1600,
|
||||
"parsing_res_list": [
|
||||
{
|
||||
"block_id": 1,
|
||||
"block_label": "figure_title",
|
||||
"block_order": 0,
|
||||
"block_bbox": [97, 114, 306, 197],
|
||||
"block_content": "Fig. 3 Schematic representation of mechanical stimulation for tissue engineering constructs",
|
||||
},
|
||||
{
|
||||
"block_id": 2,
|
||||
"block_label": "image",
|
||||
"block_order": 1,
|
||||
"block_bbox": [374, 117, 1089, 436],
|
||||
"block_content": "",
|
||||
},
|
||||
{
|
||||
"block_id": 3,
|
||||
"block_label": "paragraph_title",
|
||||
"block_order": 2,
|
||||
"block_bbox": [96, 470, 513, 520],
|
||||
"block_content": "2.5 Electrical stimulation for tissue-engineered articular cartilage",
|
||||
},
|
||||
{
|
||||
"block_id": 4,
|
||||
"block_label": "text",
|
||||
"block_order": 3,
|
||||
"block_bbox": [95, 543, 583, 918],
|
||||
"block_content": "As hyaline cartilage is an avascular tissue, the synovial fluid is responsible for chondrocyte nutrition and maintenance.",
|
||||
},
|
||||
{
|
||||
"block_id": 5,
|
||||
"block_label": "text",
|
||||
"block_order": 4,
|
||||
"block_bbox": [95, 920, 583, 1043],
|
||||
"block_content": "Regenerative pathways for cartilage regeneration on iPSCs engineered constructs can be stimulated by endogenous electrical stimulation.",
|
||||
},
|
||||
{
|
||||
"block_id": 6,
|
||||
"block_label": "text",
|
||||
"block_order": 5,
|
||||
"block_bbox": [95, 1045, 583, 1265],
|
||||
"block_content": "After incorporating ES on stem cells, this stimulus generates ATP oscillations driven by calcium oscillations.",
|
||||
},
|
||||
{
|
||||
"block_id": 7,
|
||||
"block_label": "paragraph_title",
|
||||
"block_order": 6,
|
||||
"block_bbox": [607, 893, 736, 917],
|
||||
"block_content": "3 Discussion",
|
||||
},
|
||||
{
|
||||
"block_id": 8,
|
||||
"block_label": "text",
|
||||
"block_order": 7,
|
||||
"block_bbox": [604, 943, 1094, 1341],
|
||||
"block_content": "Bioreactors can be one of the most efficient and reliable methods for testing in vitro articular cartilage-engineered constructs.",
|
||||
},
|
||||
{
|
||||
"block_id": 9,
|
||||
"block_label": "text",
|
||||
"block_order": 8,
|
||||
"block_bbox": [95, 1268, 583, 1418],
|
||||
"block_content": "Kwon et al. demonstrated that ES also drives ATP oscillations by cAMP modulation, leading to chondrogenic differentiation in the absence of exogenous growth factors.",
|
||||
},
|
||||
],
|
||||
},
|
||||
"inputImage": "",
|
||||
}
|
||||
|
||||
with patch("paperforge.worker.ocr.render_pdf_page_cached", return_value=page_image):
|
||||
rendered = render_page_blocks(vault, 8, result, images_dir, page_cache_dir, pdf_doc=None)
|
||||
|
||||
body_lines = [line for line in rendered if line and not line.startswith("<!-- page")]
|
||||
|
||||
assert body_lines[0].startswith("Fig. 3 Schematic representation")
|
||||
assert body_lines[1].startswith("![[")
|
||||
assert body_lines[2] == "### 2.5 Electrical stimulation for tissue-engineered articular cartilage"
|
||||
assert body_lines[6].startswith("Kwon et al. demonstrated")
|
||||
assert body_lines[7] == "### 3 Discussion"
|
||||
|
||||
|
||||
def test_validate_block_order_preserves_full_width_blocks_between_columns() -> None:
|
||||
from paperforge.worker.ocr import validate_block_order
|
||||
|
||||
blocks = [
|
||||
{
|
||||
"block_id": 1,
|
||||
"block_label": "abstract",
|
||||
"block_order": 6,
|
||||
"block_bbox": [96, 603, 1094, 800],
|
||||
"block_content": "BACKGROUND",
|
||||
},
|
||||
{
|
||||
"block_id": 2,
|
||||
"block_label": "abstract",
|
||||
"block_order": 7,
|
||||
"block_bbox": [97, 802, 1094, 851],
|
||||
"block_content": "METHODS",
|
||||
},
|
||||
{
|
||||
"block_id": 3,
|
||||
"block_label": "abstract",
|
||||
"block_order": 8,
|
||||
"block_bbox": [96, 899, 1093, 977],
|
||||
"block_content": "CONCLUSION",
|
||||
},
|
||||
{
|
||||
"block_id": 4,
|
||||
"block_label": "text",
|
||||
"block_order": 9,
|
||||
"block_bbox": [97, 1000, 872, 1025],
|
||||
"block_content": "Keywords",
|
||||
},
|
||||
{
|
||||
"block_id": 5,
|
||||
"block_label": "paragraph_title",
|
||||
"block_order": 10,
|
||||
"block_bbox": [98, 1062, 246, 1086],
|
||||
"block_content": "1 Introduction",
|
||||
},
|
||||
{
|
||||
"block_id": 6,
|
||||
"block_label": "text",
|
||||
"block_order": 11,
|
||||
"block_bbox": [96, 1113, 582, 1190],
|
||||
"block_content": "Introduction body",
|
||||
},
|
||||
{
|
||||
"block_id": 7,
|
||||
"block_label": "text",
|
||||
"block_order": 12,
|
||||
"block_bbox": [604, 1063, 1094, 1414],
|
||||
"block_content": "Right column body",
|
||||
},
|
||||
]
|
||||
|
||||
ordered = validate_block_order(blocks, page_width=1191)
|
||||
|
||||
assert [block["block_content"] for block in ordered[:5]] == [
|
||||
"BACKGROUND",
|
||||
"METHODS",
|
||||
"CONCLUSION",
|
||||
"Keywords",
|
||||
"1 Introduction",
|
||||
]
|
||||
|
||||
|
||||
def test_render_page_blocks_keeps_abstract_heading_before_abstract_body_on_first_page(tmp_path: Path) -> None:
|
||||
from paperforge.worker.ocr import render_page_blocks
|
||||
|
||||
vault = tmp_path / "vault"
|
||||
images_dir = vault / "System" / "PaperForge" / "ocr" / "KEY" / "images"
|
||||
page_cache_dir = vault / "System" / "PaperForge" / "ocr" / "KEY" / "pages"
|
||||
images_dir.mkdir(parents=True)
|
||||
page_cache_dir.mkdir(parents=True)
|
||||
|
||||
result = {
|
||||
"prunedResult": {
|
||||
"width": 1200,
|
||||
"height": 1600,
|
||||
"parsing_res_list": [
|
||||
{"block_id": 1, "block_label": "doc_title", "block_order": 0, "block_bbox": [96, 210, 949, 284], "block_content": "Paper Title"},
|
||||
{"block_id": 2, "block_label": "text", "block_order": 1, "block_bbox": [95, 312, 761, 365], "block_content": "Author One · Author Two"},
|
||||
{"block_id": 3, "block_label": "text", "block_order": 2, "block_bbox": [96, 485, 880, 528], "block_content": "Received: 10 February 2023"},
|
||||
{"block_id": 4, "block_label": "paragraph_title", "block_order": 3, "block_bbox": [97, 577, 181, 599], "block_content": "Abstract"},
|
||||
{"block_id": 5, "block_label": "abstract", "block_order": 4, "block_bbox": [96, 603, 1094, 800], "block_content": "BACKGROUND: Background text."},
|
||||
{"block_id": 6, "block_label": "abstract", "block_order": 5, "block_bbox": [97, 802, 1094, 851], "block_content": "METHODS: Methods text."},
|
||||
{"block_id": 7, "block_label": "abstract", "block_order": 6, "block_bbox": [96, 899, 1093, 977], "block_content": "CONCLUSION: Conclusion text."},
|
||||
{"block_id": 8, "block_label": "text", "block_order": 7, "block_bbox": [97, 1000, 872, 1025], "block_content": "Keywords one two three"},
|
||||
{"block_id": 9, "block_label": "paragraph_title", "block_order": 8, "block_bbox": [98, 1062, 246, 1086], "block_content": "1 Introduction"},
|
||||
{"block_id": 10, "block_label": "text", "block_order": 9, "block_bbox": [96, 1113, 582, 1190], "block_content": "Introduction body."},
|
||||
],
|
||||
},
|
||||
"inputImage": "",
|
||||
}
|
||||
|
||||
rendered = render_page_blocks(vault, 1, result, images_dir, page_cache_dir, pdf_doc=None)
|
||||
body_lines = [line for line in rendered if line and not line.startswith("<!-- page")]
|
||||
|
||||
abstract_idx = body_lines.index("### Abstract")
|
||||
background_idx = body_lines.index("BACKGROUND: Background text.")
|
||||
methods_idx = body_lines.index("METHODS: Methods text.")
|
||||
conclusion_idx = body_lines.index("CONCLUSION: Conclusion text.")
|
||||
keywords_idx = body_lines.index("Keywords one two three")
|
||||
intro_idx = body_lines.index("### 1 Introduction")
|
||||
|
||||
assert abstract_idx < background_idx < methods_idx < conclusion_idx < keywords_idx < intro_idx
|
||||
|
||||
|
||||
def test_media_clusters_merge_side_by_side_panels_with_small_gap() -> None:
|
||||
from paperforge.worker.ocr import media_clusters
|
||||
|
||||
blocks = [
|
||||
{"block_id": 1, "block_label": "image", "block_bbox": [366, 165, 631, 766], "block_content": ""},
|
||||
{"block_id": 2, "block_label": "image", "block_bbox": [675, 118, 1091, 738], "block_content": ""},
|
||||
]
|
||||
|
||||
block_to_cluster, clusters = media_clusters(blocks)
|
||||
|
||||
assert len(clusters) == 1
|
||||
assert block_to_cluster[1] == block_to_cluster[2]
|
||||
|
||||
|
||||
def test_embedded_figure_text_excludes_body_paragraph_like_text() -> None:
|
||||
from paperforge.worker.ocr import is_embedded_figure_text_block
|
||||
|
||||
|
|
|
|||
|
|
@ -57,3 +57,41 @@ def test_frontmatter_note_preserves_ocr_redo_true():
|
|||
result = frontmatter_note(entry, preserved_ocr_redo=True)
|
||||
assert "ocr_redo: true" in result
|
||||
assert "ocr_redo: false" not in result
|
||||
|
||||
|
||||
def test_migrate_to_workspace_deletes_existing_workspace_fulltext(tmp_path, monkeypatch):
|
||||
from paperforge.worker import sync
|
||||
|
||||
vault = tmp_path / "vault"
|
||||
literature = vault / "Resources" / "Literature"
|
||||
ocr = vault / "PaperForge" / "ocr"
|
||||
domain_dir = literature / "test_domain"
|
||||
domain_dir.mkdir(parents=True)
|
||||
|
||||
key = "ABCD1234"
|
||||
title = "Test Paper"
|
||||
flat_note = domain_dir / f"{key} - {title}.md"
|
||||
flat_note.write_text(
|
||||
"---\n"
|
||||
f'title: "{title}"\n'
|
||||
f"zotero_key: {key}\n"
|
||||
"---\n\n"
|
||||
"# Test Paper\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
workspace_dir = domain_dir / f"{key} - {title}"
|
||||
workspace_dir.mkdir()
|
||||
(workspace_dir / f"{key}.md").write_text("# Existing note\n", encoding="utf-8")
|
||||
(workspace_dir / "fulltext.md").write_text("old workspace fulltext\n", encoding="utf-8")
|
||||
|
||||
monkeypatch.setattr(sync.asset_index, "read_index", lambda _vault: {"items": []})
|
||||
|
||||
paths = {
|
||||
"literature": literature,
|
||||
"ocr": ocr,
|
||||
}
|
||||
|
||||
sync.migrate_to_workspace(vault, paths)
|
||||
|
||||
assert not (workspace_dir / "fulltext.md").exists()
|
||||
|
|
|
|||
Loading…
Reference in a new issue