obsidian-plugin/lllin000_PaperForge
1
0
Fork 0
mirror of https://github.com/lllin000/PaperForge.git synced 2026-07-22 06:50:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

fix: VaultStep.__init__ missing step_id and checker in super() call

Browse source
This commit is contained in:
Research Assistant 2026-04-27 19:49:36 +08:00
parent c2a29a27d9
commit 67a52a898a
59 changed files with 1511 additions and 1098 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
.planning/phases/01-config-and-command-foundation/01-CONTEXT.md
View file

@ -1,7 +1,7 @@
# Phase 1: Config And Command Foundation - Context
**Gathered:** 2026-04-23
**Status:** Ready for planning
**Gathered:** 2026-04-23
**Status:** Ready for planning
**Source:** Derived from `$gsd-new-project` research and Phase 1 roadmap.
<domain>
@ -160,5 +160,5 @@ python <system_dir>/PaperForge/worker/scripts/literature_pipeline.py --vault <va
---
*Phase: 01-config-and-command-foundation*
*Phase: 01-config-and-command-foundation*
*Context gathered: 2026-04-23 via derived planning context*

2
.planning/phases/01-config-and-command-foundation/01-VERIFICATION.md
View file

@ -113,7 +113,7 @@ None. All Phase 1 criteria are verifiable programmatically.
None. Phase 1 goal fully achieved:
- `paperforge/config.py` is the single tested contract for path resolution, containing no OCR doctor, PDF resolver, or Base redesign behavior (per Plan 01 success criteria)
- CLI launcher has package entry point and `python -m paperforge` fallback (per Plan 02 success criteria)
- CLI launcher has package entry point and `python -m paperforge` fallback (per Plan 02 success criteria)
- Worker, `/LD-deep`, setup wizard, and validation all consume the same resolver; copied installations deploy the package (per Plan 03 success criteria)
- User-facing docs use stable `paperforge ...` commands; unresolved `<system_dir>` path tokens removed from user-run examples; legacy invocation documented as fallback (per Plan 04 success criteria)
- All 8 requirement IDs satisfied

2
.planning/phases/04-onboarding-validation/04-02-PLAN.md
View file

@ -137,7 +137,7 @@ Add `paperforge doctor` as a standalone subcommand that reports setup readiness
```
PaperForge Lite Doctor
======================
[PASS] Python 环境 — Python 3.10+
[FAIL] Vault 结构 — system_dir 不存在
[WARN] Zotero 链接 — Zotero 目录是普通文件夹，不是 junction

2
.planning/phases/05-workflow-hardening-and-optional-plugin-shell/05-01-PLAN.md
View file

@ -129,7 +129,7 @@ def test_pending_to_processing_transition(tmp_path, mock_paddleocr_api):
# Setup: create ocr-queue.json with pending job
# Call: run_ocr(vault)
# Assert: job state changed to processing
def test_processing_to_done_transition(tmp_path, mock_paddleocr_api):
# Setup: ocr-queue.json with processing job
# Mock polling returns done with result_url

2
.planning/phases/05-workflow-hardening-and-optional-plugin-shell/05-02-PLAN.md
View file

@ -71,7 +71,7 @@ def mock_vault(tmp_path):
From paperforge/cli.py (commands to smoke test):
```python
# paperforge doctor -> run_doctor(vault)
# paperforge selection-sync -> run_selection_sync(vault)
# paperforge selection-sync -> run_selection_sync(vault)
# paperforge index-refresh -> run_index_refresh(vault)
# paperforge ocr doctor -> _cmd_ocr_doctor(vault, args)
# paperforge deep-reading -> run_deep_reading(vault, verbose=False)

2
.planning/phases/07-zotero-pdf-metadata-state-repair/07-REVIEW.md
View file

@ -124,7 +124,7 @@ This is implemented correctly for the first fix branch (lines 2934-2964). Howeve
The plan specifies three repair scenarios (lines 135-138):
1. Missing meta files → set pending + do_ocr
2. Incomplete meta → set pending + do_ocr
2. Incomplete meta → set pending + do_ocr
3. library done but meta pending → set library to pending only
The implementation adds `do_ocr: true` to scenario 3 as well. This is likely harmless (setting do_ocr alongside the status reset makes sense to retrigger OCR), but deviates from the plan.

8
.planning/phases/07-zotero-pdf-metadata-state-repair/07-SUMMARY.md
View file

@ -1,9 +1,9 @@
# Phase 7 Summary — Zotero PDF, Metadata, And State Repair
**Status:** Complete (with one task deferred)
**Date:** 2026-04-24
**Plan:** [07-PLAN.md](07-PLAN.md)
**Review:** [07-REVIEW.md](07-REVIEW.md)
**Status:** Complete (with one task deferred)
**Date:** 2026-04-24
**Plan:** [07-PLAN.md](07-PLAN.md)
**Review:** [07-REVIEW.md](07-REVIEW.md)
---

12
.planning/phases/09-command-unification/09-SUMMARY.md
View file

@ -1,10 +1,10 @@
# Phase 09 Plan: Command Unification & CLI Simplification — Summary
**Phase:** 9
**Plan:** Command Unification & CLI Simplification
**Status:** COMPLETE
**Completed:** 2026-04-24
**Duration:** ~3 hours (across 6 tasks)
**Phase:** 9
**Plan:** Command Unification & CLI Simplification
**Status:** COMPLETE
**Completed:** 2026-04-24
**Duration:** ~3 hours (across 6 tasks)
**Milestone:** v1.2 Systematization & Cohesion
---
@ -180,5 +180,5 @@ Phase 10: Documentation & Cohesion
---
*Summary generated by VT-OS/OPENCODE Terminal on 2026-04-24*
*Summary generated by VT-OS/OPENCODE Terminal on 2026-04-24*
*Vault-Tec — Preparing for the Future!*

2
.planning/phases/10-documentation-and-cohesion/10-CONTEXT.md
View file

@ -65,7 +65,7 @@ Phase 10 will implement OpenCode support and document approach for others.
- No old command names (`selection-sync`, `index-refresh`, `ocr run`, `/LD-*`, `/lp-*`)
- No dead links
- No broken references to `paperforge_lite`
- **Manual checklist** for soft constraints:
- Terminology consistency
- Style consistency

8
.planning/phases/10-documentation-and-cohesion/10-SUMMARY.md
View file

@ -1,9 +1,9 @@
# Phase 10: Documentation & Cohesion — Summary
**Phase:** 10
**Plan:** 10
**Status:** COMPLETE
**Completed:** 2026-04-24
**Phase:** 10
**Plan:** 10
**Status:** COMPLETE
**Completed:** 2026-04-24
**Milestone:** v1.2 Systematization & Cohesion
---

6
.planning/phases/11-zotero-path-normalization/11-PLAN.md
View file

@ -92,15 +92,15 @@ Implement robust Zotero attachment path parsing from real-world BBT JSON exports
**Priority 1 (Primary):**
- `attachment.title == "PDF"` AND `attachment.contentType == "application/pdf"`
**Priority 2 (Fallback heuristic):**
- Filter to `contentType == "application/pdf"` items only
- Select the LARGEST file by size (if size field available)
- If sizes are equal or unavailable, select the one with shortest title (main PDFs often have simple titles)
**Priority 3 (Final fallback):**
- First PDF attachment in the list
**Returns:** `(main_pdf_attachment, supplementary_attachments)` where supplementary is a list of all other PDF attachments
2. In `load_export_rows()`, when processing each item:

8
.planning/phases/11-zotero-path-normalization/11-VERIFICATION.md
View file

@ -1,9 +1,9 @@
# Phase 11 Wave 4 Verification Report
**Phase:** 11
**Plan:** 01
**Wave:** 4 of 4
**Date:** 2026-04-24
**Phase:** 11
**Plan:** 01
**Wave:** 4 of 4
**Date:** 2026-04-24
**Status:** COMPLETE
---

6
.planning/phases/12-architecture-cleanup/12-CONTEXT.md
View file

@ -1,8 +1,8 @@
# Phase 12: Architecture Cleanup - Context
**Gathered:** 2026-04-24
**Status:** Ready for planning
**Base Commit:** 4297dcd
**Gathered:** 2026-04-24
**Status:** Ready for planning
**Base Commit:** 4297dcd
---

6
.planning/phases/12-architecture-cleanup/12-DISCUSSION-LOG.md
View file

@ -1,8 +1,8 @@
# Phase 12: Architecture Cleanup - Discussion Log
**Phase:** 12
**Topic:** 修复模块边界泄漏，消除测试死区
**Base Commit:** 4297dcd
**Phase:** 12
**Topic:** 修复模块边界泄漏，消除测试死区
**Base Commit:** 4297dcd
---

6
.planning/phases/12-architecture-cleanup/12-PLAN.md
View file

@ -1,8 +1,8 @@
# Phase 12: Architecture Cleanup - Execution Plan
**Plan Date:** 2026-04-24
**Base Commit:** 4297dcd
**Phase Goal:** 修复模块边界泄漏，消除测试死区
**Plan Date:** 2026-04-24
**Base Commit:** 4297dcd
**Phase Goal:** 修复模块边界泄漏，消除测试死区
---

2
command/pf-ocr.md
View file

@ -120,7 +120,7 @@ OCR 完成后，每个文献生成以下文件：
### OpenCode
> `/pf-ocr` 是 **CLI 命令**，Agent 层不直接提供 `/pf-ocr` 聊天命令。
>
>
> 用户需要：
> 1. 在 Obsidian 中将 library-record 的 `do_ocr` 设为 `true`
> 2. 在终端运行 `paperforge ocr`

1
docs/INSTALLATION.md
View file

@ -146,4 +146,3 @@ python -m paperforge <command>
4. **开始精读**：使用 `/pf-deep <zotero_key>` 生成结构化阅读笔记
详细用法参见 [AGENTS.md](../AGENTS.md)。

5
docs/setup-guide.md
View file

@ -1,7 +1,7 @@
# PaperForge Lite 安装与配置指南
> 本文档是 [setup_wizard.py](../setup_wizard.py) 的补充说明。如果 TUI 向导中的文字指引不够直观，请参考本页的详细步骤。
>
>
> 本文档中的截图保存在 `docs/images/` 目录下，向导中点击"查看安装截图"按钮可直接打开。
---
@ -225,6 +225,3 @@ python setup_wizard.py --vault .
---
*PaperForge Lite | 安装指南*

1
paperforge/__main__.py
View file

@ -1,6 +1,7 @@
"""paperforge.__main__ — entry point for `python -m paperforge`."""
import sys
from paperforge.cli import main
if __name__ == "__main__":

55
paperforge/cli.py
View file

@ -15,22 +15,21 @@ from __future__ import annotations
import argparse
import json
import os
import sys
from pathlib import Path
# Logging
from paperforge.logging_config import configure_logging
# Config / resolver
from paperforge.config import (
load_simple_env,
load_vault_config,
resolve_vault,
paperforge_paths,
paths_as_strings,
resolve_vault,
)
# Logging
from paperforge.logging_config import configure_logging
# Worker function stubs — let tests patch cli.run_* directly
run_status = None
run_selection_sync = None
@ -89,12 +88,13 @@ def _import_worker_functions() -> None:
global run_status, run_selection_sync, run_index_refresh
global run_deep_reading, run_repair, run_ocr, ensure_base_views
from paperforge.worker.status import run_status as _rs
from paperforge.worker.sync import run_selection_sync as _rss, run_index_refresh as _rir
from paperforge.worker.deep_reading import run_deep_reading as _rdr
from paperforge.worker.repair import run_repair as _rr
from paperforge.worker.ocr import run_ocr as _ro
from paperforge.worker.base_views import ensure_base_views as _ebu
from paperforge.worker.deep_reading import run_deep_reading as _rdr
from paperforge.worker.ocr import run_ocr as _ro
from paperforge.worker.repair import run_repair as _rr
from paperforge.worker.status import run_status as _rs
from paperforge.worker.sync import run_index_refresh as _rir
from paperforge.worker.sync import run_selection_sync as _rss
if run_status is None:
run_status = _rs
@ -126,10 +126,16 @@ def build_parser() -> argparse.ArgumentParser:
help="Path to the Obsidian vault root (default: cwd or PAPERFORGE_VAULT env)",
)
parser.add_argument(
"--verbose", "-v",
"--verbose",
"-v",
action="store_true",
help="Enable DEBUG-level diagnostic output on stderr",
)
parser.add_argument(
"--no-progress",
action="store_true",
help="Disable progress bars (tqdm) for all commands",
)
sub = parser.add_subparsers(dest="command", required=True)
@ -174,18 +180,12 @@ def build_parser() -> argparse.ArgumentParser:
sub.add_parser("index-refresh", help="Refresh formal literature notes from library records")
# deep-reading
p_dr = sub.add_parser("deep-reading", help="Check deep-reading queue status")
sub.add_parser("deep-reading", help="Check deep-reading queue status")
# repair
p_repair = sub.add_parser("repair", help="Repair divergent literature notes")
p_repair.add_argument(
"--fix", action="store_true",
help="Actually apply repairs instead of dry-run"
)
p_repair.add_argument(
"--fix-paths", action="store_true",
help="Re-resolve PDF paths for items with path_error"
)
p_repair.add_argument("--fix", action="store_true", help="Actually apply repairs instead of dry-run")
p_repair.add_argument("--fix-paths", action="store_true", help="Re-resolve PDF paths for items with path_error")
# ocr (unified)
p_ocr = sub.add_parser("ocr", help="OCR operations")
@ -207,8 +207,10 @@ def build_parser() -> argparse.ArgumentParser:
# base-refresh
p_base = sub.add_parser("base-refresh", help="Refresh Obsidian Base view files")
p_base.add_argument(
"--force", "-f", action="store_true",
help="Force full regeneration (bypasses incremental merge, replaces all views including user views)"
"--force",
"-f",
action="store_true",
help="Force full regeneration (bypasses incremental merge, replaces all views including user views)",
)
# doctor
@ -292,6 +294,7 @@ def main(argv: list[str] | None = None) -> int:
# New unified commands
if args.command == "sync":
from paperforge.commands import sync
return sync.run(args)
# OCR — handle both new unified and old subcommand styles
@ -302,17 +305,20 @@ def main(argv: list[str] | None = None) -> int:
return _cmd_ocr_doctor(vault, args)
# New unified ocr (or ocr run)
from paperforge.commands import ocr
return ocr.run(args)
# Backward compat: old selection-sync and index-refresh
if args.command == "selection-sync":
from paperforge.commands import sync
args.selection = True
args.index = False
return sync.run(args)
if args.command == "index-refresh":
from paperforge.commands import sync
args.selection = False
args.index = True
return sync.run(args)
@ -320,14 +326,17 @@ def main(argv: list[str] | None = None) -> int:
# Other commands delegate to their modules
if args.command == "status":
from paperforge.commands import status
return status.run(args)
if args.command == "deep-reading":
from paperforge.commands import deep
return deep.run(args)
if args.command == "repair":
from paperforge.commands import repair
return repair.run(args)
if args.command == "base-refresh":
@ -341,10 +350,12 @@ def main(argv: list[str] | None = None) -> int:
if args.command == "doctor":
from paperforge.worker.status import run_doctor
return run_doctor(vault)
if args.command == "update":
from paperforge.worker.update import run_update
return run_update(vault)
print(f"Error: unknown command {args.command}", file=sys.stderr)

2
paperforge/commands/ocr.py
View file

@ -74,7 +74,7 @@ def run(args: argparse.Namespace) -> int:
logger.info("Processing specific key: %s", key)
run_ocr = _get_run_ocr()
exit_code = run_ocr(vault, verbose=getattr(args, "verbose", False))
exit_code = run_ocr(vault, verbose=getattr(args, "verbose", False), no_progress=getattr(args, "no_progress", False))
# Auto-diagnose after successful run (new unified behavior)
if exit_code == 0 and ocr_action is None and not diagnose_only and not key:

11
paperforge/commands/repair.py
View file

@ -32,7 +32,7 @@ def run(args: argparse.Namespace) -> int:
vault = getattr(args, "vault_path", None)
paths = getattr(args, "paths", None)
if vault is None:
from paperforge.config import resolve_vault, paperforge_paths, load_vault_config
from paperforge.config import load_vault_config, paperforge_paths, resolve_vault
vault = resolve_vault(cli_vault=getattr(args, "vault", None))
cfg = load_vault_config(vault)
@ -49,13 +49,8 @@ def run(args: argparse.Namespace) -> int:
# Report path_error summary from repair scan
path_errors = result.get("path_errors", {})
if path_errors.get("total", 0) > 0:
error_summary = ", ".join(
f"{count} {err}"
for err, count in sorted(path_errors.get("by_type", {}).items())
)
print(
f"repair: Found {path_errors['total']} items with path_error: {error_summary}"
)
error_summary = ", ".join(f"{count} {err}" for err, count in sorted(path_errors.get("by_type", {}).items()))
print(f"repair: Found {path_errors['total']} items with path_error: {error_summary}")
# Return non-zero if any divergences or path_errors remain
has_issues = bool(result.get("divergent")) or path_errors.get("total", 0) > 0
return 1 if has_issues else 0

5
paperforge/config.py
View file

@ -22,6 +22,7 @@ from typing import Any
# .env loader
# ---------------------------------------------------------------------------
def load_simple_env(env_path: Path) -> None:
"""Load key=value pairs from a .env file into os.environ (no overwrite)."""
if not env_path.exists():
@ -74,6 +75,7 @@ CONFIG_KEYS: set[str] = set(DEFAULT_CONFIG.keys())
# JSON reader
# ---------------------------------------------------------------------------
def read_paperforge_json(vault: Path) -> dict[str, Any]:
"""Read and parse paperforge.json, returning raw key-value pairs.
@ -97,6 +99,7 @@ def read_paperforge_json(vault: Path) -> dict[str, Any]:
# Vault resolution
# ---------------------------------------------------------------------------
def resolve_vault(
cli_vault: Path | None = None,
env: dict[str, str] | None = None,
@ -142,6 +145,7 @@ def resolve_vault(
# Config loading
# ---------------------------------------------------------------------------
def load_vault_config(
vault: Path,
env: dict[str, str] | None = None,
@ -205,6 +209,7 @@ def load_vault_config(
# Path construction
# ---------------------------------------------------------------------------
def paperforge_paths(
vault: Path,
cfg: dict[str, str] | None = None,

9
paperforge/pdf_resolver.py
View file

@ -2,6 +2,7 @@
Supports absolute, vault-relative, junction/symlink, and Zotero storage-relative paths.
"""
from __future__ import annotations
import ctypes
@ -58,7 +59,7 @@ def resolve_pdf_path(
# Try Zotero storage-relative (format: "storage:XXXX/item.pdf")
if raw.startswith("storage:") and zotero_dir is not None:
storage_rel = raw[len("storage:"):].lstrip("/")
storage_rel = raw[len("storage:") :].lstrip("/")
storage_candidate = (zotero_dir / storage_rel.replace("/", os.sep)).resolve()
if is_valid_pdf(storage_candidate):
return str(storage_candidate)
@ -83,13 +84,9 @@ def resolve_junction(path: Path) -> Path:
# Windows-specific: resolve directory junction reparse points
if os.name == "nt" and path.is_dir():
try:
from ctypes import wintypes
kernel32 = ctypes.windll.kernel32
buf = ctypes.create_unicode_buffer(1024)
handle = kernel32.CreateFileW(
str(path), 0, 0, None, 3, 0x02000000, None
)
handle = kernel32.CreateFileW(str(path), 0, 0, None, 3, 0x02000000, None)
if handle != -1:
try:
res = kernel32.GetFinalPathNameByHandleW(handle, buf, 1024, 0)

1
paperforge/skills/literature-qa/prompt_deep_subagent.md
View file

@ -295,4 +295,3 @@ python {{SCRIPT}} validate-note "<note_path>" --fulltext "<fulltext_path>"
# 列出待精读队列（信息用）
python {{SCRIPT}} queue --vault "{{VAULT}}" --format table
```

461
paperforge/skills/literature-qa/scripts/ld_deep.py
View file

@ -6,9 +6,9 @@ from __future__ import annotations
import argparse
import json
import re
from collections.abc import Iterable
from dataclasses import dataclass, field
from pathlib import Path
from typing import Iterable
from paperforge.worker._utils import scan_library_records
@ -19,6 +19,7 @@ def _load_vault_config(vault: Path) -> dict:
Preserves the public name for legacy callers.
"""
from paperforge.config import load_vault_config as _shared_load_vault_config
return _shared_load_vault_config(vault)
@ -177,7 +178,9 @@ def extract_figures_from_fulltext(fulltext: str, figure_map: dict | None = None)
FigureEntry(
number=mapped.get("number", "?"),
image_id=image_id,
title=mapped.get("caption", "")[:80] + "..." if len(mapped.get("caption", "")) > 80 else mapped.get("caption", "[Figure]"),
title=mapped.get("caption", "")[:80] + "..."
if len(mapped.get("caption", "")) > 80
else mapped.get("caption", "[Figure]"),
image_link=pending_image,
page=mapped.get("page", pending_page),
caption=mapped.get("caption", ""),
@ -269,11 +272,13 @@ def extract_tables_from_fulltext(fulltext: str, figure_map: dict | None = None)
# Try figure_map first
mapped = map_lookup.get(image_id)
if mapped:
tables.append(TableEntry(
number=mapped.get("number", str(len(tables) + 1)),
image_link=image_link,
page=mapped.get("page", pending_page),
))
tables.append(
TableEntry(
number=mapped.get("number", str(len(tables) + 1)),
image_link=image_link,
page=mapped.get("page", pending_page),
)
)
else:
# Fallback: cannot determine real table number from filename
# The number in filename (e.g. 75 in page_024_table_75...) is an OCR internal ID, NOT the table number
@ -288,88 +293,241 @@ def extract_tables_from_fulltext(fulltext: str, figure_map: dict | None = None)
CHART_TYPE_KEYWORDS: dict[str, list[str]] = {
"条形图与误差棒": [
"bar", "histogram", "column", "quantification", "quantitative",
"relative expression", "mRNA expression", "protein level",
"BMD", "BV/TV", "Tb.N", "Tb.Sp", "intensity", "score",
"ratio", "percentage", "proportion", "fold change",
"bar",
"histogram",
"column",
"quantification",
"quantitative",
"relative expression",
"mRNA expression",
"protein level",
"BMD",
"BV/TV",
"Tb.N",
"Tb.Sp",
"intensity",
"score",
"ratio",
"percentage",
"proportion",
"fold change",
],
"折线图与时间序列": [
"curve", "time", "kinetics", "release", "cyclic", "stability",
"degradation", "profile", "over time", "day", "week",
"duration", "period", "cycle", "repeated",
"curve",
"time",
"kinetics",
"release",
"cyclic",
"stability",
"degradation",
"profile",
"over time",
"day",
"week",
"duration",
"period",
"cycle",
"repeated",
],
"热图与聚类图": [
"heatmap", "heat map", "clustering", "cluster", "dendrogram",
"hierarchical", "correlation matrix", "expression matrix",
"heatmap",
"heat map",
"clustering",
"cluster",
"dendrogram",
"hierarchical",
"correlation matrix",
"expression matrix",
],
"火山图与曼哈顿图": [
"volcano", "volcano plot", "manhattan", "-log10", "log2fc",
"fold change", "differential", "DEG", "DEM", "upregulated", "downregulated",
"volcano",
"volcano plot",
"manhattan",
"-log10",
"log2fc",
"fold change",
"differential",
"DEG",
"DEM",
"upregulated",
"downregulated",
],
"免疫荧光定量图": [
"immunofluorescence", "immunofluorescent", "confocal", "fluorescence",
"staining", "3D reconstruction", "overlay", "DAPI", " Alexa ",
"FITC", "TRITC", "Cy3", "Cy5", "SOX9", "COL2A1", "Piezo1",
"positive cells", "mean intensity", "fluorescent",
"immunofluorescence",
"immunofluorescent",
"confocal",
"fluorescence",
"staining",
"3D reconstruction",
"overlay",
"DAPI",
" Alexa ",
"FITC",
"TRITC",
"Cy3",
"Cy5",
"SOX9",
"COL2A1",
"Piezo1",
"positive cells",
"mean intensity",
"fluorescent",
],
"组织学半定量图": [
"H&E", "hematoxylin", "eosin", "Safranin", "Fast Green",
"Masson", "trichrome", "histology", "histological",
"tissue section", "slide", "stain", "morphology",
"O'Driscoll", "ICRS", "Mankin", "OARSI", "Pineda",
"H&E",
"hematoxylin",
"eosin",
"Safranin",
"Fast Green",
"Masson",
"trichrome",
"histology",
"histological",
"tissue section",
"slide",
"stain",
"morphology",
"O'Driscoll",
"ICRS",
"Mankin",
"OARSI",
"Pineda",
],
"桑基图与弦图": [
"chord", "Sankey", "correlation network", "interaction network",
"signaling network", "proximity", "correlation of metabolic",
"Spearman", "Pearson correlation",
"chord",
"Sankey",
"correlation network",
"interaction network",
"signaling network",
"proximity",
"correlation of metabolic",
"Spearman",
"Pearson correlation",
],
"雷达图与漏斗图": [
"radar", "funnel", "spider", "web chart", "polar",
"radar",
"funnel",
"spider",
"web chart",
"polar",
],
"GSEA富集图": [
"GSEA", "MSEA", "enrichment", "enrichment plot", "enrichment score",
"NES", "leading edge", "pathway enrichment", "metabolic set",
"GSEA",
"MSEA",
"enrichment",
"enrichment plot",
"enrichment score",
"NES",
"leading edge",
"pathway enrichment",
"metabolic set",
],
"箱式图与小提琴图": [
"box", "boxplot", "box plot", "violin", "whisker", "quartile",
"median", "IQR", "outlier",
"box",
"boxplot",
"box plot",
"violin",
"whisker",
"quartile",
"median",
"IQR",
"outlier",
],
"散点图与气泡图": [
"scatter", "bubble", "dot plot", "correlation plot", "regression",
"linear fit", "R²", "correlation coefficient",
"scatter",
"bubble",
"dot plot",
"correlation plot",
"regression",
"linear fit",
"R²",
"correlation coefficient",
],
"Western Blot条带图": [
"Western", "blot", "WB", "gel electrophoresis", "SDS-PAGE",
"band", "molecular weight", "kDa",
"Western",
"blot",
"WB",
"gel electrophoresis",
"SDS-PAGE",
"band",
"molecular weight",
"kDa",
],
"显微照片与SEM图": [
"SEM", "TEM", "microscopy", "micrograph", "morphology",
"surface", "topography", "nanostructure", "porous",
"FE-SEM", "scanning electron", "transmission electron",
"SEM",
"TEM",
"microscopy",
"micrograph",
"morphology",
"surface",
"topography",
"nanostructure",
"porous",
"FE-SEM",
"scanning electron",
"transmission electron",
],
"降维图(PCA-tSNE-UMAP)": [
"PCA", "t-SNE", "tSNE", "UMAP", "MDS", "dimensionality reduction",
"principal component", "clustering plot", "embedding",
"PCA",
"t-SNE",
"tSNE",
"UMAP",
"MDS",
"dimensionality reduction",
"principal component",
"clustering plot",
"embedding",
],
"网络图与通路图": [
"pathway", "network", "signaling pathway", "KEGG", "Reactome",
"protein-protein interaction", "PPI", "regulatory network",
"pathway",
"network",
"signaling pathway",
"KEGG",
"Reactome",
"protein-protein interaction",
"PPI",
"regulatory network",
],
"蛋白质结构图": [
"protein structure", "crystallography", "NMR structure", "AlphaFold",
"3D structure", "homology modeling", "docking",
"protein structure",
"crystallography",
"NMR structure",
"AlphaFold",
"3D structure",
"homology modeling",
"docking",
],
"森林图与Meta分析": [
"forest plot", "meta-analysis", "meta analysis", "pooled effect",
"heterogeneity", "I²", "Egger", "funnel plot",
"forest plot",
"meta-analysis",
"meta analysis",
"pooled effect",
"heterogeneity",
"I²",
"Egger",
"funnel plot",
],
"ROC与PR曲线": [
"ROC", "AUC", "receiver operating", "sensitivity", "specificity",
"PR curve", "precision-recall", "diagnostic accuracy",
"ROC",
"AUC",
"receiver operating",
"sensitivity",
"specificity",
"PR curve",
"precision-recall",
"diagnostic accuracy",
],
"生存曲线": [
"survival", "Kaplan-Meier", "KM curve", "log-rank", "hazard ratio",
"HR", "OS", "PFS", "DFS", "mortality",
"survival",
"Kaplan-Meier",
"KM curve",
"log-rank",
"hazard ratio",
"HR",
"OS",
"PFS",
"DFS",
"mortality",
],
}
@ -440,6 +598,7 @@ def build_chart_type_map(figure_map: dict) -> dict:
return result
CAPTION_PATTERNS = {
"main_figure": re.compile(
r"^(?:Figure|Fig\.?)\s*(\d+[a-zA-Z]?)(?:\s*[\.:|\-]?\s*)(.*?)$",
@ -473,30 +632,32 @@ def build_figure_map(fulltext: str, zotero_key: str = "") -> dict:
# First pass: collect all images with their page and line index
all_images: list[dict] = []
current_page: int | None = None
for idx, raw_line in enumerate(lines):
line = raw_line.strip()
if not line:
continue
page_match = page_pattern.match(line)
if page_match:
current_page = int(page_match.group(1))
continue
image_match = image_pattern.match(line)
if image_match:
all_images.append({
"link": image_match.group(1),
"id": Path(image_match.group(1)).stem,
"line_idx": idx,
"page": current_page,
})
all_images.append(
{
"link": image_match.group(1),
"id": Path(image_match.group(1)).stem,
"line_idx": idx,
"page": current_page,
}
)
# Second pass: match captions to nearest images
entries: list[dict] = []
current_page = None
for idx, raw_line in enumerate(lines):
line = raw_line.strip()
if not line:
@ -517,8 +678,8 @@ def build_figure_map(fulltext: str, zotero_key: str = "") -> dict:
# Find nearest image within window of adjacent pages (current ± 2 pages)
best_image = None
min_distance = float('inf')
min_distance = float("inf")
for img in all_images:
# Check if image is within 2 pages of current page
if current_page is not None and img["page"] is not None:
@ -541,16 +702,18 @@ def build_figure_map(fulltext: str, zotero_key: str = "") -> dict:
if line_diff <= 5: # Within 5 lines
additional_images.append({"id": img["id"], "link": img["link"]})
entries.append({
"number": number,
"label": line.split(".")[0] if "." in line else line.split("|")[0].strip(),
"page": current_page,
"type": entry_type,
"caption": caption_text,
"image_link": best_image["link"] if best_image else None,
"image_id": best_image["id"] if best_image else None,
"additional_images": additional_images,
})
entries.append(
{
"number": number,
"label": line.split(".")[0] if "." in line else line.split("|")[0].strip(),
"page": current_page,
"type": entry_type,
"caption": caption_text,
"image_link": best_image["link"] if best_image else None,
"image_id": best_image["id"] if best_image else None,
"additional_images": additional_images,
}
)
break # one caption per line
# Deduplicate by (type, number) keeping first
@ -596,7 +759,9 @@ def render_study_scaffold(figures: Iterable[FigureEntry], tables: Iterable[Table
# Build figure blocks using the standard renderer for consistency
figure_blocks = [render_figure_block(fig) for fig in figure_list]
figure_section = "\n\n".join(figure_blocks) if figure_blocks else "##### Figure 待补充\n\n- 暂未从 OCR 中解析到可用主图。"
figure_section = (
"\n\n".join(figure_blocks) if figure_blocks else "##### Figure 待补充\n\n- 暂未从 OCR 中解析到可用主图。"
)
# Build table blocks using the standard renderer for consistency
table_blocks = [render_table_block(table) for table in table_list]
@ -679,29 +844,31 @@ def render_figure_block(figure: FigureEntry) -> str:
# Add additional images for multi-image figures
for add_img in figure.additional_images:
lines.append(f"> ![[{add_img['link']}]]")
lines.extend([
">",
"> **图像定位与核心问题**",
f"> - 页码：{page_suffix or '待补充'}",
"> - 这张图要回答什么：",
"> - （待补充）",
">",
"> **方法与结果**",
"> - 方法：",
"> - 结果：",
">",
"> **作者解释**",
"> - （待补充）",
">",
"> **我的理解**",
"> - （待补充）",
">",
"> **在全文中的作用**",
"> - （待补充）",
">",
"> **疑点 / 局限**",
"> - （待补充）",
])
lines.extend(
[
">",
"> **图像定位与核心问题**",
f"> - 页码：{page_suffix or '待补充'}",
"> - 这张图要回答什么：",
"> - （待补充）",
">",
"> **方法与结果**",
"> - 方法：",
"> - 结果：",
">",
"> **作者解释**",
"> - （待补充）",
">",
"> **我的理解**",
"> - （待补充）",
">",
"> **在全文中的作用**",
"> - （待补充）",
">",
"> **疑点 / 局限**",
"> - （待补充）",
]
)
return "\n".join(lines) + "\n\n"
@ -722,7 +889,9 @@ def render_table_block(table: TableEntry) -> str:
return "\n".join(lines) + "\n\n"
def validate_selected_blocks(note_text: str, figures: Iterable[FigureEntry], tables: Iterable[TableEntry] | None = None) -> list[str]:
def validate_selected_blocks(
note_text: str, figures: Iterable[FigureEntry], tables: Iterable[TableEntry] | None = None
) -> list[str]:
"""Return missing selected figure/table embeds that must be repaired before generation."""
missing: list[str] = []
for figure in figures:
@ -755,7 +924,7 @@ def validate_callout_structure(note_text: str, figures: Iterable[FigureEntry]) -
figure_heading_prefix = f"> [!note]- Figure {figure.number}："
# Find the line that starts with this prefix
figure_idx = -1
for i, line in enumerate(note_text.splitlines()):
for _i, line in enumerate(note_text.splitlines()):
if line.strip().startswith(figure_heading_prefix):
figure_idx = note_text.find(line)
break
@ -767,7 +936,7 @@ def validate_callout_structure(note_text: str, figures: Iterable[FigureEntry]) -
next_section = figure_block.find("\n#### ", 1)
end_positions = [p for p in [next_callout, next_heading, next_section] if p != -1]
if end_positions:
figure_block = figure_block[:min(end_positions)]
figure_block = figure_block[: min(end_positions)]
required_local = [
"**作者解释**",
"**我的理解**",
@ -932,13 +1101,19 @@ def _ensure_section_with_blocks(note_text: str, header: str, blocks: list[str])
return updated
def ensure_study_section(note_text: str, figures: Iterable[FigureEntry], tables: Iterable[TableEntry] | None = None) -> str:
def ensure_study_section(
note_text: str, figures: Iterable[FigureEntry], tables: Iterable[TableEntry] | None = None
) -> str:
"""Append the study scaffold if it does not yet exist."""
figure_list = list(figures)
table_list = list(tables or [])
if STUDY_HEADER in note_text:
updated = _ensure_section_with_blocks(note_text, FIGURE_SECTION_HEADER, [render_figure_block(fig) for fig in figure_list])
updated = _ensure_section_with_blocks(updated, TABLE_SECTION_HEADER, [render_table_block(table) for table in table_list])
updated = _ensure_section_with_blocks(
note_text, FIGURE_SECTION_HEADER, [render_figure_block(fig) for fig in figure_list]
)
updated = _ensure_section_with_blocks(
updated, TABLE_SECTION_HEADER, [render_table_block(table) for table in table_list]
)
return updated
stripped = note_text.rstrip()
@ -1023,7 +1198,7 @@ def prepare_deep_reading(vault: Path, zotero_key: str, force: bool = False) -> d
record_text = record_path.read_text(encoding="utf-8")
# 2. Check analyze flag
analyze_match = re.search(r'^analyze:\s*(true|false)$', record_text, re.MULTILINE)
analyze_match = re.search(r"^analyze:\s*(true|false)$", record_text, re.MULTILINE)
if not analyze_match or analyze_match.group(1) != "true":
result["message"] = f"[ERROR] analyze != true in {record_path}. Set analyze: true first."
return result
@ -1032,7 +1207,7 @@ def prepare_deep_reading(vault: Path, zotero_key: str, force: bool = False) -> d
status_match = re.search(r'^deep_reading_status:\s*"?(.*?)"?$', record_text, re.MULTILINE)
dr_status = status_match.group(1).strip() if status_match else "pending"
if dr_status == "done" and not force:
result["message"] = f"[WARN] deep_reading_status already 'done'. Use --force to re-run."
result["message"] = "[WARN] deep_reading_status already 'done'. Use --force to re-run."
return result
# 4. Check OCR / fulltext availability
@ -1098,7 +1273,9 @@ def prepare_deep_reading(vault: Path, zotero_key: str, force: bool = False) -> d
figure_map_path = ocr_dir / "figure-map.json"
fulltext_text = fulltext_md.read_text(encoding="utf-8")
figure_map = build_figure_map(fulltext_text, zotero_key=zotero_key)
figure_map["generated_at"] = __import__("datetime").datetime.now(__import__("datetime").timezone.utc).isoformat()
figure_map["generated_at"] = (
__import__("datetime").datetime.now(__import__("datetime").timezone.utc).isoformat()
)
figure_map_path.parent.mkdir(parents=True, exist_ok=True)
figure_map_path.write_text(json.dumps(figure_map, ensure_ascii=False, indent=2), encoding="utf-8")
written_paths.append(figure_map_path)
@ -1132,13 +1309,9 @@ def prepare_deep_reading(vault: Path, zotero_key: str, force: bool = False) -> d
formal_note.write_text(updated, encoding="utf-8")
result["figures"] = [
{"number": f.number, "image_id": f.image_id, "page": f.page, "title": f.title}
for f in planned_figures
]
result["tables"] = [
{"number": t.number, "page": t.page, "image_link": t.image_link}
for t in planned_tables
{"number": f.number, "image_id": f.image_id, "page": f.page, "title": f.title} for f in planned_figures
]
result["tables"] = [{"number": t.number, "page": t.page, "image_link": t.image_link} for t in planned_tables]
result["status"] = "ok"
result["message"] = (
@ -1168,16 +1341,19 @@ def scan_deep_reading_queue(vault: Path) -> list[dict]:
"""
all_records = scan_library_records(vault)
# Filter: only records that still need deep reading
queue = [r for r in all_records if r['deep_reading_status'] != 'done']
queue = [r for r in all_records if r["deep_reading_status"] != "done"]
# Sort: OCR completed first, then by domain, then by key
queue.sort(key=lambda row: (
0 if row['ocr_status'] == 'done' else 1,
row['domain'],
row['zotero_key'],
))
queue.sort(
key=lambda row: (
0 if row["ocr_status"] == "done" else 1,
row["domain"],
row["zotero_key"],
)
)
return queue
import contextlib
import sys
# Fix Windows console encoding for Unicode output
@ -1187,6 +1363,7 @@ if sys.platform == "win32":
except AttributeError:
# Python < 3.7 fallback
import codecs
sys.stdout = codecs.getwriter("utf-8")(sys.stdout.buffer)
@ -1199,19 +1376,29 @@ def main() -> int:
scaffold_parser = subparsers.add_parser("ensure-scaffold", help="Append a deep-reading scaffold to a note")
scaffold_parser.add_argument("note", type=Path)
scaffold_parser.add_argument("--fulltext", type=Path, help="Optional fulltext markdown used to build figure headings")
scaffold_parser.add_argument(
"--fulltext", type=Path, help="Optional fulltext markdown used to build figure headings"
)
scaffold_parser.add_argument("--figures", help="Comma-separated selected figure numbers to insert")
scaffold_parser.add_argument("--tables", help="Comma-separated selected table numbers to insert")
validate_parser = subparsers.add_parser("validate-selected", help="Validate selected figure/table embeds are present in the note")
validate_parser = subparsers.add_parser(
"validate-selected", help="Validate selected figure/table embeds are present in the note"
)
validate_parser.add_argument("note", type=Path)
validate_parser.add_argument("--fulltext", type=Path, required=True, help="OCR fulltext markdown used to resolve figure/table embeds")
validate_parser.add_argument(
"--fulltext", type=Path, required=True, help="OCR fulltext markdown used to resolve figure/table embeds"
)
validate_parser.add_argument("--figures", help="Comma-separated selected figure numbers to validate")
validate_parser.add_argument("--tables", help="Comma-separated selected table numbers to validate")
full_validate_parser = subparsers.add_parser("validate-note", help="Run the full structural validation for a deep-reading note")
full_validate_parser = subparsers.add_parser(
"validate-note", help="Run the full structural validation for a deep-reading note"
)
full_validate_parser.add_argument("note", type=Path)
full_validate_parser.add_argument("--fulltext", type=Path, required=True, help="OCR fulltext markdown used to resolve figure/table embeds")
full_validate_parser.add_argument(
"--fulltext", type=Path, required=True, help="OCR fulltext markdown used to resolve figure/table embeds"
)
full_validate_parser.add_argument("--figures", help="Comma-separated selected figure numbers to validate")
full_validate_parser.add_argument("--tables", help="Comma-separated selected table numbers to validate")
@ -1224,8 +1411,12 @@ def main() -> int:
map_parser.add_argument("--key", default="", help="Zotero key for output metadata")
map_parser.add_argument("--out", type=Path, help="Optional output JSON path (default: stdout)")
chart_type_parser = subparsers.add_parser("chart-type-scan", help="Scan figure captions for chart types and recommend chart-reading guides")
chart_type_parser.add_argument("figure_map", type=Path, help="Path to figure-map.json generated by figure-map command")
chart_type_parser = subparsers.add_parser(
"chart-type-scan", help="Scan figure captions for chart types and recommend chart-reading guides"
)
chart_type_parser.add_argument(
"figure_map", type=Path, help="Path to figure-map.json generated by figure-map command"
)
chart_type_parser.add_argument("--out", type=Path, help="Optional output JSON path (default: stdout)")
prepare_parser = subparsers.add_parser("prepare", help="One-click prepare all mechanical steps for deep reading")
@ -1247,7 +1438,9 @@ def main() -> int:
if args.command == "figure-index":
fulltext = args.fulltext.read_text(encoding="utf-8")
for figure in extract_figures_from_fulltext(fulltext):
print(f"Figure {figure.number}\tid={figure.image_id}\tpage={figure.page}\timage={figure.image_link}\ttitle={figure.title}")
print(
f"Figure {figure.number}\tid={figure.image_id}\tpage={figure.page}\timage={figure.image_link}\ttitle={figure.title}"
)
return 0
if args.command == "ensure-scaffold":
@ -1260,10 +1453,8 @@ def main() -> int:
figure_map = None
figure_map_path = args.fulltext.parent / "figure-map.json"
if figure_map_path.exists():
try:
with contextlib.suppress(json.JSONDecodeError, UnicodeDecodeError):
figure_map = json.loads(figure_map_path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, UnicodeDecodeError):
pass
figure_candidates = extract_figures_from_fulltext(fulltext, figure_map)
table_candidates = extract_tables_from_fulltext(fulltext, figure_map)
selected_figures = {part.strip() for part in (args.figures or "").split(",") if part.strip()}
@ -1285,7 +1476,9 @@ def main() -> int:
table_candidates = extract_tables_from_fulltext(fulltext)
selected_figures = {part.strip() for part in (args.figures or "").split(",") if part.strip()}
selected_tables = {part.strip() for part in (args.tables or "").split(",") if part.strip()}
figures = select_entries_by_numbers(figure_candidates, selected_figures, attr="image_id") if selected_figures else []
figures = (
select_entries_by_numbers(figure_candidates, selected_figures, attr="image_id") if selected_figures else []
)
tables = select_entries_by_numbers(table_candidates, selected_tables) if selected_tables else []
missing = validate_selected_blocks(note_text, figures, tables)
if missing:
@ -1302,10 +1495,8 @@ def main() -> int:
figure_map = None
figure_map_path = args.fulltext.parent / "figure-map.json"
if figure_map_path.exists():
try:
with contextlib.suppress(json.JSONDecodeError, UnicodeDecodeError):
figure_map = json.loads(figure_map_path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, UnicodeDecodeError):
pass
figure_candidates = extract_figures_from_fulltext(fulltext, figure_map)
table_candidates = extract_tables_from_fulltext(fulltext)
selected_figures = {part.strip() for part in (args.figures or "").split(",") if part.strip()}

22
paperforge/worker/__init__.py
View file

@ -1,19 +1,19 @@
"""PaperForge worker modules — migrated from pipeline/worker/scripts."""
from paperforge.worker.sync import (
load_export_rows,
run_selection_sync,
run_index_refresh,
obsidian_wikilink_for_pdf,
absolutize_vault_path,
_normalize_attachment_path,
_identify_main_pdf,
)
from paperforge.worker.base_views import build_base_views, ensure_base_views, merge_base_views
from paperforge.worker.deep_reading import run_deep_reading
from paperforge.worker.ocr import run_ocr
from paperforge.worker.repair import run_repair
from paperforge.worker.status import run_doctor, run_status
from paperforge.worker.deep_reading import run_deep_reading
from paperforge.worker.base_views import build_base_views, merge_base_views, ensure_base_views
from paperforge.worker.sync import (
_identify_main_pdf,
_normalize_attachment_path,
absolutize_vault_path,
load_export_rows,
obsidian_wikilink_for_pdf,
run_index_refresh,
run_selection_sync,
)
__all__ = [
"load_export_rows",

144
paperforge/worker/_utils.py
View file

@ -1,4 +1,5 @@
from __future__ import annotations
import json
import logging
import re
@ -9,25 +10,28 @@ logger = logging.getLogger(__name__)
# --- Constants ---
STANDARD_VIEW_NAMES = frozenset([
"控制面板", "推荐分析", "待 OCR", "OCR 完成",
"待深度阅读", "深度阅读完成", "正式卡片", "全记录"
])
STANDARD_VIEW_NAMES = frozenset(
["控制面板", "推荐分析", "待 OCR", "OCR 完成", "待深度阅读", "深度阅读完成", "正式卡片", "全记录"]
)
# --- Journal Database ---
def read_json(path: Path):
return json.loads(path.read_text(encoding='utf-8'))
return json.loads(path.read_text(encoding="utf-8"))
_JOURNAL_DB: dict[str, dict] | None = None
def load_journal_db(vault: Path) -> dict[str, dict]:
"""Load zoterostyle.json journal database."""
global _JOURNAL_DB
if _JOURNAL_DB is not None:
return _JOURNAL_DB
from paperforge.config import load_vault_config
zoterostyle_path = vault / load_vault_config(vault)['system_dir'] / 'Zotero' / 'zoterostyle.json'
zoterostyle_path = vault / load_vault_config(vault)["system_dir"] / "Zotero" / "zoterostyle.json"
if zoterostyle_path.exists():
try:
_JOURNAL_DB = read_json(zoterostyle_path)
@ -37,98 +41,111 @@ def load_journal_db(vault: Path) -> dict[str, dict]:
_JOURNAL_DB = {}
return _JOURNAL_DB
def lookup_impact_factor(journal_name: str, extra: str, vault: Path) -> str:
"""Lookup impact factor: prefer zoterostyle.json, fallback to extra field."""
if not journal_name:
return ''
return ""
journal_db = load_journal_db(vault)
if journal_name in journal_db:
rank_data = journal_db[journal_name].get('rank', {})
rank_data = journal_db[journal_name].get("rank", {})
if isinstance(rank_data, dict):
sciif = rank_data.get('sciif', '')
sciif = rank_data.get("sciif", "")
if sciif:
return str(sciif)
if extra:
if_match = re.search('影响因子[:：]\\s*([0-9.]+)', extra)
if_match = re.search("影响因子[:：]\\s*([0-9.]+)", extra)
if if_match:
return if_match.group(1)
return ''
return ""
# --- JSON I/O ---
def write_json(path: Path, data) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding='utf-8')
path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
def read_jsonl(path: Path):
rows = []
if not path.exists():
return rows
for line in path.read_text(encoding='utf-8').splitlines():
for line in path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if line:
rows.append(json.loads(line))
return rows
def write_jsonl(path: Path, rows) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
text = '\n'.join((json.dumps(row, ensure_ascii=False) for row in rows))
text = "\n".join(json.dumps(row, ensure_ascii=False) for row in rows)
if text:
text += '\n'
path.write_text(text, encoding='utf-8')
text += "\n"
path.write_text(text, encoding="utf-8")
# --- YAML Helpers ---
def yaml_quote(value: str) -> str:
if isinstance(value, bool):
return 'true' if value else 'false'
return '"' + str(value or '').replace('\\', '\\\\').replace('"', '\\"') + '"'
return "true" if value else "false"
return '"' + str(value or "").replace("\\", "\\\\").replace('"', '\\"') + '"'
def yaml_block(value: str) -> list[str]:
value = (value or '').strip()
value = (value or "").strip()
if not value:
return ['abstract: |-', ' ']
lines = ['abstract: |-']
return ["abstract: |-", " "]
lines = ["abstract: |-"]
for line in value.splitlines():
lines.append(f' {line}')
lines.append(f" {line}")
return lines
def yaml_list(key: str, values) -> list[str]:
cleaned = [str(value).strip() for value in values or [] if str(value).strip()]
cleaned = [str(value).strip() for value in values or [] if value is not None and str(value).strip()]
if not cleaned:
return [f'{key}: []']
lines = [f'{key}:']
return [f"{key}: []"]
lines = [f"{key}:"]
for value in cleaned:
lines.append(f' - {yaml_quote(value)}')
lines.append(f" - {yaml_quote(value)}")
return lines
# --- String / Path Utils ---
def slugify_filename(text: str) -> str:
cleaned = re.sub('[<>:"/\\\\|?*]+', '', text).strip()
return cleaned[:120] or 'untitled'
cleaned = re.sub('[<>:"/\\\\|?*]+', "", text).strip()
return cleaned[:120] or "untitled"
def _extract_year(value: str) -> str:
match = re.search('(19|20)\\d{2}', value or '')
return match.group(0) if match else ''
match = re.search("(19|20)\\d{2}", value or "")
return match.group(0) if match else ""
# --- Deep-Reading Queue ---
def _resolve_formal_note_path(vault: Path, zotero_key: str, domain: str) -> Path | None:
"""Resolve formal literature note by zotero_key."""
from paperforge.config import paperforge_paths
lit_root = paperforge_paths(vault)['literature']
lit_root = paperforge_paths(vault)["literature"]
domain_dir = lit_root / domain
if not domain_dir.exists():
return None
frontmatter_pattern = re.compile(
fr'^\s*zotero_key:\s*"?{re.escape(zotero_key)}"?\s*$', re.MULTILINE
)
for note_path in domain_dir.rglob('*.md'):
frontmatter_pattern = re.compile(rf'^\s*zotero_key:\s*"?{re.escape(zotero_key)}"?\s*$', re.MULTILINE)
for note_path in domain_dir.rglob("*.md"):
try:
text = note_path.read_text(encoding='utf-8')
text = note_path.read_text(encoding="utf-8")
except UnicodeDecodeError:
text = note_path.read_text(encoding='utf-8', errors='ignore')
text = note_path.read_text(encoding="utf-8", errors="ignore")
if frontmatter_pattern.search(text):
return note_path
return None
@ -154,8 +171,8 @@ def scan_library_records(vault: Path) -> list[dict]:
from paperforge.config import paperforge_paths
paths = paperforge_paths(vault)
records_root = paths.get('library_records')
ocr_root = paths.get('ocr')
records_root = paths.get("library_records")
ocr_root = paths.get("ocr")
if not records_root or not records_root.exists():
return []
@ -165,53 +182,54 @@ def scan_library_records(vault: Path) -> list[dict]:
if not domain_dir.is_dir():
continue
domain = domain_dir.name
for record_path in sorted(domain_dir.glob('*.md')):
for record_path in sorted(domain_dir.glob("*.md")):
try:
text = record_path.read_text(encoding='utf-8')
text = record_path.read_text(encoding="utf-8")
except Exception:
continue
# Extract frontmatter fields
zotero_key_match = re.search(r'^zotero_key:\s*(.+)$', text, re.MULTILINE)
analyze_match = re.search(r'^analyze:\s*(true|false)$', text, re.MULTILINE)
zotero_key_match = re.search(r"^zotero_key:\s*(.+)$", text, re.MULTILINE)
analyze_match = re.search(r"^analyze:\s*(true|false)$", text, re.MULTILINE)
title_match = re.search(r'^title:\s*"?(.+?)"?$', text, re.MULTILINE)
do_ocr_match = re.search(r'^do_ocr:\s*(true|false)$', text, re.MULTILINE)
do_ocr_match = re.search(r"^do_ocr:\s*(true|false)$", text, re.MULTILINE)
status_match = re.search(r'^deep_reading_status:\s*"?(.*?)"?$', text, re.MULTILINE)
zotero_key = (
zotero_key_match.group(1).strip().strip('"').strip("'")
if zotero_key_match else record_path.stem
zotero_key_match.group(1).strip().strip('"').strip("'") if zotero_key_match else record_path.stem
)
is_analyze = analyze_match is not None and analyze_match.group(1) == 'true'
title = title_match.group(1).strip().strip('"') if title_match else ''
do_ocr = do_ocr_match is not None and do_ocr_match.group(1) == 'true'
dr_status = status_match.group(1).strip() if status_match else 'pending'
is_analyze = analyze_match is not None and analyze_match.group(1) == "true"
title = title_match.group(1).strip().strip('"') if title_match else ""
do_ocr = do_ocr_match is not None and do_ocr_match.group(1) == "true"
dr_status = status_match.group(1).strip() if status_match else "pending"
if not is_analyze:
continue
# Check OCR status from meta.json
meta_path = ocr_root / zotero_key / 'meta.json'
ocr_status = 'pending'
meta_path = ocr_root / zotero_key / "meta.json"
ocr_status = "pending"
if meta_path.exists():
try:
meta = read_json(meta_path)
ocr_status = str(meta.get('ocr_status', 'pending')).strip().lower()
ocr_status = str(meta.get("ocr_status", "pending")).strip().lower()
except Exception:
pass
# Resolve formal note path
note_path = _resolve_formal_note_path(vault, zotero_key, domain)
results.append({
'zotero_key': zotero_key,
'domain': domain,
'title': title,
'analyze': True,
'do_ocr': do_ocr,
'deep_reading_status': dr_status,
'ocr_status': ocr_status,
'note_path': note_path,
})
results.append(
{
"zotero_key": zotero_key,
"domain": domain,
"title": title,
"analyze": True,
"do_ocr": do_ocr,
"deep_reading_status": dr_status,
"ocr_status": ocr_status,
"note_path": note_path,
}
)
return results

22
scripts/update-paperforge.ps1
View file

@ -1,6 +1,6 @@
# PaperForge Lite Update Script for Windows
# 一键更新脚本 — 双击运行即可
#
#
# 功能：
# 1. 自动检测安装方式（pip/pip-editable/git/zip）
# 2. 执行对应的更新命令
@ -62,7 +62,7 @@ try {
if ($pkgPath) {
$pkgDir = Split-Path $pkgPath -Parent
$installPath = $pkgDir
if ($pkgDir -match "site-packages|dist-packages") {
# pip install (non-editable)
$installMethod = "pip"
@ -100,7 +100,7 @@ switch ($installMethod) {
"pip" {
Write-Color "`n[更新] 通过 pip 升级..." "Cyan"
Write-Color "命令: pip install --upgrade paperforge" "Gray"
if (-not $Force) {
$confirm = Read-Host "确认更新? [y/N]"
if ($confirm -notmatch "^[Yy]") {
@ -108,15 +108,15 @@ switch ($installMethod) {
exit 0
}
}
& $pip.Source install --upgrade paperforge
$success = ($LASTEXITCODE -eq 0)
}
"pip-editable" {
Write-Color "`n[更新] pip editable 模式 detected" "Cyan"
Write-Color "步骤 1: git pull 拉取最新代码..." "Yellow"
if (-not $Force) {
$confirm = Read-Host "确认更新? [y/N]"
if ($confirm -notmatch "^[Yy]") {
@ -124,7 +124,7 @@ switch ($installMethod) {
exit 0
}
}
Push-Location $installPath
try {
git pull origin master
@ -137,11 +137,11 @@ switch ($installMethod) {
Pop-Location
}
}
"git" {
Write-Color "`n[更新] 通过 git pull 更新..." "Cyan"
Write-Color "命令: git pull origin master" "Gray"
if (-not $Force) {
$confirm = Read-Host "确认更新? [y/N]"
if ($confirm -notmatch "^[Yy]") {
@ -149,11 +149,11 @@ switch ($installMethod) {
exit 0
}
}
git pull origin master
$success = ($LASTEXITCODE -eq 0)
}
default {
Write-Color "`n[错误] 无法自动检测安装方式" "Red"
Write-Color @"

6
scripts/validate_setup.py
View file

@ -5,7 +5,6 @@ from __future__ import annotations
import json
import os
import sys
from pathlib import Path
@ -24,6 +23,7 @@ def load_config(vault: Path) -> dict:
# Try shared resolver first (01-03 and later installs)
try:
from paperforge.config import load_vault_config as _shared_load
return _shared_load(vault)
except ImportError:
pass
@ -109,7 +109,9 @@ def validate_agent(vault: Path, cfg: dict) -> list[tuple[bool, str]]:
chart_dir = skill_root / "chart-reading"
if chart_dir.exists():
guide_count = len(list(chart_dir.glob("*.md")))
results.append((guide_count >= 14, f"[{'OK' if guide_count >= 14 else 'WARN'}] chart-reading guides: {guide_count}"))
results.append(
(guide_count >= 14, f"[{'OK' if guide_count >= 14 else 'WARN'}] chart-reading guides: {guide_count}")
)
return results

66
scripts/welcome.py
View file

@ -3,31 +3,32 @@
from __future__ import annotations
import time
import sys
import time
class Colors:
"""ANSI color codes (safe for most terminals)."""
BLACK = '\033[30m'
RED = '\033[31m'
GREEN = '\033[32m'
YELLOW = '\033[33m'
BLUE = '\033[34m'
MAGENTA = '\033[35m'
CYAN = '\033[36m'
WHITE = '\033[37m'
BRIGHT_BLACK = '\033[90m'
BRIGHT_RED = '\033[91m'
BRIGHT_GREEN = '\033[92m'
BRIGHT_YELLOW = '\033[93m'
BRIGHT_BLUE = '\033[94m'
BRIGHT_MAGENTA = '\033[95m'
BRIGHT_CYAN = '\033[96m'
BRIGHT_WHITE = '\033[97m'
BOLD = '\033[1m'
DIM = '\033[2m'
ENDC = '\033[0m'
BLACK = "\033[30m"
RED = "\033[31m"
GREEN = "\033[32m"
YELLOW = "\033[33m"
BLUE = "\033[34m"
MAGENTA = "\033[35m"
CYAN = "\033[36m"
WHITE = "\033[37m"
BRIGHT_BLACK = "\033[90m"
BRIGHT_RED = "\033[91m"
BRIGHT_GREEN = "\033[92m"
BRIGHT_YELLOW = "\033[93m"
BRIGHT_BLUE = "\033[94m"
BRIGHT_MAGENTA = "\033[95m"
BRIGHT_CYAN = "\033[96m"
BRIGHT_WHITE = "\033[97m"
BOLD = "\033[1m"
DIM = "\033[2m"
ENDC = "\033[0m"
def clear_screen():
@ -73,7 +74,9 @@ def show_welcome():
("6", "Validation", "Verify installation"),
]
for num, title, desc in steps:
print(f" {Colors.BRIGHT_CYAN}[{num}]{Colors.ENDC} {Colors.BOLD}{title:<22}{Colors.ENDC} {Colors.DIM}-> {desc}{Colors.ENDC}")
print(
f" {Colors.BRIGHT_CYAN}[{num}]{Colors.ENDC} {Colors.BOLD}{title:<22}{Colors.ENDC} {Colors.DIM}-> {desc}{Colors.ENDC}"
)
# Bottom border
draw_border(70, "=", Colors.BRIGHT_BLUE)
@ -87,7 +90,10 @@ def show_progress(step: int, total: int, message: str):
bar = "#" * filled + "-" * (width - filled)
percent = int(100 * step / total)
print(f"\r{Colors.BRIGHT_CYAN}[{bar}]{Colors.ENDC} {Colors.BOLD}{percent}%{Colors.ENDC} {Colors.DIM}{message}{Colors.ENDC}", end="")
print(
f"\r{Colors.BRIGHT_CYAN}[{bar}]{Colors.ENDC} {Colors.BOLD}{percent}%{Colors.ENDC} {Colors.DIM}{message}{Colors.ENDC}",
end="",
)
sys.stdout.flush()
if step == total:
@ -101,11 +107,11 @@ def show_completion():
# Completion frame
completion = f"""
{Colors.BRIGHT_GREEN}======================================================================{Colors.ENDC}
{Colors.BOLD}{Colors.BRIGHT_WHITE}[OK] INSTALLATION COMPLETE{Colors.ENDC}
{Colors.BRIGHT_YELLOW}Your knowledge forge is ready!{Colors.ENDC}
{Colors.BOLD}{Colors.BRIGHT_WHITE}[OK] INSTALLATION COMPLETE{Colors.ENDC}
{Colors.BRIGHT_YELLOW}Your knowledge forge is ready!{Colors.ENDC}
{Colors.BRIGHT_GREEN}======================================================================{Colors.ENDC}
"""
print(completion)
@ -123,13 +129,13 @@ def show_install_menu():
"""Show the interactive installation menu."""
print(f"\n{Colors.BOLD}{Colors.BRIGHT_WHITE}Please select an option:{Colors.ENDC}\n")
print(f" {Colors.BRIGHT_GREEN}[1]{Colors.ENDC} {Colors.BOLD}Start Installation{Colors.ENDC}")
print(f" Configure vault, Zotero, OCR, and deploy scripts")
print(" Configure vault, Zotero, OCR, and deploy scripts")
print()
print(f" {Colors.BRIGHT_YELLOW}[2]{Colors.ENDC} {Colors.BOLD}Verify Setup{Colors.ENDC}")
print(f" Check existing installation and fix issues")
print(" Check existing installation and fix issues")
print()
print(f" {Colors.BRIGHT_BLUE}[3]{Colors.ENDC} {Colors.BOLD}View Documentation{Colors.ENDC}")
print(f" Open README and usage guide")
print(" Open README and usage guide")
print()
print(f" {Colors.BRIGHT_RED}[4]{Colors.ENDC} {Colors.BOLD}Exit{Colors.ENDC}")
print()

222
setup_wizard.py
View file

@ -7,6 +7,7 @@ PaperForge Lite Setup Wizard (Textual Step-by-Step)
Usage:
python setup_wizard.py --vault /path/to/vault
"""
from __future__ import annotations
import argparse
@ -17,7 +18,6 @@ import subprocess
import sys
import webbrowser
from pathlib import Path
from typing import Optional
if sys.platform == "win32":
import winreg
@ -25,33 +25,39 @@ else:
winreg = None
from textual.app import App, ComposeResult
from textual.containers import Container, Grid, Horizontal, Vertical
from textual.containers import Container, Horizontal, Vertical
from textual.message import Message
from textual.reactive import reactive
from textual.screen import Screen
from textual.widgets import (
Button,
ContentSwitcher,
Footer,
Header,
Label,
Markdown,
ProgressBar,
Static,
Tree,
)
# =============================================================================
# Agent Platform Configurations
# =============================================================================
AGENT_CONFIGS = {
"opencode": {"name": "OpenCode", "skill_dir": ".opencode/skills", "command_dir": ".opencode/command", "config_file": None},
"opencode": {
"name": "OpenCode",
"skill_dir": ".opencode/skills",
"command_dir": ".opencode/command",
"config_file": None,
},
"cursor": {"name": "Cursor", "skill_dir": ".cursor/skills", "config_file": ".cursor/settings.json"},
"claude": {"name": "Claude Code", "skill_dir": ".claude/skills", "config_file": ".claude/skills.json"},
"windsurf": {"name": "Windsurf", "skill_dir": ".windsurf/skills", "config_file": None},
"github_copilot": {"name": "GitHub Copilot", "skill_dir": ".github/skills", "config_file": ".github/copilot-instructions.md"},
"github_copilot": {
"name": "GitHub Copilot",
"skill_dir": ".github/skills",
"config_file": ".github/copilot-instructions.md",
},
"cline": {"name": "Cline", "skill_dir": ".clinerules/skills", "config_file": ".clinerules"},
"augment": {"name": "Augment", "skill_dir": ".augment/skills", "config_file": None},
"trae": {"name": "Trae", "skill_dir": ".trae/skills", "config_file": None},
@ -62,6 +68,7 @@ AGENT_CONFIGS = {
# Detection Logic (unchanged from previous version)
# =============================================================================
class CheckResult:
def __init__(self, name: str):
self.name = name
@ -75,7 +82,7 @@ class EnvChecker:
def __init__(self, vault: Path):
self.vault = vault
self.manual_zotero_path: Optional[Path] = None
self.manual_zotero_path: Path | None = None
self.system_dir: str = "99_System" # 可由用户自定义
self.results: dict[str, CheckResult] = {
"python": CheckResult("Python 版本"),
@ -143,11 +150,11 @@ class EnvChecker:
r.action_required = True
return r
def _find_zotero(self, manual_path: Optional[Path] = None) -> Optional[Path]:
def _find_zotero(self, manual_path: Path | None = None) -> Path | None:
# 如果提供了手动路径，优先使用
if manual_path and manual_path.exists():
return manual_path
system = platform.system()
if system == "Windows":
# ...existing detection code...
@ -309,9 +316,7 @@ class EnvChecker:
try:
data = json.loads(jf.read_text(encoding="utf-8"))
# Better BibTeX JSON 是 dict 格式（含 items），也兼容 list 格式
if isinstance(data, dict) and data.get("items"):
valid.append(jf.name)
elif isinstance(data, list) and len(data) > 0:
if isinstance(data, dict) and data.get("items") or isinstance(data, list) and len(data) > 0:
valid.append(jf.name)
except Exception:
pass
@ -374,16 +379,19 @@ class WelcomeStep(StepScreen):
def compose(self) -> ComposeResult:
yield from super().compose()
yield Static(r"""
______ ___ ______ _________________ ___________ _____ _____
yield Static(
r"""
______ ___ ______ _________________ ___________ _____ _____
| ___ \/ _ \ | ___ \ ___| ___ \ ___| _ | ___ \ __ \| ___|
| |_/ / /_\ \| |_/ / |__ | |_/ / |_ | | | | |_/ / | \/| |__
| __/| _ || __/| __|| /| _| | | | | /| | __ | __|
| | | | | || | | |___| |\ \| | \ \_/ / |\ \| |_\ \| |___
\_| \_| |_/\_| \____/\_| \_\_| \___/\_| \_|\____/\____/
[+] Forge Your Knowledge Into Power [+]
""", classes="logo")
| |_/ / /_\ \| |_/ / |__ | |_/ / |_ | | | | |_/ / | \/| |__
| __/| _ || __/| __|| /| _| | | | | /| | __ | __|
| | | | | || | | |___| |\ \| | \ \_/ / |\ \| |_\ \| |___
\_| \_| |_/\_| \____/\_| \_\_| \___/\_| \_|\____/\____/
[+] Forge Your Knowledge Into Power [+]
""",
classes="logo",
)
yield Markdown("""
**PaperForge Lite** 是一个连接 Zotero 与 Obsidian 的文献工作流工具。
@ -488,7 +496,7 @@ class VaultStep(StepScreen):
def __init__(self, step_id: str, checker: EnvChecker, vault: str = "", **kwargs):
kwargs.setdefault("id", step_id)
super().__init__(**kwargs)
super().__init__(step_id=step_id, checker=checker, **kwargs)
self.step_id = step_id
self.checker = checker
self.step_idx = int(step_id.split("-")[1])
@ -502,6 +510,7 @@ PaperForge 需要知道你的 **Obsidian Vault 位置**，以及你想要的目
你可以保留默认名称，也可以自定义。
""")
from textual.widgets import Input
yield Static("Obsidian Vault 路径 (绝对路径):", classes="step-title")
yield Input(value=self._vault, placeholder="D:\\Documents\\MyVault", id="input-vault-path")
yield Static("", id="vault-error", classes="status-bar")
@ -615,7 +624,7 @@ class ZoteroStep(StepScreen):
# 自动检测 Zotero 数据目录
detected = self._detect_zotero_data()
default_value = str(detected) if detected else ""
yield Markdown("""
**Zotero 数据目录**存放了你的文献数据库和 PDF 附件。
@ -638,6 +647,7 @@ class ZoteroStep(StepScreen):
> ⚠️ **不要**填 Vault 里面的路径，也不要在这里创建新文件夹
""")
from textual.widgets import Input
username = os.environ.get("USERNAME", os.environ.get("USER", "YourName"))
yield Static("Zotero 数据目录:", classes="step-title")
yield Input(
@ -651,7 +661,7 @@ class ZoteroStep(StepScreen):
id="btn-row",
)
def _detect_zotero_data(self) -> Optional[Path]:
def _detect_zotero_data(self) -> Path | None:
"""Build default Zotero data path from current username."""
home = Path.home()
# Default: C:/Users/<username>/Zotero (Windows) or ~/Zotero (Unix)
@ -663,6 +673,7 @@ class ZoteroStep(StepScreen):
def on_button_pressed(self, event: Button.Pressed) -> None:
if event.button.id == "btn-link-zotero":
from textual.widgets import Input
path_str = self.query_one("#input-zotero-data", Input).value.strip()
if not path_str:
self.set_status("请填写 Zotero 数据目录路径", False)
@ -687,18 +698,18 @@ class ZoteroStep(StepScreen):
is_inside_vault = True
except ValueError:
pass
if is_inside_vault:
# 在 Vault 内部，直接通过
self.app.zotero_data_dir = str(zotero_data)
self.set_status("Zotero 数据目录已确认", True)
self.app.post_message(StepPassed(self.step_idx))
return
# 在 Vault 外部，创建 Junction
system_dir = getattr(self.app, 'vault_config', {}).get('system_dir', '99_System')
system_dir = getattr(self.app, "vault_config", {}).get("system_dir", "99_System")
junction_path = vault / system_dir / "Zotero"
# Remove existing
if junction_path.exists() or junction_path.is_symlink():
try:
@ -714,7 +725,9 @@ class ZoteroStep(StepScreen):
if sys.platform == "win32":
subprocess.run(
["cmd", "/c", "mklink", "/J", str(junction_path), str(zotero_data)],
check=True, capture_output=True, shell=False,
check=True,
capture_output=True,
shell=False,
)
else:
junction_path.symlink_to(zotero_data, target_is_directory=True)
@ -769,7 +782,7 @@ class JsonStep(StepScreen):
def compose(self) -> ComposeResult:
yield from super().compose()
system_dir = getattr(self.app, 'vault_config', {}).get('system_dir', '99_System')
system_dir = getattr(self.app, "vault_config", {}).get("system_dir", "99_System")
yield Markdown(f"""
**Better BibTeX 自动导出**是 PaperForge 的数据来源。
@ -836,6 +849,7 @@ class DeployStep(StepScreen):
这些操作不会覆盖你的数据文件。
""")
from textual.widgets import Input
yield Static("PaddleOCR API Key:", classes="step-title")
yield Input(placeholder="粘贴你的 PaddleOCR API Key", id="input-api-key")
yield Static("PaddleOCR API URL:", classes="step-title")
@ -865,15 +879,13 @@ class DeployStep(StepScreen):
for idx, name in required_steps:
if not step_states[idx]:
incomplete.append(f"步骤 {idx}: {name}")
if incomplete:
self.set_status(
f"[无法部署] 以下步骤未完成：\n" + "\n".join(incomplete) +
"\n请先返回并完成上述步骤",
False
"[无法部署] 以下步骤未完成：\n" + "\n".join(incomplete) + "\n请先返回并完成上述步骤", False
)
return
self.set_status("正在部署...", None)
success = self._deploy()
if success:
@ -885,12 +897,12 @@ class DeployStep(StepScreen):
def _deploy(self) -> bool:
"""Deploy scripts and create config files."""
vault = self.checker.vault
vault_config = getattr(self.app, 'vault_config', {})
system_dir = vault_config.get('system_dir', '99_System')
resources_dir = vault_config.get('resources_dir', '03_Resources')
literature_dir = vault_config.get('literature_dir', 'Literature')
control_dir = vault_config.get('control_dir', 'LiteratureControl')
base_dir = vault_config.get('base_dir', '05_Bases')
vault_config = getattr(self.app, "vault_config", {})
system_dir = vault_config.get("system_dir", "99_System")
resources_dir = vault_config.get("resources_dir", "03_Resources")
literature_dir = vault_config.get("literature_dir", "Literature")
control_dir = vault_config.get("control_dir", "LiteratureControl")
base_dir = vault_config.get("base_dir", "05_Bases")
def apply_user_paths(text: str, skill_dir_value: str = "") -> str:
agent_config_dir = str(Path(skill_dir_value or ".opencode/skills").parent).replace("\\", "/")
@ -916,15 +928,15 @@ class DeployStep(StepScreen):
for old, new in replacements.items():
text = text.replace(old, new)
return text
# 1. 获取 agent 配置
agent_config = getattr(self.app, 'agent_config', None)
agent_config = getattr(self.app, "agent_config", None)
if not agent_config:
self.set_status("错误：未选择 Agent 平台", False)
return False
skill_dir = agent_config.get('skill_dir', '.opencode/skills')
skill_dir = agent_config.get("skill_dir", ".opencode/skills")
# 2. 确定安装包根目录（wizard 所在目录的父目录）
wizard_dir = Path(__file__).parent.resolve()
# 如果 wizard 在 github-release/ 下，repo_root 就是 github-release/
@ -940,7 +952,7 @@ class DeployStep(StepScreen):
else:
self.set_status(f"错误：找不到安装包文件。请在 PaperForge 解压目录下运行此向导。当前: {wizard_dir}", False)
return False
# 3. 创建目录（使用用户自定义路径）
pf_path = vault / system_dir / "PaperForge"
dirs = [
@ -956,18 +968,27 @@ class DeployStep(StepScreen):
]
for d in dirs:
d.mkdir(parents=True, exist_ok=True)
# 4. 复制脚本（从安装包到 Vault）
import shutil
# Copy worker modules to PaperForge/worker/scripts/
worker_src = repo_root / "paperforge/worker/sync.py"
worker_dst = pf_path / "worker/scripts/sync.py"
if worker_src.exists():
import shutil
shutil.copy2(worker_src, worker_dst)
# Also copy other worker modules
for mod in ["ocr.py", "repair.py", "status.py", "deep_reading.py", "update.py", "base_views.py", "__init__.py"]:
for mod in [
"ocr.py",
"repair.py",
"status.py",
"deep_reading.py",
"update.py",
"base_views.py",
"__init__.py",
]:
mod_src = repo_root / "paperforge/worker" / mod
if mod_src.exists():
shutil.copy2(mod_src, pf_path / "worker/scripts" / mod)
@ -980,7 +1001,7 @@ class DeployStep(StepScreen):
else:
self.set_status(f"错误：找不到 worker 脚本: {worker_src}", False)
return False
# Copy ld_deep.py (prefer paperforge/skills, fallback to skills/)
ld_src = repo_root / "paperforge/skills/literature-qa/scripts/ld_deep.py"
if not ld_src.exists():
@ -1002,7 +1023,7 @@ class DeployStep(StepScreen):
else:
self.set_status(f"错误：找不到 prompt_deep_subagent.md: {prompt_src}", False)
return False
# Copy chart-reading guides (prefer paperforge/skills, fallback to skills/)
chart_src = repo_root / "paperforge/skills/literature-qa/chart-reading"
if not chart_src.exists():
@ -1013,7 +1034,7 @@ class DeployStep(StepScreen):
shutil.copy2(f, chart_dst / f.name)
# Copy OpenCode command files when the target platform supports them.
if getattr(self.app, 'agent_key', '') == 'opencode':
if getattr(self.app, "agent_key", "") == "opencode":
command_src = repo_root / "command"
command_dst = vault / agent_config.get("command_dir", ".opencode/command")
if command_src.exists() and command_src.is_dir():
@ -1029,16 +1050,20 @@ class DeployStep(StepScreen):
shutil.copytree(docs_src, docs_dst, dirs_exist_ok=True)
for doc in docs_dst.rglob("*.md"):
doc.write_text(apply_user_paths(doc.read_text(encoding="utf-8"), skill_dir), encoding="utf-8")
# 5. 创建 .env（放到 PaperForge 目录下）
from textual.widgets import Input
api_key = self.query_one("#input-api-key", Input).value.strip()
api_url = self.query_one("#input-api-url", Input).value.strip() or "https://paddleocr.aistudio-app.com/api/v2/ocr/jobs"
api_url = (
self.query_one("#input-api-url", Input).value.strip()
or "https://paddleocr.aistudio-app.com/api/v2/ocr/jobs"
)
if not api_key:
self.set_status("请填写 PaddleOCR API Key", False)
return False
# 把 .env 放在 PaperForge 目录下
env_path = pf_path / ".env"
env_content = f"""# PaperForge 配置文件
@ -1068,7 +1093,7 @@ ZOTERO_DATA_DIR={getattr(self.app, 'zotero_data_dir', '')}
json.dumps({"domains": export_domains}, indent=2, ensure_ascii=False),
encoding="utf-8",
)
# 6. 创建 paperforge.json（包含用户自定义路径）
pf_json = vault / "paperforge.json"
existing_config = {}
@ -1077,28 +1102,30 @@ ZOTERO_DATA_DIR={getattr(self.app, 'zotero_data_dir', '')}
existing_config = json.loads(pf_json.read_text(encoding="utf-8"))
except Exception:
existing_config = {}
existing_config.update({
"version": existing_config.get("version", "1.2.0"),
"agent_platform": agent_config.get('name', 'OpenCode'),
"agent_key": getattr(self.app, 'agent_key', 'opencode'),
"skill_dir": skill_dir,
"command_dir": agent_config.get("command_dir", ""),
"system_dir": system_dir,
"resources_dir": resources_dir,
"literature_dir": literature_dir,
"control_dir": control_dir,
"base_dir": base_dir,
"paperforge_path": f"{system_dir}/PaperForge",
"zotero_data_dir": getattr(self.app, 'zotero_data_dir', ''),
"zotero_link": getattr(self.app, 'zotero_link', f"{system_dir}/Zotero"),
"vault_config": {
existing_config.update(
{
"version": existing_config.get("version", "1.2.0"),
"agent_platform": agent_config.get("name", "OpenCode"),
"agent_key": getattr(self.app, "agent_key", "opencode"),
"skill_dir": skill_dir,
"command_dir": agent_config.get("command_dir", ""),
"system_dir": system_dir,
"resources_dir": resources_dir,
"literature_dir": literature_dir,
"control_dir": control_dir,
"base_dir": base_dir,
},
})
"paperforge_path": f"{system_dir}/PaperForge",
"zotero_data_dir": getattr(self.app, "zotero_data_dir", ""),
"zotero_link": getattr(self.app, "zotero_link", f"{system_dir}/Zotero"),
"vault_config": {
"system_dir": system_dir,
"resources_dir": resources_dir,
"literature_dir": literature_dir,
"control_dir": control_dir,
"base_dir": base_dir,
},
}
)
pf_json.write_text(json.dumps(existing_config, indent=2, ensure_ascii=False), encoding="utf-8")
agents_src = repo_root / "AGENTS.md"
@ -1106,7 +1133,7 @@ ZOTERO_DATA_DIR={getattr(self.app, 'zotero_data_dir', '')}
if agents_src.exists():
agents_text = apply_user_paths(agents_src.read_text(encoding="utf-8"), skill_dir)
agents_dst.write_text(agents_text, encoding="utf-8")
# 7. 验证文件完整性
self.set_status("验证文件完整性...", None)
checks = {
@ -1119,20 +1146,22 @@ ZOTERO_DATA_DIR={getattr(self.app, 'zotero_data_dir', '')}
"导出目录": (pf_path / "exports").exists(),
"OCR 目录": (pf_path / "ocr").exists(),
}
missing = [k for k, v in checks.items() if not v]
if missing:
self.set_status(f"验证失败: {', '.join(missing)}", False)
return False
self.set_status("文件验证通过，初始化系统...", True)
# 8. 运行初始化命令（模拟测试）
try:
# 测试 selection-sync（会报错因为没 JSON，但测试脚本能否运行）
result = subprocess.run(
[sys.executable, str(worker_dst), "--vault", str(vault), "status"],
capture_output=True, text=True, timeout=10,
capture_output=True,
text=True,
timeout=10,
)
if result.returncode == 0:
self.set_status("工作流脚本运行正常", True)
@ -1146,7 +1175,9 @@ ZOTERO_DATA_DIR={getattr(self.app, 'zotero_data_dir', '')}
try:
result = subprocess.run(
[sys.executable, "-m", "pip", "install", "-e", str(repo_root)],
capture_output=True, text=True, timeout=60,
capture_output=True,
text=True,
timeout=60,
)
if result.returncode == 0:
self.set_status("PaperForge 工具包安装完成（paperforge 命令已全局注册）", True)
@ -1164,9 +1195,9 @@ class DoneStep(StepScreen):
def compose(self) -> ComposeResult:
yield from super().compose()
vault_config = getattr(self.app, 'vault_config', {})
system_dir = vault_config.get('system_dir', '99_System')
worker_cmd = f"python -m paperforge sync --vault ."
vault_config = getattr(self.app, "vault_config", {})
system_dir = vault_config.get("system_dir", "99_System")
worker_cmd = "python -m paperforge sync --vault ."
yield Markdown(f"""
## 安装完成！
@ -1276,8 +1307,10 @@ paperforge deep-reading # 查看精读队列
# Custom Messages
# =============================================================================
class StepPassed(Message):
"""步骤通过消息"""
def __init__(self, step_idx: int):
super().__init__()
self.step_idx = step_idx
@ -1285,6 +1318,7 @@ class StepPassed(Message):
class RestartWizard(Message):
"""重新开始消息"""
def __init__(self):
super().__init__()
@ -1293,32 +1327,33 @@ class RestartWizard(Message):
# Main App
# =============================================================================
class SetupWizardApp(App):
"""PaperForge 安装向导主应用"""
CSS = """
Screen { align: center middle; }
.wizard-container { width: 95%; height: 95%; border: solid green; }
.sidebar { width: 25%; height: 100%; border: solid gray; padding: 1; }
.sidebar-title { text-align: center; text-style: bold; color: cyan; padding: 1; }
.step-tree { height: 1fr; }
.main-area { width: 75%; height: 100%; padding: 1; }
.progress-area { height: auto; padding: 0 1; }
.content-area { height: 1fr; border: solid blue; padding: 1; overflow-y: auto; }
.logo { text-align: center; color: ansi_bright_cyan; text-style: bold; height: auto; }
.step-title { text-style: bold; color: yellow; }
.status-bar { height: auto; padding: 1; }
.step-content { padding: 1; }
.step-content Markdown { padding: 0 1; }
#btn-row { height: auto; padding: 1; }
#btn-row Button { margin: 0 1; }
.done { color: green; }
.current { color: yellow; text-style: bold; }
.pending { color: gray; }
@ -1434,7 +1469,7 @@ class SetupWizardApp(App):
self.current_step = 0
self.step_states = [False] * len(STEP_TITLES)
for screen in self.step_screens.values():
if hasattr(screen, 'set_status'):
if hasattr(screen, "set_status"):
screen.set_status("")
def open_screenshot(self, filename: str) -> None:
@ -1456,6 +1491,7 @@ class SetupWizardApp(App):
# Entry
# =============================================================================
def _find_vault() -> Path | None:
"""Find vault by looking for paperforge.json in current or parent dirs."""
current = Path(".").resolve()

46
tests/conftest.py
View file

@ -3,12 +3,9 @@
from __future__ import annotations
import json
import os
import shutil
import sys
import tempfile
from collections.abc import Generator
from pathlib import Path
from typing import Generator
import pytest
@ -38,9 +35,7 @@ def create_test_vault() -> Path:
base_dir = vault / "05_Bases"
skill_dir = vault / ".opencode" / "skills" / "literature-qa" / "scripts"
for d in [
exports_dir, ocr_dir, literature_dir, records_dir, base_dir, skill_dir
]:
for d in [exports_dir, ocr_dir, literature_dir, records_dir, base_dir, skill_dir]:
d.mkdir(parents=True, exist_ok=True)
# Create paperforge.json
@ -65,8 +60,7 @@ def create_test_vault() -> Path:
# Create .env with PADDLEOCR_API_TOKEN
env_path = pf_dir / ".env"
env_path.write_text(
"PADDLEOCR_API_TOKEN=test_token\n"
"PADDLEOCR_JOB_URL=https://example.com/api\n",
"PADDLEOCR_API_TOKEN=test_token\nPADDLEOCR_JOB_URL=https://example.com/api\n",
encoding="utf-8",
)
@ -92,20 +86,20 @@ def create_test_vault() -> Path:
'doi: "10.1016/j.jse.2024.01.001"\n'
'date: "2024-03-15"\n'
'collection_path: ""\n'
'has_pdf: true\n'
"has_pdf: true\n"
'pdf_path: "[[99_System/Zotero/storage/TSTONE001/TSTONE001.pdf]]"\n'
'fulltext_md_path: "[[99_System/PaperForge/ocr/TSTONE001/fulltext.md]]"\n'
'recommend_analyze: true\n'
'analyze: true\n'
'do_ocr: true\n'
"recommend_analyze: true\n"
"analyze: true\n"
"do_ocr: true\n"
'ocr_status: "done"\n'
'deep_reading_status: "pending"\n'
'analysis_note: ""\n'
'collection_group:\n'
"collection_group:\n"
' - "骨科"\n'
'collections:\n'
"collections:\n"
' - "骨科"\n'
'collection_tags:\n'
"collection_tags:\n"
' - "骨科"\n'
'first_author: "John Smith"\n'
'journal: "Journal of Shoulder and Elbow Surgery"\n'
@ -117,7 +111,11 @@ def create_test_vault() -> Path:
)
# Create formal note for TSTONE001
note_path = literature_dir / "骨科" / "TSTONE001 - Biomechanical Comparison of Suture Anchor Fixations in Rotator Cuff Repair.md"
note_path = (
literature_dir
/ "骨科"
/ "TSTONE001 - Biomechanical Comparison of Suture Anchor Fixations in Rotator Cuff Repair.md"
)
note_path.parent.mkdir(parents=True, exist_ok=True)
note_path.write_text(
"---\n"
@ -127,9 +125,9 @@ def create_test_vault() -> Path:
'journal: "Journal of Shoulder and Elbow Surgery"\n'
'impact_factor: "5.2"\n'
'category: "骨科"\n'
'tags:\n'
' - 文献阅读\n'
' - 骨科\n'
"tags:\n"
" - 文献阅读\n"
" - 骨科\n"
'keywords: ["biomechanics", "rotator cuff"]\n'
'pdf_link: "[[99_System/Zotero/storage/TSTONE001/TSTONE001.pdf]]"\n'
"---\n\n"
@ -140,10 +138,16 @@ def create_test_vault() -> Path:
)
# Create Zotero storage with mock PDF
zotero_dir = system_dir / "Zotero" / "storage" / "TSTONE001"
# PDF goes at Zotero/KEY/filename.pdf so resolve_pdf_path can find it
zotero_dir = system_dir / "Zotero" / "TSTONE001"
zotero_dir.mkdir(parents=True, exist_ok=True)
(zotero_dir / "TSTONE001.pdf").write_text("mock pdf content", encoding="utf-8")
# Also create in storage/ subdirectory for legacy path resolution
storage_dir = system_dir / "Zotero" / "storage" / "TSTONE001"
storage_dir.mkdir(parents=True, exist_ok=True)
(storage_dir / "TSTONE001.pdf").write_text("mock pdf content", encoding="utf-8")
# Copy ld_deep.py to skill_dir (simulating deployment)
ld_deep_src = REPO_ROOT / "paperforge" / "skills" / "literature-qa" / "scripts" / "ld_deep.py"
if ld_deep_src.exists():

2
tests/sandbox/TestZoteroData/zotero.sqlite
View file

@ -1 +1 @@
SQLite format 3SQLite format 3SQLite format 3SQLite format 3
SQLite format 3SQLite format 3SQLite format 3SQLite format 3

2
tests/sandbox/exports/运动医学.json
View file

@ -51,4 +51,4 @@
]
}
]
}
}

10
tests/sandbox/generate_ocr_fixture.py
View file

@ -15,7 +15,6 @@ import sys
from datetime import datetime, timezone
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent.parent
FIXTURE_DIR = REPO_ROOT / "tests" / "sandbox" / "ocr-complete" / "TSTONE001"
EXPORT_JSON = REPO_ROOT / "tests" / "sandbox" / "exports" / "骨科.json"
@ -108,8 +107,10 @@ def generate_fixtures() -> None:
str(ld_deep_script),
"figure-map",
str(fulltext_path),
"--key", "TSTONE001",
"--out", str(figure_map_path),
"--key",
"TSTONE001",
"--out",
str(figure_map_path),
],
capture_output=True,
text=True,
@ -128,7 +129,8 @@ def generate_fixtures() -> None:
str(ld_deep_script),
"chart-type-scan",
str(figure_map_path),
"--out", str(chart_type_map_path),
"--out",
str(chart_type_map_path),
],
capture_output=True,
text=True,

12
tests/sandbox/generate_sandbox.py
View file

@ -14,6 +14,7 @@ Creates tests/sandbox/ with:
Wizard creates inside 00_TestVault/:
00_System/Zotero -> junction -> TestZoteroData (user points to our TestZoteroData)
"""
from __future__ import annotations
import json
@ -159,7 +160,8 @@ def build() -> None:
exports_abs = str(EXPORTS.resolve())
readme = SANDBOX / "README.md"
readme.write_text(f"""# PaperForge Lite — Test Sandbox
readme.write_text(
f"""# PaperForge Lite — Test Sandbox
## 用途
测试 PaperForge Lite 安装向导 `setup_wizard.py` 的完整流程。
@ -215,15 +217,17 @@ paperforge status
- 目录名故意和真实 vault 不同，避免硬编码测试不出来
- PDF 是最小化假文件（pymupdf 可读，内容为空）
- 不要往 sandbox 加真实数据
""", encoding="utf-8")
""",
encoding="utf-8",
)
pdf_count = sum(1 for papers in PAPERS.values() for p in papers if p["has_pdf"])
paper_count = sum(len(papers) for papers in PAPERS.values())
print(f"\nSandbox ready: {SANDBOX}")
print(f" TestZoteroData/storage/ — {pdf_count} PDFs")
print(f" exports/ — {len(PAPERS)} BBT JSON files ({paper_count} papers)")
print(f" 00_TestVault/ — EMPTY (wizard fills this)")
print(f"\nRun:")
print(" 00_TestVault/ — EMPTY (wizard fills this)")
print("\nRun:")
print(f" python setup_wizard.py --vault {VAULT.resolve()}")

2
tests/sandbox/ocr-complete/TSTONE001/chart-type-map.json
View file

@ -66,4 +66,4 @@
"recommended_guides": []
}
]
}
}

2
tests/sandbox/ocr-complete/TSTONE001/figure-map.json
View file

@ -77,4 +77,4 @@
],
"supplementary_figures": [],
"supplementary_tables": []
}
}

26
tests/sandbox/ocr-complete/TSTONE001/json/result.json Normal file
View file

@ -0,0 +1,26 @@
{
"pages": [
{
"page_num": 1,
"markdown": "Biomechanical Comparison of Suture Anchor Fixations in Rotator Cuff Repair\n\nJohn Smith, Jane Doe\nJournal of Shoulder and Elbow Surgery, 2024\n\n## Abstract\n\nThis study compares the biomechanical properties of various suture anchor configurations used in rotator cuff repair surgery.\n\n## Introduction\n\nRotator cuff tears are a common shoulder pathology. Surgical repair using suture anchors remains the gold standard treatment."
},
{
"page_num": 2,
"markdown": "## Methods\n\n### Patient Demographics\n\nTable 1: Patient demographics and baseline characteristics (n=45).\n\n### Biomechanical Testing\n\nFigure 2: Biomechanical testing setup. (A) Custom loading fixture. (B) Cyclic loading protocol.\n\n## Results\n\n### Load to Failure\n\nFigure 3: Load to failure comparison. Error bars represent standard deviation. *p < 0.05 vs single row."
},
{
"page_num": 3,
"markdown": "### Stiffness Analysis\n\nFigure 4: Stiffness comparison between repair techniques.\n\nTable 2: Biomechanical properties summary.\n\n## Discussion\n\nThe double row technique demonstrated significantly higher load to failure compared to single row repair.\n\n## Conclusion\n\nDouble row suture anchor fixation provides superior biomechanical properties for rotator cuff repair."
}
],
"figures": [
{"page": 1, "filename": "page_001_fig_01.png", "caption": "Overview of suture anchor placement techniques."},
{"page": 2, "filename": "page_002_fig_02.png", "caption": "Biomechanical testing setup."},
{"page": 3, "filename": "page_003_fig_03.png", "caption": "Load to failure comparison."},
{"page": 3, "filename": "page_003_fig_04.png", "caption": "Stiffness comparison between repair techniques."}
],
"tables": [
{"page": 2, "filename": "page_002_table_01.png", "caption": "Patient demographics and baseline characteristics."},
{"page": 3, "filename": "page_003_table_02.png", "caption": "Biomechanical properties summary."}
]
}

2
tests/sandbox/ocr-complete/TSTONE001/meta.json
View file

@ -3,4 +3,4 @@
"ocr_status": "done",
"page_count": 3,
"generated_at": "2026-04-24T05:04:53.026897+00:00"
}
}

36
tests/test_base_preservation.py
View file

@ -1,13 +1,12 @@
"""Tests for incremental merge and user-view preservation (Phase 3, Plan 02)."""
import pytest
from pathlib import Path
from paperforge.worker.base_views import (
PAPERFORGE_VIEW_PREFIX,
build_base_views,
ensure_base_views,
merge_base_views,
build_base_views,
substitute_config_placeholders,
PAPERFORGE_VIEW_PREFIX,
STANDARD_VIEW_NAMES,
)
from paperforge.worker.sync import slugify_filename
@ -46,13 +45,16 @@ class TestIncrementalMerge:
content = domain_base.read_text(encoding="utf-8")
assert content.count("type: table") == 8
user_custom = content + '''
user_custom = (
content
+ """
- type: table
name: "My Custom Dashboard"
order:
- title
- year
'''
"""
)
domain_base.write_text(user_custom, encoding="utf-8")
ensure_base_views(self.vault, self.paths, self.config, force=False)
@ -84,12 +86,15 @@ class TestIncrementalMerge:
ensure_base_views(self.vault, self.paths, self.config, force=False)
content1 = domain_base.read_text(encoding="utf-8")
user_custom = content1 + '''
user_custom = (
content1
+ """
- type: table
name: "My Custom View"
order:
- title
'''
"""
)
domain_base.write_text(user_custom, encoding="utf-8")
ensure_base_views(self.vault, self.paths, self.config, force=True)
@ -106,15 +111,14 @@ class TestIncrementalMerge:
content1 = domain_base.read_text(encoding="utf-8")
modified = content1.replace(
'filter: \'ocr_status = "done"\'',
'filter: \'ocr_status = "done" AND has_pdf = true\''
"filter: 'ocr_status = \"done\"'", "filter: 'ocr_status = \"done\" AND has_pdf = true'"
)
domain_base.write_text(modified, encoding="utf-8")
ensure_base_views(self.vault, self.paths, self.config, force=False)
refreshed = domain_base.read_text(encoding="utf-8")
assert 'filter: \'ocr_status = "done"\'' in refreshed
assert "filter: 'ocr_status = \"done\"'" in refreshed
assert "has_pdf = true" not in refreshed
def test_new_domain_base_is_created_on_first_run(self):
@ -202,7 +206,7 @@ class TestLiteratureHubBase:
class TestMergeBaseViews:
def test_merge_base_views_preserves_user_views(self):
"""merge_base_views preserves views without PAPERFORGE_VIEW_PREFIX."""
existing = '''
existing = """
filters:
and:
- file.inFolder("骨科")
@ -221,7 +225,7 @@ views:
name: "My Custom View"
order:
- title
'''
"""
views = build_base_views("骨科")
result = merge_base_views(existing, views)
@ -240,7 +244,7 @@ views:
def test_merge_base_views_unknown_placeholder_unchanged(self):
"""Unknown placeholders in content are left unchanged after merge."""
existing = '''
existing = """
filters:
and:
- file.inFolder("骨科")
@ -249,7 +253,7 @@ views:
name: "控制面板"
order:
- file.name
'''
"""
views = build_base_views("骨科")
result = merge_base_views(existing, views)

23
tests/test_base_views.py
View file

@ -1,6 +1,5 @@
"""Tests for the 8-view Base generation system (Phase 3, Plan 01)."""
import pytest
from pathlib import Path
from paperforge.worker.base_views import (
build_base_views,
substitute_config_placeholders,
@ -15,10 +14,7 @@ class TestBuildBaseViews:
def test_all_view_names_present(self):
views = build_base_views("骨科")
names = [v["name"] for v in views]
expected = [
"控制面板", "推荐分析", "待 OCR", "OCR 完成",
"待深度阅读", "深度阅读完成", "正式卡片", "全记录"
]
expected = ["控制面板", "推荐分析", "待 OCR", "OCR 完成", "待深度阅读", "深度阅读完成", "正式卡片", "全记录"]
assert names == expected
def test_each_view_has_required_keys(self):
@ -38,7 +34,7 @@ class TestBuildBaseViews:
def test_pending_ocr_filter(self):
views = build_base_views("骨科")
pending = next(v for v in views if v["name"] == "待 OCR")
assert 'do_ocr = true' in pending["filter"]
assert "do_ocr = true" in pending["filter"]
assert 'ocr_status = "pending"' in pending["filter"]
def test_ocr_done_filter(self):
@ -61,10 +57,7 @@ class TestSubstituteConfigPlaceholders:
vault.mkdir()
lib_rec = vault / "03_Resources" / "LiteratureControl" / "library-records"
lib_rec.mkdir(parents=True)
result = substitute_config_placeholders(
content,
{"library_records": lib_rec, "vault": vault}
)
result = substitute_config_placeholders(content, {"library_records": lib_rec, "vault": vault})
assert "${LIBRARY_RECORDS}" not in result
assert "03_Resources/LiteratureControl/library-records" in result
@ -77,8 +70,7 @@ class TestSubstituteConfigPlaceholders:
lib_rec.mkdir()
lit.mkdir()
result = substitute_config_placeholders(
content,
{"library_records": lib_rec, "literature": lit, "vault": vault}
content, {"library_records": lib_rec, "literature": lit, "vault": vault}
)
assert "${LIBRARY_RECORDS}" not in result
assert "${LITERATURE}" not in result
@ -94,8 +86,5 @@ class TestSubstituteConfigPlaceholders:
vault.mkdir()
lib_rec = vault / "LR"
lib_rec.mkdir()
result = substitute_config_placeholders(
content,
{"library_records": lib_rec, "vault": vault}
)
result = substitute_config_placeholders(content, {"library_records": lib_rec, "vault": vault})
assert "\\" not in result # Should use forward slash

1
tests/test_cli_paths.py
View file

@ -4,7 +4,6 @@
import io
import json
from contextlib import redirect_stdout
from pathlib import Path
import pytest

22
tests/test_cli_worker_dispatch.py
View file

@ -1,11 +1,11 @@
# Tests for paperforge CLI worker dispatch
# These tests prove the locked command surface without invoking real workers.
import pytest
import json
import sys
from pathlib import Path
from unittest.mock import patch, MagicMock
from unittest.mock import patch
import pytest
# ----------------------------------------------------------------------
# Worker function stubs — capture calls for assertion
@ -33,7 +33,7 @@ def stub_run_deep_reading(vault: Path, verbose: bool = False) -> int:
return 0
def stub_run_ocr(vault: Path, verbose: bool = False) -> int:
def stub_run_ocr(vault: Path, verbose: bool = False, no_progress: bool = False) -> int:
CAPTURED_CALLS.append(("run_ocr", vault))
return 0
@ -69,7 +69,9 @@ def mock_vault(tmp_path):
def test_status_dispatch(clean_captured, mock_vault):
"""main(['--vault', vault, 'status']) calls run_status."""
import importlib
import paperforge.cli as cli
importlib.reload(cli)
with patch.object(cli, "run_status", stub_run_status):
@ -82,7 +84,9 @@ def test_status_dispatch(clean_captured, mock_vault):
def test_selection_sync_dispatch(clean_captured, mock_vault):
"""main(['--vault', vault, 'selection-sync']) calls run_selection_sync."""
import importlib
import paperforge.cli as cli
importlib.reload(cli)
with patch.object(cli, "run_selection_sync", stub_run_selection_sync):
@ -95,7 +99,9 @@ def test_selection_sync_dispatch(clean_captured, mock_vault):
def test_index_refresh_dispatch(clean_captured, mock_vault):
"""main(['--vault', vault, 'index-refresh']) calls run_index_refresh."""
import importlib
import paperforge.cli as cli
importlib.reload(cli)
with patch.object(cli, "run_index_refresh", stub_run_index_refresh):
@ -108,7 +114,9 @@ def test_index_refresh_dispatch(clean_captured, mock_vault):
def test_deep_reading_dispatch(clean_captured, mock_vault):
"""main(['--vault', vault, 'deep-reading']) calls run_deep_reading."""
import importlib
import paperforge.cli as cli
importlib.reload(cli)
with patch.object(cli, "run_deep_reading", stub_run_deep_reading):
@ -121,7 +129,9 @@ def test_deep_reading_dispatch(clean_captured, mock_vault):
def test_ocr_run_dispatch(clean_captured, mock_vault):
"""main(['--vault', vault, 'ocr', 'run']) calls run_ocr."""
import importlib
import paperforge.cli as cli
importlib.reload(cli)
with patch.object(cli, "run_ocr", stub_run_ocr):
@ -134,7 +144,9 @@ def test_ocr_run_dispatch(clean_captured, mock_vault):
def test_ocr_alias_dispatch(clean_captured, mock_vault):
"""main(['--vault', vault, 'ocr']) calls run_ocr (alias for 'ocr run')."""
import importlib
import paperforge.cli as cli
importlib.reload(cli)
with patch.object(cli, "run_ocr", stub_run_ocr):
@ -147,7 +159,9 @@ def test_ocr_alias_dispatch(clean_captured, mock_vault):
def test_ocr_doctor_dispatch(clean_captured, mock_vault):
"""main(['--vault', vault, 'ocr', 'doctor']) calls _cmd_ocr_doctor."""
import importlib
import paperforge.cli as cli
importlib.reload(cli)
with patch.object(cli, "_cmd_ocr_doctor", lambda vault, args: 0):

46
tests/test_command_docs.py
View file

@ -1,4 +1,3 @@
# -*- coding: utf-8 -*-
"""
Test suite for PaperForge Lite command documentation.
@ -8,11 +7,11 @@ instead of unresolved <system_dir> token paths.
Scope: User-run command examples in markdown files.
Excluded: Architecture diagrams, frontmatter field examples in AGENTS.md.
"""
from __future__ import annotations
import re
from pathlib import Path
from typing import Optional
import pytest
@ -59,6 +58,7 @@ def code_block_lines(content: str) -> list[str]:
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture
def command_docs() -> dict[str, Path]:
base = REPO_ROOT / "command"
@ -85,6 +85,7 @@ def user_facing_docs() -> dict[str, Path]:
# Task 1 tests — stable commands present
# ---------------------------------------------------------------------------
class TestStableCommandsPresent:
"""Verify command docs contain stable paperforge commands."""
@ -107,10 +108,9 @@ class TestStableCommandsPresent:
code_lines = code_block_lines(content)
# Join all lines for substring search
combined = "\n".join(code_lines)
assert expected_cmd in combined, (
f"[{doc_key}] Expected stable command '{expected_cmd}' "
f"not found in code blocks. Code lines:\n{code_lines}"
)
assert (
expected_cmd in combined
), f"[{doc_key}] Expected stable command '{expected_cmd}' not found in code blocks. Code lines:\n{code_lines}"
def test_pf_deep_mentions_paperforge_deep_reading(
self,
@ -118,9 +118,9 @@ class TestStableCommandsPresent:
) -> None:
"""pf-deep.md must mention 'paperforge deep-reading' for queue preflight."""
content = read_fileutf8(command_docs["pf-deep"])
assert "paperforge deep-reading" in content, (
"pf-deep.md must mention 'paperforge deep-reading' for queue preflight"
)
assert (
"paperforge deep-reading" in content
), "pf-deep.md must mention 'paperforge deep-reading' for queue preflight"
def test_pf_deep_mentions_paperforge_paths_json(
self,
@ -128,16 +128,16 @@ class TestStableCommandsPresent:
) -> None:
"""pf-deep.md must reference 'paperforge paths --json' for path resolution."""
content = read_fileutf8(command_docs["pf-deep"])
assert "paperforge paths --json" in content or "paperforge paths" in content, (
"pf-deep.md must reference 'paperforge paths' or 'paperforge paths --json' "
"for script path discovery"
)
assert (
"paperforge paths --json" in content or "paperforge paths" in content
), "pf-deep.md must reference 'paperforge paths' or 'paperforge paths --json' for script path discovery"
# ---------------------------------------------------------------------------
# Task 1 tests — unresolved tokens absent from user-run examples
# ---------------------------------------------------------------------------
class TestUnresolvedTokensAbsentFromUserRunExamples:
"""
Limit unresolved-token rejection to user-run command examples.
@ -180,10 +180,9 @@ class TestUnresolvedTokensAbsentFromUserRunExamples:
code_lines = code_block_lines(content)
combined = "\n".join(code_lines)
match = self.LEGACY_PYTHON_LITERATURE_PIPELINE.search(combined)
assert match is None, (
f"[README] Found legacy unresolved token path in code blocks: {match.group(0)!r}\n"
f"Code lines:\n{code_lines}"
)
assert (
match is None
), f"[README] Found legacy unresolved token path in code blocks: {match.group(0)!r}\nCode lines:\n{code_lines}"
def test_installation_no_legacy_python_literature_pipeline_in_examples(
self,
@ -204,6 +203,7 @@ class TestUnresolvedTokensAbsentFromUserRunExamples:
# Task 1 tests — paperforge paths/status examples in user-facing docs
# ---------------------------------------------------------------------------
class TestPaperforgeCommandExamplesInUserDocs:
"""README, INSTALLATION, setup-guide must show stable paperforge command examples."""
@ -223,16 +223,16 @@ class TestPaperforgeCommandExamplesInUserDocs:
) -> None:
"""User-facing docs must contain at least one 'paperforge status' or 'paperforge paths' example."""
content = read_fileutf8(user_facing_docs[doc_key])
assert re.search(pattern, content), (
f"[{doc_key}] Expected to find stable paperforge command pattern {pattern!r} "
f"in user-facing documentation."
)
assert re.search(
pattern, content
), f"[{doc_key}] Expected to find stable paperforge command pattern {pattern!r} in user-facing documentation."
# ---------------------------------------------------------------------------
# New tests — unified commands present in user-facing docs
# ---------------------------------------------------------------------------
class TestUnifiedCommandsInUserDocs:
"""AGENTS.md, README must reference new unified commands, not old ones."""
@ -296,9 +296,7 @@ class TestUnifiedCommandsInUserDocs:
if "命令迁移说明" in line or "迁移" in line.lower() or "migration" in line.lower():
migration_started = True
if "paperforge ocr run" in line and not migration_started:
pytest.fail(
f"[{doc_key}] Found old command 'paperforge ocr run' outside migration section: {line}"
)
pytest.fail(f"[{doc_key}] Found old command 'paperforge ocr run' outside migration section: {line}")
# ---------------------------------------------------------------------------

95
tests/test_config.py
View file

@ -6,19 +6,19 @@ These tests prove:
- CONF-03: All consumers use the same resolver
- CONF-04: Top-level and nested paperforge.json keys are both honored
"""
from __future__ import annotations
import json
import os
from pathlib import Path
import pytest
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture
def minimal_vault(tmp_path: Path) -> Path:
"""A vault with no paperforge.json."""
@ -32,15 +32,18 @@ def vault_with_nested_config(tmp_path: Path) -> Path:
vault.mkdir()
pf = vault / "paperforge.json"
pf.write_text(
json.dumps({
"vault_config": {
"system_dir": "CustomSystem",
"resources_dir": "CustomResources",
"literature_dir": "CustomLiterature",
"control_dir": "CustomControl",
"base_dir": "CustomBases",
}
}, ensure_ascii=False),
json.dumps(
{
"vault_config": {
"system_dir": "CustomSystem",
"resources_dir": "CustomResources",
"literature_dir": "CustomLiterature",
"control_dir": "CustomControl",
"base_dir": "CustomBases",
}
},
ensure_ascii=False,
),
encoding="utf-8",
)
return vault
@ -53,18 +56,21 @@ def vault_with_top_level_config(tmp_path: Path) -> Path:
vault.mkdir()
pf = vault / "paperforge.json"
pf.write_text(
json.dumps({
"vault_config": {
"system_dir": "NestedSystem",
"resources_dir": "NestedResources",
json.dumps(
{
"vault_config": {
"system_dir": "NestedSystem",
"resources_dir": "NestedResources",
},
# Legacy top-level keys — these take precedence per CONF-04
"system_dir": "LegacySystem",
"resources_dir": "LegacyResources",
"literature_dir": "LegacyLiterature",
"control_dir": "LegacyControl",
"base_dir": "LegacyBases",
},
# Legacy top-level keys — these take precedence per CONF-04
"system_dir": "LegacySystem",
"resources_dir": "LegacyResources",
"literature_dir": "LegacyLiterature",
"control_dir": "LegacyControl",
"base_dir": "LegacyBases",
}, ensure_ascii=False),
ensure_ascii=False,
),
encoding="utf-8",
)
return vault
@ -108,40 +114,48 @@ def env_dict() -> dict[str, str]:
# DEFAULT_CONFIG tests — truths from must_haves
# ---------------------------------------------------------------------------
def test_default_system_dir_is_99_System():
"""Built-in default for system_dir must be '99_System'."""
from paperforge.config import DEFAULT_CONFIG
assert DEFAULT_CONFIG["system_dir"] == "99_System"
def test_default_resources_dir_is_03_Resources():
"""Built-in default for resources_dir must be '03_Resources'."""
from paperforge.config import DEFAULT_CONFIG
assert DEFAULT_CONFIG["resources_dir"] == "03_Resources"
def test_default_literature_dir():
from paperforge.config import DEFAULT_CONFIG
assert DEFAULT_CONFIG["literature_dir"] == "Literature"
def test_default_control_dir():
from paperforge.config import DEFAULT_CONFIG
assert DEFAULT_CONFIG["control_dir"] == "LiteratureControl"
def test_default_base_dir():
from paperforge.config import DEFAULT_CONFIG
assert DEFAULT_CONFIG["base_dir"] == "05_Bases"
def test_default_skill_dir():
from paperforge.config import DEFAULT_CONFIG
assert DEFAULT_CONFIG["skill_dir"] == ".opencode/skills"
def test_default_command_dir():
from paperforge.config import DEFAULT_CONFIG
assert DEFAULT_CONFIG["command_dir"] == ".opencode/command"
@ -149,9 +163,11 @@ def test_default_command_dir():
# ENV_KEYS coverage — CONF-01
# ---------------------------------------------------------------------------
def test_env_keys_has_all_required_overrides():
"""All PAPERFORGE_* env vars must be registered in ENV_KEYS."""
from paperforge.config import ENV_KEYS
required = {
"PAPERFORGE_VAULT",
"PAPERFORGE_SYSTEM_DIR",
@ -170,6 +186,7 @@ def test_env_keys_has_all_required_overrides():
# load_vault_config precedence — CONF-01, CONF-04
# ---------------------------------------------------------------------------
def test_env_overrides_nested_json(env_dict):
"""PAPERFORGE_SYSTEM_DIR overrides nested vault_config.system_dir (CONF-01)."""
from paperforge.config import load_vault_config
@ -186,12 +203,13 @@ def test_env_overrides_nested_json(env_dict):
env_dict["PAPERFORGE_SYSTEM_DIR"] = "EnvSystem"
cfg = load_vault_config(vault, env=env_dict)
assert cfg["system_dir"] == "EnvSystem", (
f"Expected 'EnvSystem' from PAPERFORGE_SYSTEM_DIR, got '{cfg['system_dir']}'"
)
assert (
cfg["system_dir"] == "EnvSystem"
), f"Expected 'EnvSystem' from PAPERFORGE_SYSTEM_DIR, got '{cfg['system_dir']}'"
# Cleanup
import shutil
shutil.rmtree(vault, ignore_errors=True)
@ -207,11 +225,12 @@ def test_explicit_overrides_win_over_env(env_dict):
overrides = {"system_dir": "OverrideSystem"}
cfg = load_vault_config(vault, env=env_dict, overrides=overrides)
assert cfg["system_dir"] == "OverrideSystem", (
f"Expected 'OverrideSystem' from explicit override, got '{cfg['system_dir']}'"
)
assert (
cfg["system_dir"] == "OverrideSystem"
), f"Expected 'OverrideSystem' from explicit override, got '{cfg['system_dir']}'"
import shutil
shutil.rmtree(vault, ignore_errors=True)
@ -237,17 +256,17 @@ def test_top_level_keys_override_nested_for_backward_compat(tmp_path: Path):
vault = tmp_path / "vault_legacy"
vault.mkdir()
(vault / "paperforge.json").write_text(
json.dumps({
"vault_config": {"system_dir": "NestedSystem"},
"system_dir": "LegacySystem",
}),
json.dumps(
{
"vault_config": {"system_dir": "NestedSystem"},
"system_dir": "LegacySystem",
}
),
encoding="utf-8",
)
cfg = load_vault_config(vault)
assert cfg["system_dir"] == "LegacySystem", (
f"Expected 'LegacySystem' from top-level key, got '{cfg['system_dir']}'"
)
assert cfg["system_dir"] == "LegacySystem", f"Expected 'LegacySystem' from top-level key, got '{cfg['system_dir']}'"
def test_defaults_used_when_no_json(tmp_path: Path):
@ -266,6 +285,7 @@ def test_defaults_used_when_no_json(tmp_path: Path):
# paperforge_paths key inventory — CONF-02, CONF-03
# ---------------------------------------------------------------------------
def test_paperforge_paths_returns_exact_keys(tmp_path: Path):
"""paperforge_paths() must return exactly the required user-facing keys."""
from paperforge.config import paperforge_paths
@ -369,11 +389,13 @@ def test_paperforge_paths_includes_ld_deep_script(tmp_path: Path):
# paths_as_strings — JSON serializable output
# ---------------------------------------------------------------------------
def test_paths_as_strings_returns_string_values():
"""paths_as_strings must return dict[str, str] with all values as strings."""
from paperforge.config import paths_as_strings
from pathlib import Path
from paperforge.config import paths_as_strings
paths = {
"vault": Path("/some/vault"),
"system": Path("/some/vault/99_System"),
@ -404,6 +426,7 @@ def test_paths_as_strings_returns_string_values():
# resolve_vault precedence
# ---------------------------------------------------------------------------
def test_resolve_vault_precedence_explicit_first(tmp_path: Path):
"""resolve_vault returns explicit cli_vault first."""
from paperforge.config import resolve_vault

19
tests/test_doctor.py
View file

@ -1,9 +1,9 @@
import pytest
import json
import sys
from pathlib import Path
from unittest.mock import patch
import pytest
def stub_run_doctor(vault: Path) -> int:
return 0
@ -11,7 +11,9 @@ def stub_run_doctor(vault: Path) -> int:
def test_doctor_command_exists(clean_captured, mock_vault):
import importlib
import paperforge.cli as cli
importlib.reload(cli)
with patch("paperforge.worker.status.run_doctor", stub_run_doctor):
@ -22,17 +24,21 @@ def test_doctor_command_exists(clean_captured, mock_vault):
def test_doctor_python_check():
from paperforge.worker.status import run_doctor
import inspect
from paperforge.worker.status import run_doctor
sig = inspect.signature(run_doctor)
assert "vault" in sig.parameters
def test_doctor_returns_int():
from paperforge.worker.status import run_doctor
import inspect
sig = inspect.signature(run_doctor)
assert sig.return_annotation in (int, "int") or True
from paperforge.worker.status import run_doctor
inspect.signature(run_doctor)
assert True
@pytest.fixture
@ -57,6 +63,7 @@ def mock_vault(tmp_path):
def test_doctor_on_empty_vault(tmp_path, capsys):
from paperforge.worker.status import run_doctor
pf_cfg = {
"system_dir": "99_System",
"resources_dir": "03_Resources",

27
tests/test_ld_deep_config.py
View file

@ -1,12 +1,13 @@
"""Test compatibility: ld_deep uses shared resolver for path construction."""
from __future__ import annotations
import json
import sys
from importlib.util import module_from_spec, spec_from_file_location
from pathlib import Path
import pytest
from importlib.util import spec_from_file_location, module_from_spec
# Pre-load ld_deep so its functions are available
_REPO_ROOT = Path(__file__).parent.parent
@ -28,7 +29,7 @@ def tmp_vault(tmp_path: Path) -> Path:
(paperforge / "ocr").mkdir(parents=True)
resources = tmp_path / "03_Resources"
literature = resources / "Literature"
resources / "Literature"
control = resources / "LiteratureControl"
(control / "library-records").mkdir(parents=True)
@ -58,19 +59,20 @@ class TestDeepLoadVaultConfig:
def test_load_vault_config_keys(self, tmp_vault: Path) -> None:
"""_load_vault_config returns same keys as shared resolver."""
from paperforge.config import load_vault_config as shared_load
import ld_deep
from paperforge.config import load_vault_config as shared_load
shared_cfg = shared_load(tmp_vault)
ld_cfg = ld_deep._load_vault_config(tmp_vault)
assert set(ld_cfg.keys()) == set(shared_cfg.keys()), (
f"Key mismatch: ld_deep={set(ld_cfg.keys())} vs shared={set(shared_cfg.keys())}"
)
assert set(ld_cfg.keys()) == set(
shared_cfg.keys()
), f"Key mismatch: ld_deep={set(ld_cfg.keys())} vs shared={set(shared_cfg.keys())}"
for key in shared_cfg:
assert ld_cfg.get(key) == shared_cfg.get(key), (
f"Key '{key}' differs: ld_deep={ld_cfg.get(key)!r} vs shared={shared_cfg.get(key)!r}"
)
assert ld_cfg.get(key) == shared_cfg.get(
key
), f"Key '{key}' differs: ld_deep={ld_cfg.get(key)!r} vs shared={shared_cfg.get(key)!r}"
def test_env_override_respected(self, tmp_vault: Path, monkeypatch) -> None:
"""PAPERFORGE_SYSTEM_DIR env var is respected."""
@ -93,13 +95,12 @@ class TestDeepPaperforgePaths:
for key in ["ocr", "records", "literature"]:
assert key in paths, f"Missing expected key: {key}"
def test_paperforge_paths_values_match_shared_resolver(
self, tmp_vault: Path
) -> None:
def test_paperforge_paths_values_match_shared_resolver(self, tmp_vault: Path) -> None:
"""Values for ocr, records, literature match paperforge_paths()."""
from paperforge.config import paperforge_paths as shared_paths
import ld_deep
from paperforge.config import paperforge_paths as shared_paths
shared = shared_paths(tmp_vault)
ld_paths = ld_deep._paperforge_paths(tmp_vault)

2
tests/test_ocr_classify.py
View file

@ -2,12 +2,12 @@
Maps exceptions to (state, suggestion) pairs per the D-03 taxonomy.
"""
from __future__ import annotations
import json
from unittest.mock import MagicMock
import pytest
import requests
from paperforge.ocr_diagnostics import classify_error

36
tests/test_ocr_doctor.py
View file

@ -2,9 +2,7 @@
import json
import os
from unittest.mock import patch, MagicMock
import pytest
from unittest.mock import MagicMock, patch
from paperforge.ocr_diagnostics import ocr_doctor
@ -123,15 +121,17 @@ def test_l4_live_success():
delete_resp = MagicMock()
delete_resp.status_code = 204
with patch.dict(os.environ, env, clear=True):
with patch(
with (
patch.dict(os.environ, env, clear=True),
patch(
"paperforge.ocr_diagnostics.requests.get",
side_effect=[get_resp] + [poll_resp] * 10,
):
with patch("paperforge.ocr_diagnostics.requests.post", return_value=post_resp):
with patch("paperforge.ocr_diagnostics.requests.delete", return_value=delete_resp):
with patch("paperforge.ocr_diagnostics.time.sleep"):
result = ocr_doctor(config=None, live=True)
),
patch("paperforge.ocr_diagnostics.requests.post", return_value=post_resp),
):
with patch("paperforge.ocr_diagnostics.requests.delete", return_value=delete_resp):
with patch("paperforge.ocr_diagnostics.time.sleep"):
result = ocr_doctor(config=None, live=True)
assert result["level"] == 4
assert result["passed"] is True
@ -153,15 +153,17 @@ def test_l4_live_failure():
delete_resp = MagicMock()
delete_resp.status_code = 204
with patch.dict(os.environ, env, clear=True):
with patch(
with (
patch.dict(os.environ, env, clear=True),
patch(
"paperforge.ocr_diagnostics.requests.get",
side_effect=[get_resp] + [poll_resp] * 10,
):
with patch("paperforge.ocr_diagnostics.requests.post", return_value=post_resp):
with patch("paperforge.ocr_diagnostics.requests.delete", return_value=delete_resp):
with patch("paperforge.ocr_diagnostics.time.sleep"):
result = ocr_doctor(config=None, live=True)
),
patch("paperforge.ocr_diagnostics.requests.post", return_value=post_resp),
):
with patch("paperforge.ocr_diagnostics.requests.delete", return_value=delete_resp):
with patch("paperforge.ocr_diagnostics.time.sleep"):
result = ocr_doctor(config=None, live=True)
assert result["level"] == 4
assert result["passed"] is False

393
tests/test_ocr_preflight.py
View file

@ -3,13 +3,12 @@
Verifies that missing PDFs are caught before API submission and produce
ocr_status: nopdf.
"""
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
class TestOcrPreflight:
"""Tests for PDF preflight behavior in run_ocr()."""
@ -36,59 +35,51 @@ class TestOcrPreflight:
mock_paths["exports"].mkdir()
mock_paths["ocr"].mkdir()
with patch(
"paperforge.worker.ocr.pipeline_paths",
return_value=mock_paths,
):
with patch(
with (
patch(
"paperforge.worker.ocr.pipeline_paths",
return_value=mock_paths,
),
patch(
"paperforge.worker.ocr.load_control_actions",
return_value={"ABC123": {"do_ocr": True}},
):
with patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": "ABC123",
"attachments": [],
"title": "No PDF Paper",
}
],
):
with patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
{
"zotero_key": "ABC123",
"has_pdf": False,
"pdf_path": "",
}
],
):
with patch(
"paperforge.worker.ocr.ensure_ocr_meta",
return_value={},
):
with patch(
"paperforge.worker.ocr.write_json"
) as mock_write:
with patch(
"paperforge.worker.sync.run_selection_sync"
):
with patch(
"paperforge.worker.sync.run_index_refresh"
):
from paperforge.worker.ocr import (
run_ocr,
)
),
patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": "ABC123",
"attachments": [],
"title": "No PDF Paper",
}
],
),
patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
{
"zotero_key": "ABC123",
"has_pdf": False,
"pdf_path": "",
}
],
),
patch(
"paperforge.worker.ocr.ensure_ocr_meta",
return_value={},
),
patch("paperforge.worker.ocr.write_json") as mock_write,
patch("paperforge.worker.sync.run_selection_sync"),
patch("paperforge.worker.sync.run_index_refresh"),
):
from paperforge.worker.ocr import (
run_ocr,
)
run_ocr(vault)
run_ocr(vault)
calls = mock_write.call_args_list
assert any(
call[0][1].get("ocr_status") == "nopdf"
for call in calls
if isinstance(call[0][1], dict)
)
assert any(call[0][1].get("ocr_status") == "nopdf" for call in calls if isinstance(call[0][1], dict))
def test_missing_pdf_file_sets_nopdf(self, tmp_path: Path) -> None:
"""has_pdf=True but file missing -> ocr_status: nopdf."""
@ -97,64 +88,56 @@ class TestOcrPreflight:
mock_paths["exports"].mkdir()
mock_paths["ocr"].mkdir()
with patch(
"paperforge.worker.ocr.pipeline_paths",
return_value=mock_paths,
):
with patch(
with (
patch(
"paperforge.worker.ocr.pipeline_paths",
return_value=mock_paths,
),
patch(
"paperforge.worker.ocr.load_control_actions",
return_value={"ABC123": {"do_ocr": True}},
):
with patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": "ABC123",
"attachments": [
{
"contentType": "application/pdf",
"path": str(tmp_path / "nonexistent.pdf"),
}
],
"title": "Missing PDF Paper",
}
],
):
with patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
),
patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": "ABC123",
"attachments": [
{
"zotero_key": "ABC123",
"has_pdf": True,
"pdf_path": str(tmp_path / "nonexistent.pdf"),
"contentType": "application/pdf",
"path": str(tmp_path / "nonexistent.pdf"),
}
],
):
with patch(
"paperforge.worker.ocr.ensure_ocr_meta",
return_value={},
):
with patch(
"paperforge.worker.ocr.write_json"
) as mock_write:
with patch(
"paperforge.worker.sync.run_selection_sync"
):
with patch(
"paperforge.worker.sync.run_index_refresh"
):
from paperforge.worker.ocr import (
run_ocr,
)
"title": "Missing PDF Paper",
}
],
),
patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
{
"zotero_key": "ABC123",
"has_pdf": True,
"pdf_path": str(tmp_path / "nonexistent.pdf"),
}
],
),
patch(
"paperforge.worker.ocr.ensure_ocr_meta",
return_value={},
),
patch("paperforge.worker.ocr.write_json") as mock_write,
patch("paperforge.worker.sync.run_selection_sync"),
patch("paperforge.worker.sync.run_index_refresh"),
):
from paperforge.worker.ocr import (
run_ocr,
)
run_ocr(vault)
run_ocr(vault)
calls = mock_write.call_args_list
assert any(
call[0][1].get("ocr_status") == "nopdf"
for call in calls
if isinstance(call[0][1], dict)
)
assert any(call[0][1].get("ocr_status") == "nopdf" for call in calls if isinstance(call[0][1], dict))
def test_valid_pdf_proceeds(self, tmp_path: Path) -> None:
"""Valid resolved PDF -> proceeds to API submission with resolved path."""
@ -165,68 +148,57 @@ class TestOcrPreflight:
mock_paths["exports"].mkdir()
mock_paths["ocr"].mkdir()
with patch(
"paperforge.worker.ocr.pipeline_paths",
return_value=mock_paths,
):
with patch(
with (
patch(
"paperforge.worker.ocr.pipeline_paths",
return_value=mock_paths,
),
patch(
"paperforge.worker.ocr.load_control_actions",
return_value={"ABC123": {"do_ocr": True}},
):
with patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": "ABC123",
"attachments": [
{
"contentType": "application/pdf",
"path": str(pdf),
}
],
"title": "Valid PDF Paper",
}
],
):
with patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
),
patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": "ABC123",
"attachments": [
{
"zotero_key": "ABC123",
"has_pdf": True,
"pdf_path": str(pdf),
"contentType": "application/pdf",
"path": str(pdf),
}
],
):
with patch(
"paperforge.worker.ocr.ensure_ocr_meta",
return_value={},
):
with patch(
"paperforge.worker.ocr.write_json"
):
with patch("builtins.open") as mock_open:
with patch(
"paperforge.worker.ocr.requests.post"
) as mock_post:
mock_post.return_value = MagicMock()
mock_post.return_value.json.return_value = {
"data": {"jobId": "123"}
}
mock_post.return_value.raise_for_status = (
lambda: None
)
with patch(
"paperforge.worker.sync.run_selection_sync"
):
with patch(
"paperforge.worker.sync.run_index_refresh"
):
from paperforge.worker.ocr import (
run_ocr,
)
"title": "Valid PDF Paper",
}
],
),
patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
{
"zotero_key": "ABC123",
"has_pdf": True,
"pdf_path": str(pdf),
}
],
),
patch(
"paperforge.worker.ocr.ensure_ocr_meta",
return_value={},
),
patch("paperforge.worker.ocr.write_json"),
patch("builtins.open") as mock_open,
patch("paperforge.worker.ocr.requests.post") as mock_post,
):
mock_post.return_value = MagicMock()
mock_post.return_value.json.return_value = {"data": {"jobId": "123"}}
mock_post.return_value.raise_for_status = lambda: None
with patch("paperforge.worker.sync.run_selection_sync"), patch("paperforge.worker.sync.run_index_refresh"):
from paperforge.worker.ocr import (
run_ocr,
)
run_ocr(vault)
run_ocr(vault)
mock_open.assert_called_once()
opened_path = mock_open.call_args[0][0]
@ -241,72 +213,61 @@ class TestOcrPreflight:
mock_paths["exports"].mkdir()
mock_paths["ocr"].mkdir()
with patch(
"paperforge.pdf_resolver.resolve_pdf_path",
return_value=str(target),
):
with patch(
with (
patch(
"paperforge.pdf_resolver.resolve_pdf_path",
return_value=str(target),
),
patch(
"paperforge.worker.ocr.pipeline_paths",
return_value=mock_paths,
):
with patch(
"paperforge.worker.ocr.load_control_actions",
return_value={"ABC123": {"do_ocr": True}},
):
with patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
),
patch(
"paperforge.worker.ocr.load_control_actions",
return_value={"ABC123": {"do_ocr": True}},
),
patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": "ABC123",
"attachments": [
{
"key": "ABC123",
"attachments": [
{
"contentType": "application/pdf",
"path": str(tmp_path / "junction.pdf"),
}
],
"title": "Junction PDF Paper",
"contentType": "application/pdf",
"path": str(tmp_path / "junction.pdf"),
}
],
):
with patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
{
"zotero_key": "ABC123",
"has_pdf": True,
"pdf_path": str(tmp_path / "junction.pdf"),
}
],
):
with patch(
"paperforge.worker.ocr.ensure_ocr_meta",
return_value={},
):
with patch(
"paperforge.worker.ocr.write_json"
):
with patch("builtins.open") as mock_open:
with patch(
"paperforge.worker.ocr.requests.post"
) as mock_post:
mock_post.return_value = MagicMock()
mock_post.return_value.json.return_value = {
"data": {"jobId": "123"}
}
mock_post.return_value.raise_for_status = (
lambda: None
)
with patch(
"paperforge.worker.sync.run_selection_sync"
):
with patch(
"paperforge.worker.sync.run_index_refresh"
):
from paperforge.worker.ocr import (
run_ocr,
)
"title": "Junction PDF Paper",
}
],
),
patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
{
"zotero_key": "ABC123",
"has_pdf": True,
"pdf_path": str(tmp_path / "junction.pdf"),
}
],
),
patch(
"paperforge.worker.ocr.ensure_ocr_meta",
return_value={},
),
patch("paperforge.worker.ocr.write_json"),
patch("builtins.open") as mock_open,
patch("paperforge.worker.ocr.requests.post") as mock_post,
):
mock_post.return_value = MagicMock()
mock_post.return_value.json.return_value = {"data": {"jobId": "123"}}
mock_post.return_value.raise_for_status = lambda: None
with patch("paperforge.worker.sync.run_selection_sync"), patch("paperforge.worker.sync.run_index_refresh"):
from paperforge.worker.ocr import (
run_ocr,
)
run_ocr(vault)
run_ocr(vault)
mock_open.assert_called_once()
opened_path = mock_open.call_args[0][0]

419
tests/test_ocr_state_machine.py
View file

@ -3,11 +3,12 @@
Covers: pending -> queued/running -> done/error/blocked transitions,
sync_ocr_queue state reconciliation, and cleanup_blocked_ocr_dirs behavior.
"""
from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import MagicMock, patch, call
from unittest.mock import MagicMock, patch
import pytest
@ -48,6 +49,7 @@ def _mock_paths(vault: Path, ocr_root: Path, exports: Path, library_records: Pat
# Tests for pending -> queued transition
# ---------------------------------------------------------------------------
class TestPendingToQueuedTransition:
"""Job moves from pending to queued when submitted to the API."""
@ -58,25 +60,40 @@ class TestPendingToQueuedTransition:
key = "TESTKEY1"
meta_path = ocr_root / key / "meta.json"
meta_path.parent.mkdir()
meta_path.write_text(json.dumps({
"zotero_key": key,
"ocr_status": "pending",
}), encoding="utf-8")
meta_path.write_text(
json.dumps(
{
"zotero_key": key,
"ocr_status": "pending",
}
),
encoding="utf-8",
)
lr_dir = library_records / "骨科"
lr_dir.mkdir(parents=True)
(lr_dir / f"{key}.md").write_text("""---
(lr_dir / f"{key}.md").write_text(
"""---
zotero_key: TESTKEY1
analyze: false
do_ocr: true
---
# Test""", encoding="utf-8")
# Test""",
encoding="utf-8",
)
(exports / "骨科.json").write_text(json.dumps([
{"key": key, "title": "Test Paper", "attachments": [
{"contentType": "application/pdf", "path": "test.pdf"}
]}
]), encoding="utf-8")
(exports / "骨科.json").write_text(
json.dumps(
[
{
"key": key,
"title": "Test Paper",
"attachments": [{"contentType": "application/pdf", "path": "test.pdf"}],
}
]
),
encoding="utf-8",
)
# Real PDF file so resolve_pdf_path succeeds
real_pdf = vault / "test.pdf"
@ -88,30 +105,44 @@ do_ocr: true
with patch("paperforge.worker.ocr.pipeline_paths", return_value=paths):
with patch("paperforge.worker.ocr.load_control_actions", return_value={key: {"do_ocr": True}}):
with patch("paperforge.worker.ocr.load_export_rows", return_value=[
{"key": key, "attachments": [{"contentType": "application/pdf", "path": "test.pdf"}], "title": "Test"}
]):
with patch("paperforge.worker.ocr.sync_ocr_queue", return_value=[
{"zotero_key": key, "has_pdf": True, "pdf_path": "test.pdf", "queue_status": "pending"}
]):
with patch("paperforge.worker.ocr.ensure_ocr_meta", return_value={"zotero_key": key, "ocr_status": "pending"}):
with patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": key,
"attachments": [{"contentType": "application/pdf", "path": "test.pdf"}],
"title": "Test",
}
],
):
with patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
{"zotero_key": key, "has_pdf": True, "pdf_path": "test.pdf", "queue_status": "pending"}
],
):
with patch(
"paperforge.worker.ocr.ensure_ocr_meta",
return_value={"zotero_key": key, "ocr_status": "pending"},
):
with patch("paperforge.worker.ocr.write_json") as mock_write:
with patch("paperforge.worker.ocr.requests.post", mock_post):
with patch("paperforge.worker.sync.run_selection_sync"):
with patch("paperforge.worker.sync.run_index_refresh"):
from paperforge.worker.ocr import run_ocr
run_ocr(vault)
# Check requests.post was called (job submitted)
assert mock_post.called, "requests.post was not called - job not submitted"
# Check meta was updated to queued
meta_calls = [c for c in mock_write.call_args_list
if isinstance(c[0][1], dict) and c[0][1].get("zotero_key") == key]
meta_calls = [
c for c in mock_write.call_args_list if isinstance(c[0][1], dict) and c[0][1].get("zotero_key") == key
]
assert meta_calls, f"No meta write found for key {key}"
final_meta = meta_calls[-1][0][1]
assert final_meta.get("ocr_status") == "queued", \
f"Expected 'queued', got {final_meta.get('ocr_status')}"
assert final_meta.get("ocr_status") == "queued", f"Expected 'queued', got {final_meta.get('ocr_status')}"
assert final_meta.get("ocr_job_id") == "job-123"
@ -119,6 +150,7 @@ do_ocr: true
# Tests for processing -> done transition
# ---------------------------------------------------------------------------
class TestProcessingToDoneTransition:
"""Job moves from running/queued to done when polling returns success."""
@ -131,25 +163,40 @@ class TestProcessingToDoneTransition:
meta_path = ocr_root / key / "meta.json"
meta_path.parent.mkdir()
# meta.json already has job_id — simulating an already-submitted job
meta_path.write_text(json.dumps({
"zotero_key": key,
"ocr_status": "queued",
"ocr_job_id": "job-456",
}), encoding="utf-8")
meta_path.write_text(
json.dumps(
{
"zotero_key": key,
"ocr_status": "queued",
"ocr_job_id": "job-456",
}
),
encoding="utf-8",
)
lr_dir = library_records / "骨科"
lr_dir.mkdir(parents=True)
(lr_dir / f"{key}.md").write_text("""---
(lr_dir / f"{key}.md").write_text(
"""---
zotero_key: TESTKEY2
do_ocr: true
---
# Test""", encoding="utf-8")
# Test""",
encoding="utf-8",
)
(exports / "骨科.json").write_text(json.dumps([
{"key": key, "title": "Test Paper 2", "attachments": [
{"contentType": "application/pdf", "path": "test2.pdf"}
]}
]), encoding="utf-8")
(exports / "骨科.json").write_text(
json.dumps(
[
{
"key": key,
"title": "Test Paper 2",
"attachments": [{"contentType": "application/pdf", "path": "test2.pdf"}],
}
]
),
encoding="utf-8",
)
# Real PDF so resolve_pdf_path succeeds
real_pdf = vault / "test2.pdf"
@ -158,10 +205,7 @@ do_ocr: true
# Configure poll response: state=done
poll_response = MagicMock()
poll_response.json.return_value = {
"data": {
"state": "done",
"resultUrl": {"jsonUrl": "http://example.com/result.json"}
}
"data": {"state": "done", "resultUrl": {"jsonUrl": "http://example.com/result.json"}}
}
poll_response.raise_for_status = MagicMock()
@ -169,31 +213,43 @@ do_ocr: true
# This is sufficient since we only care about the first poll result
with patch("paperforge.worker.ocr.pipeline_paths", return_value=paths):
with patch("paperforge.worker.ocr.load_control_actions", return_value={key: {"do_ocr": True}}):
with patch("paperforge.worker.ocr.load_export_rows", return_value=[
{"key": key, "attachments": [{"contentType": "application/pdf", "path": "test2.pdf"}], "title": "Test"}
]):
with patch("paperforge.worker.ocr.sync_ocr_queue", return_value=[
{"zotero_key": key, "has_pdf": True, "pdf_path": "test2.pdf", "queue_status": "queued"}
]):
with patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": key,
"attachments": [{"contentType": "application/pdf", "path": "test2.pdf"}],
"title": "Test",
}
],
):
with patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
{"zotero_key": key, "has_pdf": True, "pdf_path": "test2.pdf", "queue_status": "queued"}
],
):
with patch("paperforge.worker.ocr.write_json") as mock_write:
with patch("paperforge.worker.ocr.requests.get", return_value=poll_response):
with patch("paperforge.worker.sync.run_selection_sync"):
with patch("paperforge.worker.sync.run_index_refresh"):
from paperforge.worker.ocr import run_ocr
run_ocr(vault)
meta_calls = [c for c in mock_write.call_args_list
if isinstance(c[0][1], dict) and c[0][1].get("zotero_key") == key]
meta_calls = [
c for c in mock_write.call_args_list if isinstance(c[0][1], dict) and c[0][1].get("zotero_key") == key
]
assert meta_calls, f"No meta write found for key {key}"
final_meta = meta_calls[-1][0][1]
assert final_meta.get("ocr_status") == "done", \
f"Expected 'done', got {final_meta.get('ocr_status')}"
assert final_meta.get("ocr_status") == "done", f"Expected 'done', got {final_meta.get('ocr_status')}"
# ---------------------------------------------------------------------------
# Tests for processing -> error transition
# ---------------------------------------------------------------------------
class TestProcessingToErrorTransition:
"""Job moves from running to error when the API returns an error state."""
@ -205,60 +261,81 @@ class TestProcessingToErrorTransition:
key = "TESTKEY3"
meta_path = ocr_root / key / "meta.json"
meta_path.parent.mkdir()
meta_path.write_text(json.dumps({
"zotero_key": key,
"ocr_status": "queued",
"ocr_job_id": "job-789",
}), encoding="utf-8")
meta_path.write_text(
json.dumps(
{
"zotero_key": key,
"ocr_status": "queued",
"ocr_job_id": "job-789",
}
),
encoding="utf-8",
)
lr_dir = library_records / "骨科"
lr_dir.mkdir(parents=True)
(lr_dir / f"{key}.md").write_text("""---
(lr_dir / f"{key}.md").write_text(
"""---
zotero_key: TESTKEY3
do_ocr: true
---
# Test""", encoding="utf-8")
# Test""",
encoding="utf-8",
)
(exports / "骨科.json").write_text(json.dumps([
{"key": key, "title": "Test Paper 3", "attachments": [
{"contentType": "application/pdf", "path": "test3.pdf"}
]}
]), encoding="utf-8")
(exports / "骨科.json").write_text(
json.dumps(
[
{
"key": key,
"title": "Test Paper 3",
"attachments": [{"contentType": "application/pdf", "path": "test3.pdf"}],
}
]
),
encoding="utf-8",
)
real_pdf = vault / "test3.pdf"
real_pdf.write_text("PDF content")
# Poll returns state=error
error_response = MagicMock()
error_response.json.return_value = {
"data": {
"state": "error",
"errorMsg": "Model inference failed"
}
}
error_response.json.return_value = {"data": {"state": "error", "errorMsg": "Model inference failed"}}
error_response.raise_for_status = MagicMock()
with patch("paperforge.worker.ocr.pipeline_paths", return_value=paths):
with patch("paperforge.worker.ocr.load_control_actions", return_value={key: {"do_ocr": True}}):
with patch("paperforge.worker.ocr.load_export_rows", return_value=[
{"key": key, "attachments": [{"contentType": "application/pdf", "path": "test3.pdf"}], "title": "Test"}
]):
with patch("paperforge.worker.ocr.sync_ocr_queue", return_value=[
{"zotero_key": key, "has_pdf": True, "pdf_path": "test3.pdf", "queue_status": "queued"}
]):
with patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": key,
"attachments": [{"contentType": "application/pdf", "path": "test3.pdf"}],
"title": "Test",
}
],
):
with patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
{"zotero_key": key, "has_pdf": True, "pdf_path": "test3.pdf", "queue_status": "queued"}
],
):
with patch("paperforge.worker.ocr.write_json") as mock_write:
with patch("paperforge.worker.ocr.requests.get", return_value=error_response):
with patch("paperforge.worker.sync.run_selection_sync"):
with patch("paperforge.worker.sync.run_index_refresh"):
from paperforge.worker.ocr import run_ocr
run_ocr(vault)
meta_calls = [c for c in mock_write.call_args_list
if isinstance(c[0][1], dict) and c[0][1].get("zotero_key") == key]
meta_calls = [
c for c in mock_write.call_args_list if isinstance(c[0][1], dict) and c[0][1].get("zotero_key") == key
]
assert meta_calls, "No meta write for our key"
final_meta = meta_calls[-1][0][1]
assert final_meta.get("ocr_status") == "error", \
f"Expected 'error', got {final_meta.get('ocr_status')}"
assert final_meta.get("ocr_status") == "error", f"Expected 'error', got {final_meta.get('ocr_status')}"
assert "Model inference failed" in final_meta.get("error", "")
@ -266,6 +343,7 @@ do_ocr: true
# Tests for processing -> blocked transition
# ---------------------------------------------------------------------------
class TestProcessingToBlockedTransition:
"""Job transitions to blocked when token is missing or PDF unreadable."""
@ -277,24 +355,39 @@ class TestProcessingToBlockedTransition:
key = "TESTKEY4"
meta_path = ocr_root / key / "meta.json"
meta_path.parent.mkdir()
meta_path.write_text(json.dumps({
"zotero_key": key,
"ocr_status": "pending",
}), encoding="utf-8")
meta_path.write_text(
json.dumps(
{
"zotero_key": key,
"ocr_status": "pending",
}
),
encoding="utf-8",
)
lr_dir = library_records / "骨科"
lr_dir.mkdir(parents=True)
(lr_dir / f"{key}.md").write_text("""---
(lr_dir / f"{key}.md").write_text(
"""---
zotero_key: TESTKEY4
do_ocr: true
---
# Test""", encoding="utf-8")
# Test""",
encoding="utf-8",
)
(exports / "骨科.json").write_text(json.dumps([
{"key": key, "title": "Test Paper 4", "attachments": [
{"contentType": "application/pdf", "path": "test4.pdf"}
]}
]), encoding="utf-8")
(exports / "骨科.json").write_text(
json.dumps(
[
{
"key": key,
"title": "Test Paper 4",
"attachments": [{"contentType": "application/pdf", "path": "test4.pdf"}],
}
]
),
encoding="utf-8",
)
# Real PDF so resolve_pdf_path succeeds
real_pdf = vault / "test4.pdf"
@ -304,6 +397,7 @@ do_ocr: true
# classify_error maps 401 -> 'blocked', so this tests the
# blocked transition path without needing to manipulate env vars.
import requests as _req
_mock_resp = MagicMock()
_mock_resp.status_code = 401
_http_err = _req.exceptions.HTTPError("401 Unauthorized")
@ -317,12 +411,22 @@ do_ocr: true
with patch("paperforge.worker.ocr.pipeline_paths", return_value=paths):
with patch("paperforge.worker.ocr.load_control_actions", return_value={key: {"do_ocr": True}}):
with patch("paperforge.worker.ocr.load_export_rows", return_value=[
{"key": key, "attachments": [{"contentType": "application/pdf", "path": "test4.pdf"}], "title": "Test"}
]):
with patch("paperforge.worker.ocr.sync_ocr_queue", return_value=[
{"zotero_key": key, "has_pdf": True, "pdf_path": "test4.pdf", "queue_status": "pending"}
]):
with patch(
"paperforge.worker.ocr.load_export_rows",
return_value=[
{
"key": key,
"attachments": [{"contentType": "application/pdf", "path": "test4.pdf"}],
"title": "Test",
}
],
):
with patch(
"paperforge.worker.ocr.sync_ocr_queue",
return_value=[
{"zotero_key": key, "has_pdf": True, "pdf_path": "test4.pdf", "queue_status": "pending"}
],
):
with patch("paperforge.worker.ocr.ensure_ocr_meta", side_effect=make_meta):
with patch("paperforge.worker.ocr.write_json") as mock_write:
with patch("paperforge.worker.sync.run_selection_sync"):
@ -330,20 +434,24 @@ do_ocr: true
with patch("paperforge.worker.ocr.requests.post", mock_post):
with patch("paperforge.worker.ocr.requests.get", MagicMock()):
from paperforge.worker.ocr import run_ocr
run_ocr(vault)
meta_calls = [c for c in mock_write.call_args_list
if isinstance(c[0][1], dict) and c[0][1].get("zotero_key") == key]
meta_calls = [
c for c in mock_write.call_args_list if isinstance(c[0][1], dict) and c[0][1].get("zotero_key") == key
]
assert meta_calls, "No meta write for our key"
final_meta = meta_calls[-1][0][1]
assert final_meta.get("ocr_status") == "blocked", \
f"Expected 'blocked' (missing token), got {final_meta.get('ocr_status')}"
assert (
final_meta.get("ocr_status") == "blocked"
), f"Expected 'blocked' (missing token), got {final_meta.get('ocr_status')}"
# ---------------------------------------------------------------------------
# Tests for sync_ocr_queue state reconciliation
# ---------------------------------------------------------------------------
class TestSyncOcrQueue:
"""sync_ocr_queue correctly reconciles existing queue with target rows."""
@ -355,35 +463,49 @@ class TestSyncOcrQueue:
key = "TESTKEY5"
# Write existing queue file with done status
ocr_queue_path = ocr_root / "ocr-queue.json"
ocr_queue_path.write_text(json.dumps([
{"zotero_key": key, "has_pdf": True, "pdf_path": "test.pdf",
"queue_status": "done", "queued_at": "2024-01-01T00:00:00Z"}
]), encoding="utf-8")
ocr_queue_path.write_text(
json.dumps(
[
{
"zotero_key": key,
"has_pdf": True,
"pdf_path": "test.pdf",
"queue_status": "done",
"queued_at": "2024-01-01T00:00:00Z",
}
]
),
encoding="utf-8",
)
# meta.json says done
meta_path = ocr_root / key / "meta.json"
meta_path.parent.mkdir()
meta_path.write_text(json.dumps({
"zotero_key": key,
"ocr_status": "done",
}), encoding="utf-8")
meta_path.write_text(
json.dumps(
{
"zotero_key": key,
"ocr_status": "done",
}
),
encoding="utf-8",
)
target_rows = [
{"zotero_key": key, "has_pdf": True, "pdf_path": "test.pdf"}
]
target_rows = [{"zotero_key": key, "has_pdf": True, "pdf_path": "test.pdf"}]
from paperforge.worker.ocr import sync_ocr_queue
result = sync_ocr_queue(paths, target_rows)
# done should be skipped
assert not any(r["zotero_key"] == key for r in result), \
"done status should be skipped"
assert not any(r["zotero_key"] == key for r in result), "done status should be skipped"
# ---------------------------------------------------------------------------
# Tests for cleanup_blocked_ocr_dirs
# ---------------------------------------------------------------------------
class TestCleanupBlockedOcrDirs:
"""cleanup_blocked_ocr_dirs removes empty blocked directories."""
@ -395,17 +517,22 @@ class TestCleanupBlockedOcrDirs:
blocked_dir = ocr_root / key
blocked_dir.mkdir()
meta_path = blocked_dir / "meta.json"
meta_path.write_text(json.dumps({
"zotero_key": key,
"ocr_status": "blocked",
}), encoding="utf-8")
meta_path.write_text(
json.dumps(
{
"zotero_key": key,
"ocr_status": "blocked",
}
),
encoding="utf-8",
)
# No fulltext.md or json/result.json -> should be removed
from paperforge.worker.ocr import cleanup_blocked_ocr_dirs
cleanup_blocked_ocr_dirs(paths)
assert not blocked_dir.exists(), \
f"Blocked dir {blocked_dir} should have been removed"
assert not blocked_dir.exists(), f"Blocked dir {blocked_dir} should have been removed"
def test_preserves_blocked_dir_with_payload(self, tmp_path: Path) -> None:
vault, ocr_root, exports, library_records = _make_vault(tmp_path)
@ -415,31 +542,37 @@ class TestCleanupBlockedOcrDirs:
blocked_dir = ocr_root / key
blocked_dir.mkdir()
meta_path = blocked_dir / "meta.json"
meta_path.write_text(json.dumps({
"zotero_key": key,
"ocr_status": "blocked",
}), encoding="utf-8")
meta_path.write_text(
json.dumps(
{
"zotero_key": key,
"ocr_status": "blocked",
}
),
encoding="utf-8",
)
# Add fulltext.md as payload
(blocked_dir / "fulltext.md").write_text("OCR result text", encoding="utf-8")
from paperforge.worker.ocr import cleanup_blocked_ocr_dirs
cleanup_blocked_ocr_dirs(paths)
assert blocked_dir.exists(), \
f"Blocked dir with payload should be preserved"
assert blocked_dir.exists(), "Blocked dir with payload should be preserved"
# ---------------------------------------------------------------------------
# Tests for state definitions
# ---------------------------------------------------------------------------
class TestOcrJobStates:
"""Valid OCR job states are: pending, queued, running, done, error, blocked, nopdf."""
def test_all_expected_states_covered(self, tmp_path: Path) -> None:
"""Ensure run_ocr handles all documented states without crashing."""
vault, ocr_root, exports, library_records = _make_vault(tmp_path)
paths = _mock_paths(vault, ocr_root, exports, library_records)
_mock_paths(vault, ocr_root, exports, library_records)
# States to test: pending, queued, running, done, error, blocked, nopdf
states = ["pending", "queued", "running", "done", "error", "blocked", "nopdf"]
@ -447,27 +580,43 @@ class TestOcrJobStates:
key = f"TESTKEY_STATE_{state}"
meta_path = ocr_root / key / "meta.json"
meta_path.parent.mkdir()
meta_path.write_text(json.dumps({
"zotero_key": key,
"ocr_status": state,
}), encoding="utf-8")
meta_path.write_text(
json.dumps(
{
"zotero_key": key,
"ocr_status": state,
}
),
encoding="utf-8",
)
lr_dir = library_records / "骨科"
lr_dir.mkdir(parents=True, exist_ok=True)
(lr_dir / f"{key}.md").write_text(f"""---
(lr_dir / f"{key}.md").write_text(
f"""---
zotero_key: {key}
do_ocr: true
---
# Test""", encoding="utf-8")
# Test""",
encoding="utf-8",
)
(exports / "骨科.json").write_text(json.dumps([
{"key": key, "title": f"Test {state}", "attachments": [
{"contentType": "application/pdf", "path": "test.pdf"}
]}
]), encoding="utf-8")
(exports / "骨科.json").write_text(
json.dumps(
[
{
"key": key,
"title": f"Test {state}",
"attachments": [{"contentType": "application/pdf", "path": "test.pdf"}],
}
]
),
encoding="utf-8",
)
# Smoke test: ensure no state raises an unhandled exception
from paperforge.worker.ocr import run_ocr
try:
run_ocr(vault)
except Exception as e:

23
tests/test_path_normalization.py
View file

@ -13,26 +13,23 @@ from __future__ import annotations
import json
import sys
from pathlib import Path
from unittest.mock import patch
import pytest
REPO_ROOT = Path(__file__).resolve().parent.parent
if str(REPO_ROOT) not in sys.path:
sys.path.insert(0, str(REPO_ROOT))
from paperforge.worker.sync import (
_normalize_attachment_path,
_identify_main_pdf,
obsidian_wikilink_for_pdf,
_normalize_attachment_path,
load_export_rows,
obsidian_wikilink_for_pdf,
)
# ---------------------------------------------------------------------------
# Test class: TestBBTPathNormalization
# ---------------------------------------------------------------------------
class TestBBTPathNormalization:
"""Tests for _normalize_attachment_path() with various BBT export formats."""
@ -110,6 +107,7 @@ class TestBBTPathNormalization:
# Test class: TestMainPdfIdentification
# ---------------------------------------------------------------------------
class TestMainPdfIdentification:
"""Tests for _identify_main_pdf() hybrid priority strategy."""
@ -153,7 +151,12 @@ class TestMainPdfIdentification:
def test_no_pdf_attachments(self) -> None:
"""No PDF attachments returns (None, [])."""
attachments = [
{"path": "storage:KEY/data.xlsx", "contentType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "title": "Data", "size": 100},
{
"path": "storage:KEY/data.xlsx",
"contentType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"title": "Data",
"size": 100,
},
]
main, supplementary = _identify_main_pdf(attachments)
@ -189,6 +192,7 @@ class TestMainPdfIdentification:
# Test class: TestWikilinkGeneration
# ---------------------------------------------------------------------------
class TestWikilinkGeneration:
"""Tests for obsidian_wikilink_for_pdf() wikilink generation."""
@ -223,9 +227,7 @@ class TestWikilinkGeneration:
return real_target
return path
monkeypatch.setattr(
"paperforge.pdf_resolver.resolve_junction", mock_resolve_junction
)
monkeypatch.setattr("paperforge.pdf_resolver.resolve_junction", mock_resolve_junction)
# Test with a path that would trigger junction resolution
result = obsidian_wikilink_for_pdf("storage:KEY/file.pdf", vault_dir, zotero_dir)
@ -291,6 +293,7 @@ class TestWikilinkGeneration:
# Test class: TestLoadExportRowsIntegration
# ---------------------------------------------------------------------------
class TestLoadExportRowsIntegration:
"""Integration tests using fixture JSON files."""

26
tests/test_pdf_resolver.py
View file

@ -6,10 +6,8 @@ and has_pdf=False scenarios.
from __future__ import annotations
import os
import json
from pathlib import Path
from unittest.mock import patch
import pytest
@ -47,9 +45,7 @@ class TestResolvePdfPath:
return target
return path
monkeypatch.setattr(
"paperforge.pdf_resolver.resolve_junction", mock_resolve_junction
)
monkeypatch.setattr("paperforge.pdf_resolver.resolve_junction", mock_resolve_junction)
result = resolve_pdf_path(str(junction), True, tmp_path)
assert result == str(target.resolve())
@ -162,9 +158,7 @@ class TestLoadExportRowsAttachmentNormalization:
"itemKey": "ABC123",
"itemType": "journalArticle",
"title": "Test Paper",
"attachments": [
{"path": "ABC123/ABC123.pdf", "contentType": "application/pdf"}
],
"attachments": [{"path": "ABC123/ABC123.pdf", "contentType": "application/pdf"}],
}
],
"collections": {},
@ -188,9 +182,7 @@ class TestLoadExportRowsAttachmentNormalization:
"itemKey": "ABC123",
"itemType": "journalArticle",
"title": "Test Paper",
"attachments": [
{"path": "storage:ABC123/ABC123.pdf", "contentType": "application/pdf"}
],
"attachments": [{"path": "storage:ABC123/ABC123.pdf", "contentType": "application/pdf"}],
}
],
"collections": {},
@ -215,9 +207,7 @@ class TestLoadExportRowsAttachmentNormalization:
"itemKey": "ABC123",
"itemType": "journalArticle",
"title": "Test Paper",
"attachments": [
{"path": abs_path, "contentType": "application/pdf"}
],
"attachments": [{"path": abs_path, "contentType": "application/pdf"}],
}
],
"collections": {},
@ -241,9 +231,7 @@ class TestLoadExportRowsAttachmentNormalization:
"itemKey": "ABC123",
"itemType": "journalArticle",
"title": "Test Paper",
"attachments": [
{"path": "", "contentType": ""}
],
"attachments": [{"path": "", "contentType": ""}],
}
],
"collections": {},
@ -267,9 +255,7 @@ class TestLoadExportRowsAttachmentNormalization:
"itemKey": "ABC123",
"itemType": "journalArticle",
"title": "Test Paper",
"attachments": [
{"path": "ABC123/ABC123.docx", "contentType": ""}
],
"attachments": [{"path": "ABC123/ABC123.docx", "contentType": ""}],
}
],
"collections": {},

24
tests/test_prepare_rollback.py
View file

@ -56,7 +56,13 @@ class TestPrepareRollback:
ocr_dir = vault / "99_System" / "PaperForge" / "ocr" / "TSTONE001"
figure_map_path = ocr_dir / "figure-map.json"
chart_type_map_path = ocr_dir / "chart-type-map.json"
formal_note = vault / "03_Resources" / "Literature" / "骨科" / "TSTONE001 - Biomechanical Comparison of Suture Anchor Fixations in Rotator Cuff Repair.md"
formal_note = (
vault
/ "03_Resources"
/ "Literature"
/ "骨科"
/ "TSTONE001 - Biomechanical Comparison of Suture Anchor Fixations in Rotator Cuff Repair.md"
)
# Remove pre-existing fixture files to test clean rollback
if figure_map_path.exists():
@ -84,7 +90,13 @@ class TestPrepareRollback:
ocr_dir = vault / "99_System" / "PaperForge" / "ocr" / "TSTONE001"
figure_map_path = ocr_dir / "figure-map.json"
chart_type_map_path = ocr_dir / "chart-type-map.json"
formal_note = vault / "03_Resources" / "Literature" / "骨科" / "TSTONE001 - Biomechanical Comparison of Suture Anchor Fixations in Rotator Cuff Repair.md"
formal_note = (
vault
/ "03_Resources"
/ "Literature"
/ "骨科"
/ "TSTONE001 - Biomechanical Comparison of Suture Anchor Fixations in Rotator Cuff Repair.md"
)
# Remove pre-existing fixture files to test clean rollback
if figure_map_path.exists():
@ -112,7 +124,13 @@ class TestPrepareRollback:
ocr_dir = vault / "99_System" / "PaperForge" / "ocr" / "TSTONE001"
figure_map_path = ocr_dir / "figure-map.json"
chart_type_map_path = ocr_dir / "chart-type-map.json"
formal_note = vault / "03_Resources" / "Literature" / "骨科" / "TSTONE001 - Biomechanical Comparison of Suture Anchor Fixations in Rotator Cuff Repair.md"
formal_note = (
vault
/ "03_Resources"
/ "Literature"
/ "骨科"
/ "TSTONE001 - Biomechanical Comparison of Suture Anchor Fixations in Rotator Cuff Repair.md"
)
original_note_text = formal_note.read_text(encoding="utf-8")

12
tests/test_repair.py
View file

@ -1,11 +1,10 @@
"""Tests for run_repair() — three-way OCR status divergence detection and repair."""
from __future__ import annotations
import json
import re
import pytest
from paperforge.worker.repair import run_repair
from paperforge.worker.sync import pipeline_paths
@ -220,6 +219,7 @@ class TestRunRepairScanOnly:
def test_verbose_output_printed(self, tmp_path, caplog):
import logging
caplog.set_level(logging.DEBUG, logger="paperforge.worker.repair")
vault = _make_vault(tmp_path)
paths = pipeline_paths(vault)
@ -227,7 +227,7 @@ class TestRunRepairScanOnly:
records_dir.mkdir(parents=True, exist_ok=True)
_write_library_record(records_dir, "KEY001", "骨科", "done")
_write_meta(paths["ocr"], "KEY001", "pending")
result = run_repair(vault, paths, verbose=True, fix=False)
run_repair(vault, paths, verbose=True, fix=False)
assert "KEY001" in caplog.text
assert "divergent" in caplog.text
@ -288,8 +288,8 @@ class TestRunRepairFixMode:
paths = pipeline_paths(vault)
records_dir = vault / "03_Resources" / "LiteratureControl" / "library-records" / "骨科"
records_dir.mkdir(parents=True, exist_ok=True)
record1_path = _write_library_record(records_dir, "KEY001", "骨科", "done", do_ocr="false")
record2_path = _write_library_record(records_dir, "KEY002", "骨科", "pending")
_write_library_record(records_dir, "KEY001", "骨科", "done", do_ocr="false")
_write_library_record(records_dir, "KEY002", "骨科", "pending")
_write_minimal_meta(paths["ocr"], "KEY001", "done")
_write_minimal_meta(paths["ocr"], "KEY002", "done")
result = run_repair(vault, paths, verbose=False, fix=True)
@ -314,7 +314,7 @@ class TestRunRepairFixMode:
lit_dir = vault / "03_Resources" / "Literature"
_write_library_record(records_dir, "KEY001", "骨科", "done", do_ocr="false")
note_path = _write_formal_note(lit_dir, "KEY001", "骨科", "done")
result = run_repair(vault, paths, verbose=False, fix=True)
run_repair(vault, paths, verbose=False, fix=True)
note_text = note_path.read_text(encoding="utf-8")
assert re.search(r'^ocr_status:\s*"?pending"?', note_text, re.MULTILINE)

149
tests/test_selection_sync_pdf.py
View file

@ -3,13 +3,12 @@
Verifies that run_selection_sync() sets ocr_status: nopdf when PDFs are missing
and preserves pending/done for valid PDFs.
"""
from __future__ import annotations
from pathlib import Path
from unittest.mock import patch
import pytest
class TestSelectionSyncPdf:
"""Tests for selection-sync PDF preflight reporting."""
@ -28,7 +27,7 @@ class TestSelectionSyncPdf:
resources.mkdir()
(resources / "LiteratureControl" / "library-records").mkdir(parents=True)
(resources / "Literature").mkdir(parents=True)
(vault / "paperforge.json").write_text('{}', encoding="utf-8")
(vault / "paperforge.json").write_text("{}", encoding="utf-8")
return vault
def _mock_export_item(self, key: str, has_pdf: bool = True, pdf_path: str = "") -> dict:
@ -52,35 +51,31 @@ class TestSelectionSyncPdf:
"""Export item with no PDF attachments → ocr_status: nopdf."""
vault = self._make_vault(tmp_path)
with patch(
"paperforge.worker.sync.pipeline_paths"
) as mock_paths:
with patch("paperforge.worker.sync.pipeline_paths") as mock_paths:
mock_paths.return_value = {
"exports": vault / "99_System" / "PaperForge" / "exports",
"ocr": vault / "99_System" / "PaperForge" / "ocr",
"library_records": vault / "03_Resources" / "LiteratureControl" / "library-records",
"literature": vault / "03_Resources" / "Literature",
}
with patch(
"paperforge.worker.sync.load_domain_config",
return_value={"domains": [{"export_file": "test.json", "domain": "TestDomain"}]},
with (
patch(
"paperforge.worker.sync.load_domain_config",
return_value={"domains": [{"export_file": "test.json", "domain": "TestDomain"}]},
),
patch("paperforge.worker.base_views.ensure_base_views"),
patch(
"paperforge.worker.sync.load_export_rows",
return_value=[self._mock_export_item("NO_PDF", has_pdf=False)],
),
):
with patch(
"paperforge.worker.base_views.ensure_base_views"
):
with patch(
"paperforge.worker.sync.load_export_rows",
return_value=[self._mock_export_item("NO_PDF", has_pdf=False)],
):
from paperforge.worker.sync import (
run_selection_sync,
)
from paperforge.worker.sync import (
run_selection_sync,
)
run_selection_sync(vault)
run_selection_sync(vault)
record_path = (
vault / "03_Resources" / "LiteratureControl" / "library-records" / "TestDomain" / "NO_PDF.md"
)
record_path = vault / "03_Resources" / "LiteratureControl" / "library-records" / "TestDomain" / "NO_PDF.md"
assert record_path.exists()
content = record_path.read_text(encoding="utf-8")
assert 'ocr_status: "nopdf"' in content
@ -91,37 +86,31 @@ class TestSelectionSyncPdf:
vault = self._make_vault(tmp_path)
missing_pdf = str(tmp_path / "missing.pdf")
with patch(
"paperforge.worker.sync.pipeline_paths"
) as mock_paths:
with patch("paperforge.worker.sync.pipeline_paths") as mock_paths:
mock_paths.return_value = {
"exports": vault / "99_System" / "PaperForge" / "exports",
"ocr": vault / "99_System" / "PaperForge" / "ocr",
"library_records": vault / "03_Resources" / "LiteratureControl" / "library-records",
"literature": vault / "03_Resources" / "Literature",
}
with patch(
"paperforge.worker.sync.load_domain_config",
return_value={"domains": [{"export_file": "test.json", "domain": "TestDomain"}]},
with (
patch(
"paperforge.worker.sync.load_domain_config",
return_value={"domains": [{"export_file": "test.json", "domain": "TestDomain"}]},
),
patch("paperforge.worker.base_views.ensure_base_views"),
patch(
"paperforge.worker.sync.load_export_rows",
return_value=[self._mock_export_item("MISSING", has_pdf=True, pdf_path=missing_pdf)],
),
):
with patch(
"paperforge.worker.base_views.ensure_base_views"
):
with patch(
"paperforge.worker.sync.load_export_rows",
return_value=[
self._mock_export_item("MISSING", has_pdf=True, pdf_path=missing_pdf)
],
):
from paperforge.worker.sync import (
run_selection_sync,
)
from paperforge.worker.sync import (
run_selection_sync,
)
run_selection_sync(vault)
run_selection_sync(vault)
record_path = (
vault / "03_Resources" / "LiteratureControl" / "library-records" / "TestDomain" / "MISSING.md"
)
record_path = vault / "03_Resources" / "LiteratureControl" / "library-records" / "TestDomain" / "MISSING.md"
assert record_path.exists()
content = record_path.read_text(encoding="utf-8")
assert 'ocr_status: "nopdf"' in content
@ -132,37 +121,31 @@ class TestSelectionSyncPdf:
pdf = tmp_path / "test.pdf"
pdf.write_bytes(b"%PDF-1.4\n")
with patch(
"paperforge.worker.sync.pipeline_paths"
) as mock_paths:
with patch("paperforge.worker.sync.pipeline_paths") as mock_paths:
mock_paths.return_value = {
"exports": vault / "99_System" / "PaperForge" / "exports",
"ocr": vault / "99_System" / "PaperForge" / "ocr",
"library_records": vault / "03_Resources" / "LiteratureControl" / "library-records",
"literature": vault / "03_Resources" / "Literature",
}
with patch(
"paperforge.worker.sync.load_domain_config",
return_value={"domains": [{"export_file": "test.json", "domain": "TestDomain"}]},
with (
patch(
"paperforge.worker.sync.load_domain_config",
return_value={"domains": [{"export_file": "test.json", "domain": "TestDomain"}]},
),
patch("paperforge.worker.base_views.ensure_base_views"),
patch(
"paperforge.worker.sync.load_export_rows",
return_value=[self._mock_export_item("VALID", has_pdf=True, pdf_path=str(pdf))],
),
):
with patch(
"paperforge.worker.base_views.ensure_base_views"
):
with patch(
"paperforge.worker.sync.load_export_rows",
return_value=[
self._mock_export_item("VALID", has_pdf=True, pdf_path=str(pdf))
],
):
from paperforge.worker.sync import (
run_selection_sync,
)
from paperforge.worker.sync import (
run_selection_sync,
)
run_selection_sync(vault)
run_selection_sync(vault)
record_path = (
vault / "03_Resources" / "LiteratureControl" / "library-records" / "TestDomain" / "VALID.md"
)
record_path = vault / "03_Resources" / "LiteratureControl" / "library-records" / "TestDomain" / "VALID.md"
assert record_path.exists()
content = record_path.read_text(encoding="utf-8")
assert 'ocr_status: "pending"' in content
@ -179,31 +162,29 @@ class TestSelectionSyncPdf:
encoding="utf-8",
)
with patch(
"paperforge.worker.sync.pipeline_paths"
) as mock_paths:
with patch("paperforge.worker.sync.pipeline_paths") as mock_paths:
mock_paths.return_value = {
"exports": vault / "99_System" / "PaperForge" / "exports",
"ocr": vault / "99_System" / "PaperForge" / "ocr",
"library_records": vault / "03_Resources" / "LiteratureControl" / "library-records",
"literature": vault / "03_Resources" / "Literature",
}
with patch(
"paperforge.worker.sync.load_domain_config",
return_value={"domains": [{"export_file": "test.json", "domain": "TestDomain"}]},
with (
patch(
"paperforge.worker.sync.load_domain_config",
return_value={"domains": [{"export_file": "test.json", "domain": "TestDomain"}]},
),
patch("paperforge.worker.base_views.ensure_base_views"),
patch(
"paperforge.worker.sync.load_export_rows",
return_value=[self._mock_export_item("CHANGED", has_pdf=False)],
),
):
with patch(
"paperforge.worker.base_views.ensure_base_views"
):
with patch(
"paperforge.worker.sync.load_export_rows",
return_value=[self._mock_export_item("CHANGED", has_pdf=False)],
):
from paperforge.worker.sync import (
run_selection_sync,
)
from paperforge.worker.sync import (
run_selection_sync,
)
run_selection_sync(vault)
run_selection_sync(vault)
content = record_path.read_text(encoding="utf-8")
assert 'ocr_status: "nopdf"' in content

39
tests/test_smoke.py
View file

@ -12,13 +12,11 @@ from __future__ import annotations
import importlib.util
import json
import os
import re
import subprocess
import sys
from pathlib import Path
from typing import Any
from unittest.mock import patch
import pytest
@ -34,6 +32,7 @@ if str(REPO_ROOT) not in sys.path:
# Helpers
# ---------------------------------------------------------------------------
def _import_ld_deep(path: Path | None = None) -> Any:
"""Import ld_deep.py using importlib, with Python 3.14 workaround."""
if path is None:
@ -67,9 +66,9 @@ class TestSetupWizard:
import setup_wizard
source = Path(setup_wizard.__file__).read_text(encoding="utf-8")
assert '"-m", "pip", "install", "-e"' in source or "pip install -e" in source, (
"setup wizard should call pip install -e"
)
assert (
'"-m", "pip", "install", "-e"' in source or "pip install -e" in source
), "setup wizard should call pip install -e"
assert "repo_root" in source, "setup wizard should reference repo_root"
@ -82,9 +81,9 @@ class TestDoctorImportability:
source = Path(run_doctor.__code__.co_filename).read_text(encoding="utf-8")
assert "importlib.util" in source, "doctor should use importlib.util for import check"
assert "spec_from_file_location" in source or "exec_module" in source, (
"doctor should actually exec_module ld_deep"
)
assert (
"spec_from_file_location" in source or "exec_module" in source
), "doctor should actually exec_module ld_deep"
def test_regression_doctor_env_name(self, test_vault: Path) -> None:
"""REG-02: doctor checks PADDLEOCR_API_TOKEN (not old env name)."""
@ -98,9 +97,9 @@ class TestDoctorImportability:
from paperforge.worker.status import run_doctor
source = Path(run_doctor.__code__.co_filename).read_text(encoding="utf-8")
assert "library.json 不存在" not in source or "*.json" in source, (
"doctor should support per-domain JSON exports"
)
assert (
"library.json 不存在" not in source or "*.json" in source
), "doctor should support per-domain JSON exports"
class TestLdDeepImport:
@ -277,14 +276,7 @@ class TestMetadataFields:
def test_regression_metadata_fields(self, test_vault: Path) -> None:
"""Verify library-record has non-empty first_author and journal."""
record_path = (
test_vault
/ "03_Resources"
/ "LiteratureControl"
/ "library-records"
/ "骨科"
/ "TSTONE001.md"
)
record_path = test_vault / "03_Resources" / "LiteratureControl" / "library-records" / "骨科" / "TSTONE001.md"
assert record_path.exists(), "library record should exist"
text = record_path.read_text(encoding="utf-8")
@ -306,14 +298,7 @@ class TestSelectionSync:
def test_regression_bbt_pdf_path(self, test_vault: Path) -> None:
"""REG-02: BBT attachment paths resolve correctly."""
record_path = (
test_vault
/ "03_Resources"
/ "LiteratureControl"
/ "library-records"
/ "骨科"
/ "TSTONE001.md"
)
record_path = test_vault / "03_Resources" / "LiteratureControl" / "library-records" / "骨科" / "TSTONE001.md"
assert record_path.exists(), "library record should exist"
text = record_path.read_text(encoding="utf-8")
assert "TSTONE001.pdf" in text, "record should reference the PDF"