fix(skill): multi-arm discovery strategy + mandatory entry rule

- AGENTS.md: new Section 0 mandatory entry rule with trigger phrases - retrieval-routing.md: rewrite Ladder A as 3-arm strategy (retrieve body + search metadata + context --collection inventory) - discover-papers.md: Step 2 uses multi-arm, large result handling, dedup - find-supporting-evidence.md: retrieve as primary fulltext pass - agent_context.py: update commands docs and rules
2026-07-22 06:50:53 +00:00 · 2026-05-20 15:39:00 +08:00 · 2026-05-20 15:39:00 +08:00 · f5d056ea5d
commit f5d056ea5d
parent 1b0aecf48e
5 changed files with 239 additions and 51 deletions
--- a/AGENTS.md
+++ b/AGENTS.md
@ -4,6 +4,34 @@

 ---

+## 0. 文献入口规则 — 必须无条件遵守
+
+**当用户提到以下任何内容时，你必须先加载 paperforge skill，再执行任何操作：**
+
+```
+"查文献" "搜文献" "找文献" "搜一下" "找一下" "查一下"
+"搜论文" "找论文" "查论文" "搜文章" "找文章" "查文章"
+"文献里" "库里" "collection"
+"找证据" "找支持" "找依据" "找参数"
+"读一下" "看看这篇" "精读"
+"DC电场" "电刺激" "galvanotaxis" "电泳" 或其他学术术语
+"保存这次" "记录一下" "提取方法论"
+"review" "看到篇" "这篇论文"
+```
+
+**禁止行为（犯过一次的错误，不再犯）：**
+- ❌ 用 `grep`/`rg`/`glob` 直接搜 vault 文件系统 — 必须用 `$PYTHON -m paperforge search/retrieve/context`
+- ❌ 只用一个搜索工具就下结论 — 必须用多臂策略（retrieve 全文 + search 元数据 + context collection）
+- ❌ 跳过大量搜索结果不告诉用户 — 超过 20 篇必须告知用户总数并提供缩小选项
+- ❌ 在 `retrieve`（语义全文搜索）可用时只用 `search`（元数据搜索）— `retrieve` 能找到正文 Methods 中的内容
+
+**加载方法：**
+
+如果是 OpenCode：`<skill name="paperforge">`
+如果是其他 Agent：读取 `.opencode/skills/paperforge/SKILL.md` 并按流程执行。
+
+---
+
 ## 1. 核心不变式

 1. **CLI 是命令真相源。** 所有 skill、workflow、文档命令引用必须对齐 `paperforge/cli.py`。
@ -62,7 +90,7 @@ JS 读（同步，不推断）:

 ## 3. 安全命令惯例

- 搜索用 `$PYTHON -m paperforge search`，不用 `grep`/`glob` 扫库。
+- 搜索用 `$PYTHON -m paperforge search` / `retrieve` / `context`，不用 `grep`/`rg`/`glob` 扫库。
 - 路径从 bootstrap 或 paper-context 获取，禁止自行拼接。
 - 未完成 paper-context 检查前不读原文（适用于 read-known-paper、deep-analyze-paper）。
 - Reading-log 不是事实源，只能用做复查定位。
--- a/paperforge/commands/agent_context.py
+++ b/paperforge/commands/agent_context.py
@ -19,7 +19,11 @@ COMMANDS = {
    },
    "retrieve": {
        "usage": "paperforge retrieve <query> --json [--limit N]",
-        "purpose": "Search OCR fulltext chunks for evidence paragraphs",
+        "purpose": "Semantic search across OCR fulltext — discover papers by body content, not just title/abstract",
+    },
+    "context": {
+        "usage": "paperforge context <key> | --domain D | --collection P | --all",
+        "purpose": "List all papers in a collection/domain (no truncation) or get single paper context pack",
    },
    "deep": {
        "usage": "/pf-deep <zotero_key>",
@ -36,10 +40,13 @@ COMMANDS = {
 }

 RULES = [
-    "Use paperforge.db via CLI commands before reading individual files.",
+    "Use paperforge CLI commands before reading individual files — never grep/glob the vault directly.",
    "Do not infer paper state from stale frontmatter when memory status is fresh.",
-    "Read source files only after resolving candidates via paper-status or search.",
+    "Read source files only after resolving candidates via paper-status, search, retrieve, or context.",
    "To locate a paper: start with collection scope if known, then expand to full library search.",
+    "Paper discovery must use multi-arm strategy: retrieve (body text) + search (metadata) + context --collection (inventory). Never rely on a single search tool.",
+    "When a search returns > 20 results, present the count to the user and offer to narrow — never silently skip large result sets.",
+    "Check embed status --json (db_exists + chunk_count) before calling retrieve; skip retrieve if vector index is unavailable.",
 ]


--- a/paperforge/skills/paperforge/atoms/retrieval-routing.md
+++ b/paperforge/skills/paperforge/atoms/retrieval-routing.md
@ -10,11 +10,88 @@ Route between vector and memory retrievers based on query type.

 ## 2. Retrieval ladders

-### Ladder A -- Paper Discovery (for discover-papers molecule)
+### Ladder A -- Paper Discovery (multi-arm strategy for discover-papers)

-1. Use `paperforge search` (metadata FTS) to find candidate papers
-2. Enrich top hits with `paperforge paper-context`
-3. Show candidate list to user
+**Prerequisite: check `retrieve` availability**
+
+Before any discovery arm, check if semantic/vector search is available:
+
+```bash
+$PYTHON -m paperforge --vault "$VAULT" embed status --json
+```
+
+If `data.db_exists == true` and `data.chunk_count > 0`, `retrieve` is usable.
+Otherwise, skip Arm 1 and Arm 3 below.
+
+**Strategy: choose discovery method based on user query type**
+
+| User says...                          | Use arms                         |
+|---------------------------------------|----------------------------------|
+| Specific technical term, method, parameter ("bipolar pulses", "galvanotaxis chamber") | **Arm 1** (fulltext) + **Arm 2** (metadata) in parallel |
+| Author name + year, topic title       | **Arm 2** (metadata) + optionally **Arm 1** |
+| "collection X里有什么" / "collection里有什么" | **Arm 3** (collection inventory) |
+| Vague / broad topic                   | **Arm 1** first, then **Arm 2** to supplement |
+
+#### Arm 1 — Fulltext semantic search (if `retrieve` available)
+
+```bash
+$PYTHON -m paperforge --vault "$VAULT" retrieve "<query>" --json --limit 30
+```
+
+- Searches OCR fulltext chunks via vector embedding
+- Catches concepts that appear only in Methods/Results/Discussion (not in title/abstract)
+- Returns up to 30 chunk hits across papers; each hit includes `paper_id`, `section`, page
+- If `ok: false` → skip this arm, proceed with Arm 2
+
+#### Arm 2 — Metadata FTS search (always available)
+
+```bash
+$PYTHON -m paperforge --vault "$VAULT" search "<query>" --json --limit 30
+    [--domain "<domain>"] \
+    [--year-from <N>] [--year-to <N>] \
+    [--ocr done|pending] \
+    [--lifecycle <lifecycle>]
+```
+
+- Uses FTS5 on title, abstract, authors, journal, domain, collection_path
+- Fast, always works
+- Limit 30 by default; user can request more
+
+#### Arm 3 — Collection/domain inventory (full list, no truncation)
+
+When the user wants to know what's in a collection or domain:
+
+```bash
+$PYTHON -m paperforge --vault "$VAULT" context --collection "<collection_path>" --json
+$PYTHON -m paperforge --vault "$VAULT" context --domain "<domain>" --json
+```
+
+- Returns **every** paper in the collection/domain (no truncation)
+- `context --collection` does prefix match on collection path
+- `context --domain` does exact match on domain field
+- If the list is very large (50+), summarize by year/author and ask user if they want to narrow
+
+#### Result deduplication
+
+- Collect all results from used arms into a single list
+- Deduplicate by `zotero_key` (keep first occurrence)
+- If `Arm 1` was used: prefer the Arm 1 entry (has fulltext match signal)
+- After dedup, enrich top hits with `paper-context`
+
+#### Large result set handling
+
+- If any arm returns > 20 results, tell the user the total count
+- Offer options: increase limit, narrow by year/domain/author, or show top N
+- The `search` default limit is 20; `retrieve` default is 5 (increase to 30 for discovery)
+- `context --collection` already returns all — no pagination needed
+
+#### Fallback: when `retrieve` is unavailable
+
+If `embed status` shows no vector index, skip Arm 1 entirely:
+
+> Semantic fulltext search unavailable (vector index not built). Falling back to metadata search only. Run `paperforge embed build` to enable fulltext discovery.
+
+Then proceed with Arm 2 only (or Arm 3 if collection/domain query).

 ### Ladder B -- Evidence Retrieval with rg

@ -44,20 +121,30 @@ When runtime-health indicates no papers have OCR or fulltext available:

 Present a candidate paper list only; no snippet verification is performed.

-## 4. Recommended default limits
+## 4. Recommended limits

- Metadata candidate set: top 10-20 papers
- Fulltext/snippet verification: top 3-5 papers
+| Phase | Default limit | Why |
+|-------|---------------|-----|
+| Discovery — `search` | 30 | Catch papers beyond top 20 |
+| Discovery — `retrieve` | 30 | Cover fulltext body matches |
+| Discovery — `context --collection` | no limit | Must list everything |
+| Enrichment — `paper-context` | top 10 | Enough for user to decide |
+| Evidence snippets | top 5 | Deep verification is expensive |

 ## 5. Semantic degradation rule

 ```text
-If semantic_enabled=false or semantic_ready=false:
- Do not call retrieve as primary path
- Fall back to metadata + rg/grep + paper-context
+Retrieve availability check (run before any discovery):
+  paperforge embed status --json  →  db_exists == true && chunk_count > 0

-If semantic is available:
- Use only for candidate expansion
- Every semantic hit must be verified by rg/fulltext/paper-context before use
+If retrieve is NOT available (no vector index):
+- Skip all retrieve calls in every molecule
+- Fall back to metadata-only strategy (search + context --collection)
+- Inform user: "Fulltext semantic search unavailable. Run 'paperforge embed build' to enable body-text discovery."
+
+If retrieve IS available:
+- Use as primary discovery arm for technical/method queries
+- Every semantic hit should be verified by paper-context before presenting as evidence
+- Retrieve is a discovery tool, not a verification tool
 ```

--- a/paperforge/skills/paperforge/molecules/discover-papers.md
+++ b/paperforge/skills/paperforge/molecules/discover-papers.md
@ -24,47 +24,87 @@
 - **范围**：domain（如"骨科"）、不指定=全库
 - **过滤条件**：OCR 状态、年份范围（`--year-from`/`--year-to`）、lifecycle

-### Step 2: 执行元数据搜索（`paperforge search`）
+### Step 2: 多臂搜索策略
+
+打开 `atoms/retrieval-routing.md` 参照 Ladder A。
+
+**先检查 `retrieve` 是否可用：**

 ```bash
-$PYTHON -m paperforge --vault "$VAULT" search <query> --json --limit 15 \
+$PYTHON -m paperforge --vault "$VAULT" embed status --json
+```
+
+取 `data.db_exists` 和 `data.chunk_count`。仅当 `db_exists == true && chunk_count > 0` 时 `retrieve` 可用。
+
+**根据用户意图选择搜索臂：**
+
+| 用户说...                           | 执行臂                              |
+|--------------------------------------|-------------------------------------|
+| 技术术语/方法/参数（"bipolar pulses"、 "galvanotaxis"） | 先用 **Arm 1**（全文语义）再 **Arm 2**（元数据补充） |
+| 作者+年份、主题关键词               | **Arm 2**（元数据）为主，可选 Arm 1 |
+| "collection X里有什么" / "库里有什么" | **Arm 3** （collection 列举）       |
+| 宽泛主题                             | **Arm 1** 先搜，**Arm 2** 补充      |
+
+#### Arm 1 — 全文语义搜索（仅当 `retrieve` 可用）
+
+```bash
+$PYTHON -m paperforge --vault "$VAULT" retrieve "<query>" --json --limit 30
+```
+
+- 搜索 OCR 全文块的向量嵌入，能匹配正文 Methods/Results/Discussion 中的概念
+- 返回 JSON：`data.chunks[]` 包含 `paper_id`（即 `zotero_key`）、`section`、`page_number`、`chunk_text`
+- 从 chunks 中提取唯一 paper_id 列表作为候选论文集合
+- 如果 `ok: false` → 跳过此臂
+
+#### Arm 2 — 元数据 FTS 搜索（始终可用）
+
+```bash
+$PYTHON -m paperforge --vault "$VAULT" search "<query>" --json --limit 30 \
    [--domain "<domain>"] \
    [--year-from <N>] [--year-to <N>] \
    [--ocr <done|pending|failed|processing>] \
    [--lifecycle <indexed|pdf_ready|fulltext_ready|deep_read_done>]
 ```

-返回 JSON 结构（候选论文清单）：
-```json
-{
-  "ok": true,
-  "data": {
-    "query": "<query>",
-    "matches": [
-      {
-        "zotero_key": "ABC12345",
-        "title": "论文标题",
-        "year": "2024",
-        "first_author": "Smith",
-        "domain": "骨科",
-        "ocr_status": "done",
-        "deep_reading_status": "pending",
-        "lifecycle": "pdf_ready",
-        "has_pdf": true
-      }
-    ],
-    "count": 5
-  }
-}
+- FTS5 搜索标题、摘要、作者、期刊、domain、collection 路径
+- 返回 JSON：`data.matches[]` 包含 `zotero_key`、`title`、`year`、`first_author` 等
+
+#### Arm 3 — Collection/Domain 列举（完整列表，不截断）
+
+```bash
+$PYTHON -m paperforge --vault "$VAULT" context --collection "<collection_path>" --json
+$PYTHON -m paperforge --vault "$VAULT" context --domain "<domain>" --json
 ```

- 如果 `ok: false` → 报告 `error.message`，问用户是否换搜索词
- 如果 `data.count == 0` → 告知用户无结果，建议换词或扩大范围
- 如果 `data.count > 0` → 进入 Step 3
+- 返回 collection 或 domain 下**所有**论文（无 limit 截断）
+- `context --collection` 按 collection 路径前缀匹配
+- `context --domain` 按 domain 字段精确匹配
+- 如果列表超过 50 篇，按年份/作者分组摘要后问用户是否缩小范围
+
+#### 结果去重
+
+- 合并所有臂的结果
+- 按 `zotero_key` 去重（保留首次出现）
+- 如果同时用了 Arm 1+2：优先保留 Arm 1 的结果（包含全文命中信号）
+- 去重后进入 Step 3 富化
+
+#### 大规模结果集处理
+
+- 如果任何一臂返回超过 20 篇，告知用户总数
+- 主动提供选项：加 limit、按年份/domain/作者缩小、只展示前 N 篇
+- 不要跳过或隐瞒大量结果的存在
+
+#### 当 `retrieve` 不可用时的降级
+
+如果 `embed status` 显示无向量索引：
+
+> 语义全文搜索不可用（向量索引未构建）。已降级到元数据搜索。运行 `paperforge embed build` 后可开启正文发现。
+
+然后只运行 Arm 2（或 Arm 3 如果是 collection/domain 查询）。

 ### Step 3: Top-hit 富化（`paperforge paper-context`）

-对每个 match，调 `paper-context` 获取更详细的可读状态：
+对候选列表中的 **前 10 篇**（如果超过 10 篇，只富化前 10），调 `paper-context` 获取详细状态：

 ```bash
 $PYTHON -m paperforge --vault "$VAULT" paper-context <zotero_key> --json
@ -72,20 +112,26 @@ $PYTHON -m paperforge --vault "$VAULT" paper-context <zotero_key> --json

 目的：拿到 `ocr_status`、`prior_notes` 数量、`analyze` 状态，帮助用户判断哪些可以直接读。

+如果列表超过 10 篇，在展示时告知用户总数和只富化了前 N 篇。
+
 ### Step 4: 展示候选清单

 格式（每条一行）：

 ```
-找到 N 篇匹配 "<query>"：
+找到 N 篇匹配 "<query>"（来自 [全文语义/元数据/collection列举]）：

 [1] ABC12345 | Smith 2024 | 论文标题 | 骨科 | OCR: done | 精读: pending | 阅读笔记: 3
 [2] DEF67890 | Jones 2023 | 论文标题 | 骨科 | OCR: done | 精读: done   | 阅读笔记: 0
 [3] GHI11111 | Wang 2022  | 论文标题 | 骨科 | OCR: pending |           | 阅读笔记: 0
+
+（共 N 篇，仅展示前 M 篇。如需查看更多，请说"更多"或指定年份/关键词缩小范围）
 ```

 关键字段：`zotero_key`、`first_author`、`year`、`title`、`domain`、`ocr_status`、`deep_reading_status`

+**来源标注**：如果使用了多臂搜索，在顶部标注命中来自哪个搜索臂（`全文语义` / `元数据` / `collection列举`）。
+
 ### Step 5: 等用户选择后续操作

 展示候选后不要自己决定下一步。等用户说：
--- a/paperforge/skills/paperforge/molecules/find-supporting-evidence.md
+++ b/paperforge/skills/paperforge/molecules/find-supporting-evidence.md
@ -27,15 +27,35 @@

 ### Step 2: 检索梯级选择（`atoms/retrieval-routing.md`）

-根据运行时状态选择合适的梯级：
+**先检查 `retrieve` 是否可用：**

-1. **Ladder B**（优先）-- 用 `rg` 在全文中定位精确证据
+```bash
+$PYTHON -m paperforge --vault "$VAULT" embed status --json
+```
+
+仅当 `data.db_exists == true && data.chunk_count > 0` 时 `retrieve` 可用。
+
+根据运行时状态，从以下路径中选择：
+
+1. **Ladder B1**（首选，当 `retrieve` 可用）-- 语义全文快速定位
+   - 直接调 `paperforge retrieve <query> --json --limit 30` 获取语义匹配的全文块
+   - `retrieve` 返回的 chunks 已包含 `section`、`page_number`、`chunk_text`、`paper_id`
+   - 按论文分组组织结果 → 直接进入 Step 3 展示
+   - 如需精确定位验证，再用 `rg` / `grep` 在论文全文中确认
+
+2. **Ladder B2**（备选，无 `retrieve` 但有 `rg`）-- 元数据→全文 grep
   - 先用 `paperforge search` 生成元数据候选集
   - 用 `runtime-health` 或 `paper-context` 筛选有 OCR/全文的论文
   - 在解析后的全文中运行 `rg` 定位匹配片段
-2. **Ladder C**（回退）-- 无 `rg` 时用 `grep`/`findstr`
-3. **Ladder D**（补充）-- 语义候选扩展（仅当 `semantic_enabled && semantic_ready`）
-4. **元数据降级** -- 当没有任何论文有 OCR/全文时，只出候选论文列表，不做片段验证
+
+3. **Ladder C**（回退）-- 无 `rg` 时用 `grep`/`findstr`
+   - 同上流程但用系统搜索工具代替 rg
+
+4. **Ladder D**（补充）-- 当 Ladder B1 命中太少时，用 `search` 做元数据补充
+   - 取 `retrieve` 结果 + `search` 结果的并集
+   - 去重后展示更完整的证据列表
+
+5. **元数据降级** -- 当没有任何论文有 OCR/全文时，只出候选论文列表，不做片段验证

 ### Step 3: 展示分组证据命中（grouped evidence hits with snippets）