mirror of
https://github.com/lllin000/PaperForge.git
synced 2026-07-22 06:50:53 +00:00
fix: improve search accuracy with multi-token LIKE and guided agent search queries
- _like_query: split multi-token queries into per-field AND matching - lookup_paper: same multi-token AND for title lookup - read-known-paper.md: restructure Step 1 by known-info type (zotero_key, DOI, author+year, title keyword, fuzzy) - discover-papers.md Arm 2: note on author+year search pattern - retrieval-routing.md Arm 2: same author+year search guidance
This commit is contained in:
parent
69d822daff
commit
f68f45f3a2
5 changed files with 49 additions and 15 deletions
|
|
@ -82,7 +82,18 @@ def _fts_query(conn, query, filter_clause, filter_params, limit):
|
|||
|
||||
|
||||
def _like_query(conn, query, filter_clause, filter_params, limit):
|
||||
like_param = f"%{query}%"
|
||||
tokens = re.findall(r"[\w\u4e00-\u9fff]+", query)
|
||||
if not tokens:
|
||||
tokens = [query]
|
||||
|
||||
field_clauses = []
|
||||
params = []
|
||||
for field in ["p.title", "p.abstract", "p.doi", "p.citation_key"]:
|
||||
token_clauses = [f"{field} LIKE ?" for _ in tokens]
|
||||
params.extend(f"%{t}%" for t in tokens)
|
||||
field_clauses.append("(" + " AND ".join(token_clauses) + ")")
|
||||
|
||||
where_clause = "(" + " OR ".join(field_clauses) + ")"
|
||||
sql = f"""
|
||||
SELECT p.zotero_key, p.citation_key, p.title, p.year, p.doi,
|
||||
p.first_author, p.journal, p.domain, p.lifecycle,
|
||||
|
|
@ -90,10 +101,10 @@ def _like_query(conn, query, filter_clause, filter_params, limit):
|
|||
substr(p.abstract, 1, 300) as abstract,
|
||||
0 as rank
|
||||
FROM papers p
|
||||
WHERE (p.title LIKE ? OR p.abstract LIKE ? OR p.doi LIKE ? OR p.citation_key LIKE ?){filter_clause}
|
||||
WHERE {where_clause}{filter_clause}
|
||||
ORDER BY p.year DESC
|
||||
LIMIT ?
|
||||
"""
|
||||
conn.row_factory = sqlite3.Row
|
||||
rows = conn.execute(sql, [like_param, like_param, like_param, like_param] + filter_params + [limit]).fetchall()
|
||||
rows = conn.execute(sql, params + filter_params + [limit]).fetchall()
|
||||
return [dict(r) for r in rows]
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@ from __future__ import annotations
|
|||
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
from paperforge.memory.builder import compute_hash
|
||||
|
|
@ -79,7 +80,7 @@ def get_memory_status(vault: Path) -> dict:
|
|||
|
||||
def _entry_from_row(row) -> dict:
|
||||
"""Reconstruct an entry dict from a papers row (sqlite3.Row)."""
|
||||
entry = {k: row[k] for k in row.keys()}
|
||||
entry = {k: row[k] for k in row}
|
||||
for key in ("has_pdf", "do_ocr", "analyze"):
|
||||
if key in entry and entry[key] is not None:
|
||||
entry[key] = bool(entry[key])
|
||||
|
|
@ -108,11 +109,16 @@ def lookup_paper(conn, query: str) -> list[dict]:
|
|||
if row:
|
||||
return [_entry_from_row(row)]
|
||||
|
||||
tokens = re.findall(r"[\w\u4e00-\u9fff]+", q)
|
||||
if not tokens:
|
||||
tokens = [q]
|
||||
title_conditions = " AND ".join("LOWER(title) LIKE ?" for _ in tokens)
|
||||
title_params = [f"%{t.lower()}%" for t in tokens]
|
||||
rows = conn.execute(
|
||||
"""SELECT * FROM papers
|
||||
WHERE LOWER(title) LIKE '%' || LOWER(?) || '%'
|
||||
f"""SELECT * FROM papers
|
||||
WHERE {title_conditions}
|
||||
LIMIT 20""",
|
||||
(q,),
|
||||
title_params,
|
||||
).fetchall()
|
||||
if rows:
|
||||
return [_entry_from_row(r) for r in rows]
|
||||
|
|
|
|||
|
|
@ -56,6 +56,7 @@ $PYTHON -m paperforge --vault "$VAULT" search "<query>" --json --limit 30
|
|||
- Uses FTS5 on title, abstract, authors, journal, domain, collection_path
|
||||
- Fast, always works
|
||||
- Limit 30 by default; user can request more
|
||||
- **Author+year search**: put author name in `<query>`, year in `--year-from`/`--year-to` (e.g. `search "Brighton" --year-from 1983 --year-to 1983`)
|
||||
|
||||
#### Arm 3 — Collection/domain inventory (full list, no truncation)
|
||||
|
||||
|
|
|
|||
|
|
@ -68,6 +68,7 @@ $PYTHON -m paperforge --vault "$VAULT" search "<query>" --json --limit 30 \
|
|||
|
||||
- FTS5 搜索标题、摘要、作者、期刊、domain、collection 路径
|
||||
- 返回 JSON:`data.matches[]` 包含 `zotero_key`、`title`、`year`、`first_author` 等
|
||||
- **作者+年份搜索**:query 放作者名,年份放 `--year-from`/`--year-to`(如 `search "Brighton" --year-from 1983 --year-to 1983`)
|
||||
|
||||
#### Arm 3 — Collection/Domain 列举(完整列表,不截断)
|
||||
|
||||
|
|
|
|||
|
|
@ -25,31 +25,46 @@
|
|||
|
||||
### Step 1: 定位论文
|
||||
|
||||
用户可能给 zotero_key、DOI、标题片段、作者+年份。按以下方式查找:
|
||||
用户可能给 zotero_key、DOI、标题关键词、作者+年份。按以下方式查找:
|
||||
|
||||
**如果用户给的是 zotero_key:**
|
||||
**如果知道 zotero_key:**
|
||||
|
||||
```bash
|
||||
$PYTHON -m paperforge --vault "$VAULT" paper-context <zotero_key> --json
|
||||
```
|
||||
|
||||
返回 JSON 包含 paper 元数据、OCR 状态、prior_notes 等。
|
||||
**如果知道 DOI:**
|
||||
|
||||
**如果用户给的是 DOI、标题片段、作者+年份,先解析成候选:**
|
||||
```bash
|
||||
$PYTHON -m paperforge --vault "$VAULT" paper-context <DOI> --json
|
||||
```
|
||||
|
||||
**如果知道 作者+年份:**
|
||||
|
||||
```bash
|
||||
$PYTHON -m paperforge --vault "$VAULT" search "<author>" --year-from <YYYY> --year-to <YYYY> --json --limit 10
|
||||
```
|
||||
|
||||
**如果知道 标题关键词:**
|
||||
|
||||
```bash
|
||||
$PYTHON -m paperforge --vault "$VAULT" search "<关键词>" --json --limit 10
|
||||
```
|
||||
|
||||
**如果只有模糊描述:**
|
||||
|
||||
```bash
|
||||
$PYTHON -m paperforge --vault "$VAULT" paper-status "<query>" --json
|
||||
```
|
||||
|
||||
如果 `paper-status` 直接唯一命中,取返回的 `zotero_key` 再调 `paper-context`。
|
||||
|
||||
**paper-status 无法唯一定位时的备选:**
|
||||
如果无唯一命中:
|
||||
|
||||
```bash
|
||||
$PYTHON -m paperforge --vault "$VAULT" search "<query>" --json --limit 5
|
||||
```
|
||||
|
||||
如果多候选,列出让用户选;用户选定后,再用选中的 `zotero_key` 调 `paper-context`。
|
||||
无论哪种方式命中,取 `zotero_key` 调 `paper-context` 获取完整信息。
|
||||
如果多候选,列出让用户选。
|
||||
如果无结果,告知用户并停止。
|
||||
|
||||
### Step 2: 加载文献内容
|
||||
|
|
|
|||
Loading…
Reference in a new issue