---
phase: 03-obsidian-bases-config-aware
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - pipeline/worker/scripts/literature_pipeline.py
  - paperforge/config.py
 autonomous: true
requirements:
  - BASE-01
  - BASE-02

must_haves:
  truths:
    - "Domain Base files contain exactly 8 named views matching the production workflow"
    - "Hardcoded paths like '03_Resources' are replaced with ${SCREAMING_SNAKE_CASE} placeholders"
    - "Placeholders are substituted with vault-relative paths at generation time"
    - "Incremental refresh ONLY updates the 8 standard PaperForge views; user-defined views and customizations are preserved"
  artifacts:
    - path: "pipeline/worker/scripts/literature_pipeline.py"
      provides: "Refactored Base file generation with 8-view support, incremental merge, and placeholder substitution"
      contains: "build_base_views, substitute_config_placeholders, merge_base_views"
    - path: "03-CONTEXT.md"
      provides: "Locked decisions from discuss-phase"
---

<objective>
Refactor the Base file generation system to support 8 views per domain with config-aware path placeholders and INCREMENTAL MERGE (not full overwrite) so user customizations are preserved.

Purpose: Generate Domain Base files matching the real vault 8-view workflow. Key requirement: user-added views and custom view settings are NEVER overwritten on refresh — only the 8 standard PaperForge views are updated incrementally.
Output: Refactored literature_pipeline.py with 8-view templates, placeholder substitution, and incremental merge.
</objective>

<read_first>
- pipeline/worker/scripts/literature_pipeline.py (lines 195-254: current write_base_file, ensure_base_views)
- paperforge/config.py (paperforge_paths function for vault-relative path resolution)
- .planning/phases/03-obsidian-bases-config-aware/03-CONTEXT.md (locked decisions)
</read_first>

<tasks>

<task type="auto">
<name>Task 1: Implement build_base_views() — 8-view template generator</name>
<read_first>
- pipeline/worker/scripts/literature_pipeline.py (lines 195-254)
</read_first>
<acceptance_criteria>
- grep "def build_base_views" pipeline/worker/scripts/literature_pipeline.py returns a match
- build_base_views("骨科") returns a list of exactly 8 dicts
- Each dict has keys: name, order, filter
- View names are exactly: 控制面板, 推荐分析, 待 OCR, OCR 完成, 待深度阅读, 深度阅读完成, 正式卡片, 全记录
- View filters match the 8 definitions in 03-CONTEXT.md Section 4
</acceptance_criteria>
<action>
Add the following functions to `pipeline/worker/scripts/literature_pipeline.py` after `base_markdown_filter()` (after line 203):

```python
def build_base_views(domain: str) -> list[dict]:
    """Build the 8-view list for a domain Base file.

    Args:
        domain: The domain name (e.g., "骨科") — passed for compatibility but each view has fixed name/filter.

    Returns:
        List of 8 view dicts, each with keys: name (str), order (list), filter (str|None).
    """
    return [
        {
            "name": "控制面板",
            "order": ["file.name", "title", "year", "has_pdf", "do_ocr", "analyze", "ocr_status", "deep_reading_status", "pdf_path", "fulltext_md_path"],
            "filter": None,
        },
        {
            "name": "推荐分析",
            "order": ["year", "title", "has_pdf", "do_ocr", "analyze", "ocr_status", "deep_reading_status", "pdf_path", "fulltext_md_path"],
            "filter": 'analyze = true AND recommend_analyze = true',
        },
        {
            "name": "待 OCR",
            "order": ["year", "title", "has_pdf", "do_ocr", "ocr_status", "pdf_path"],
            "filter": 'do_ocr = true AND ocr_status = "pending"',
        },
        {
            "name": "OCR 完成",
            "order": ["year", "title", "has_pdf", "do_ocr", "ocr_status", "pdf_path"],
            "filter": 'ocr_status = "done"',
        },
        {
            "name": "待深度阅读",
            "order": ["year", "title", "has_pdf", "do_ocr", "analyze", "ocr_status", "deep_reading_status", "pdf_path"],
            "filter": 'analyze = true AND ocr_status = "done" AND deep_reading_status = "pending"',
        },
        {
            "name": "深度阅读完成",
            "order": ["year", "title", "has_pdf", "do_ocr", "analyze", "ocr_status", "deep_reading_status", "pdf_path"],
            "filter": 'deep_reading_status = "done"',
        },
        {
            "name": "正式卡片",
            "order": ["title", "year", "has_pdf", "deep_reading_status", "pdf_path"],
            "filter": 'deep_reading_status = "done"',
        },
        {
            "name": "全记录",
            "order": ["title", "year", "has_pdf", "do_ocr", "analyze", "ocr_status", "deep_reading_status", "pdf_path", "fulltext_md_path"],
            "filter": None,
        },
    ]
```

STANDARD_VIEW_NAMES constant (add near top of file):
```python
STANDARD_VIEW_NAMES = frozenset([
    "控制面板", "推荐分析", "待 OCR", "OCR 完成",
    "待深度阅读", "深度阅读完成", "正式卡片", "全记录"
])
```
</action>
<verify>
grep -c '"name": "控制面板"' pipeline/worker/scripts/literature_pipeline.py
# Returns 1
grep -c 'build_base_views' pipeline/worker/scripts/literature_pipeline.py
# Returns at least 1 (definition) + 1+ calls
grep 'STANDARD_VIEW_NAMES' pipeline/worker/scripts/literature_pipeline.py
# Returns match
</verify>
<done>build_base_views() exists and returns exactly 8 views with correct names and filters. STANDARD_VIEW_NAMES frozenset defined.</done>

<task type="auto">
<name>Task 2: Implement substitute_config_placeholders()</name>
<read_first>
- pipeline/worker/scripts/literature_pipeline.py
</read_first>
<acceptance_criteria>
- grep "def substitute_config_placeholders" pipeline/worker/scripts/literature_pipeline.py returns a match
- substitute_config_placeholders(content, paths) with paths containing library_records, literature, control returns content with all ${LIBRARY_RECORDS}, ${LITERATURE}, ${CONTROL_DIR} replaced
- If a placeholder key is not in paths, it is left unchanged (no error)
</acceptance_criteria>
<action>
Add after `build_base_views()`:

```python
def substitute_config_placeholders(content: str, paths: dict[str, Path]) -> str:
    """Replace ${SCREAMING_SNAKE_CASE} path placeholders with vault-relative paths.

    Args:
        content: The Base file content string with ${PLACEHOLDER} tokens.
        paths: dict of path key -> Path objects (from paperforge_paths()).

    Returns:
        Content with placeholders replaced by vault-relative paths.
        Unrecognized placeholders are left unchanged.
    """
    substitutions = {
        "LIBRARY_RECORDS": paths.get("library_records"),
        "LITERATURE": paths.get("literature"),
        "CONTROL_DIR": paths.get("control"),
    }
    result = content
    for key, path in substitutions.items():
        if path is not None:
            vault = paths.get("vault")
            if vault is not None:
                try:
                    rel = path.relative_to(vault)
                    result = result.replace(f"${{{key}}}", str(rel).replace("\\", "/"))
                except ValueError:
                    result = result.replace(f"${{{key}}}", str(path).replace("\\", "/"))
    return result
```
</action>
<verify>
python -c "from pipeline.worker.scripts.literature_pipeline import substitute_config_placeholders; from pathlib import Path; c = '\${LIBRARY_RECORDS}/骨科'; r = substitute_config_placeholders(c, {'library_records': Path('D:/Vault/03_Resources/LiteratureControl/library-records'), 'vault': Path('D:/Vault')}); print(r)}"
# Output should NOT contain \${LIBRARY_RECORDS}
</verify>
<done>substitute_config_placeholders() replaces all 3 placeholders correctly.</done>

<task type="auto">
<name>Task 3: Implement merge_base_views() — incremental merge (preserves user views)</name>
<read_first>
- pipeline/worker/scripts/literature_pipeline.py
</read_first>
<acceptance_criteria>
- grep "def merge_base_views" pipeline/worker/scripts/literature_pipeline.py returns a match
- merge_base_views(existing_content, new_paperforge_views) returns merged content
- User-defined views (views NOT in STANDARD_VIEW_NAMES) are preserved in the output
- PaperForge standard views are always replaced with the new versions (incremental update)
- If existing content is empty/None, returns fresh generated content (full generation on first run)
</acceptance_criteria>
<action>
Add a new function `merge_base_views()` in `literature_pipeline.py`. This is the core of the incremental refresh logic:

```python
PAPERFORGE_VIEW_PREFIX = "# PAPERFORGE_VIEW: "

def _render_views_section(views: list[dict]) -> str:
    """Render a list of view dicts to YAML views: section."""
    lines = []
    for v in views:
        lines.append(f"  - type: table")
        lines.append(f'    name: "{v["name"]}"')
        lines.append(f"    order:")
        for col in v["order"]:
            lines.append(f"      - {col}")
        if v["filter"]:
            lines.append(f'    filter: \'{v["filter"]}\'')
    return "\n".join(lines)


def merge_base_views(existing_content: str | None, new_views: list[dict]) -> str:
    """Incrementally merge standard PaperForge views into an existing .base file.

    Strategy:
    - PaperForge generates exactly 8 views with known names (STANDARD_VIEW_NAMES).
    - Any OTHER views in the existing file are user-defined and MUST be preserved.
    - Each PaperForge view is preceded by a PAPERFORGE_VIEW_PREFIX comment marker.
    - On refresh: replace ALL PaperForge views (identified by prefix) with fresh ones.
    - User views (no prefix) are left completely untouched.

    Args:
        existing_content: Raw text of existing .base file (or None/empty for fresh generation).
        new_views: List of 8 view dicts from build_base_views().

    Returns:
        Merged .base file content with PaperForge views updated, user views preserved.
    """
    PROPERTIES_YAML = """properties:
  zotero_key:
    displayName: "Zotero Key"
  title:
    displayName: "Title"
  year:
    displayName: "Year"
  has_pdf:
    displayName: "PDF"
  do_ocr:
    displayName: "OCR"
  analyze:
    displayName: "Analyze"
  ocr_status:
    displayName: "OCR Status"
  deep_reading_status:
    displayName: "Deep Reading"
  pdf_path:
    displayName: "PDF Path"
  fulltext_md_path:
    displayName: "Fulltext"
"""

    # First-run: no existing content — generate fresh
    if not existing_content or not existing_content.strip():
        fresh_views_yaml = _render_views_section(new_views)
        return f"""filters:
  and:
    - file.inFolder("{new_views[0]['name']}")  # placeholder, will be overwritten by caller
{PROPERTIES_YAML}
views:
{fresh_views_yaml}"""

    # Parse existing file into sections:
    # We need to extract: (a) the header/filters section, (b) all view blocks, (c) user views to preserve

    # Strategy: find all view blocks and separate PaperForge ones from user ones
    lines = existing_content.split("\n")
    pf_view_names_seen = set()
    new_pf_blocks = []  # list of rendered PaperForge view blocks (with prefix markers)

    for v in new_views:
        rendered = (
            f"{PAPERFORGE_VIEW_PREFIX}{v['name']}\n"
            f"  - type: table\n"
            f'    name: "{v["name"]}"\n'
            f"    order:\n"
        )
        for col in v["order"]:
            rendered += f"      - {col}\n"
        if v["filter"]:
            rendered += f'    filter: \'{v["filter"]}\'\n'
        new_pf_blocks.append(rendered.rstrip("\n"))
        pf_view_names_seen.add(v["name"])

    # Now rebuild the file:
    # - Keep everything BEFORE the first PAPERFORGE_VIEW_PREFIX or first view: line
    # - Insert new PaperForge blocks with prefix markers
    # - Preserve any user views (non-PaperForge views found between/after PaperForge blocks)

    # Find the "views:" line and everything after it
    views_start_idx = None
    for i, line in enumerate(lines):
        if line.strip().startswith("views:"):
            views_start_idx = i
            break

    if views_start_idx is None:
        # Malformed .base file — regenerate fresh
        fresh_views_yaml = _render_views_section(new_views)
        return f"""{existing_content}

# --- PaperForge views regenerated (views: section was missing) ---
{PROPERTIES_YAML}
views:
{fresh_views_yaml}"""

    # Header is everything up to and including "views:" line
    header_lines = lines[:views_start_idx + 1]

    # Extract existing view blocks (from views: onwards)
    # We'll collect user views (unmarked) and rebuild everything
    rebuilt_views_lines = []
    i = views_start_idx + 1
    while i < len(lines):
        line = lines[i]
        # Check for PAPERFORGE_VIEW_PREFIX marker
        if line.startswith(PAPERFORGE_VIEW_PREFIX):
            view_name = line[len(PAPERFORGE_VIEW_PREFIX):].strip()
            # Skip this entire block until the next view (next "- type: table" or end)
            i += 1
            # Skip lines belonging to this view (until next "- type: table" or equal/higher indentation)
            while i < len(lines):
                next_line = lines[i]
                if next_line.startswith("- type: table") or (next_line.strip() and not next_line.startswith(" ") and not next_line.startswith("\t")):
                    break
                i += 1
            # Add the new PaperForge block for this view name
            pf_block = next((b for b in new_pf_blocks if b.startswith(f"{PAPERFORGE_VIEW_PREFIX}{view_name}\n")), None)
            if pf_block:
                rebuilt_views_lines.append(pf_block)
            continue
        elif line.strip().startswith("- type: table"):
            # This is a user view (no PAPERFORGE_VIEW_PREFIX) — preserve it
            view_block_lines = [line]
            i += 1
            while i < len(lines):
                next_line = lines[i]
                # End of view block: next "- type: table" or non-indented content
                if next_line.strip().startswith("- type: table") or (next_line.strip() and not next_line.startswith(" ") and not next_line.startswith("\t") and not next_line.strip().startswith("#") and not next_line[0] in " \t"):
                    break
                view_block_lines.append(next_line)
                i += 1
            rebuilt_views_lines.append("\n".join(view_block_lines))
            continue
        else:
            i += 1

    # Assemble final content
    result_lines = header_lines + rebuilt_views_lines
    return "\n".join(result_lines)
```

Note: This is a simplified but functional incremental merge. The key guarantee is:
1. All 8 PaperForge views are ALWAYS replaced with fresh data on refresh
2. Any view in the existing file whose name is NOT in STANDARD_VIEW_NAMES is preserved verbatim
3. User customizations to view order, filter expressions, etc. on standard views are also replaced (this is the trade-off — standard views are always freshly generated to match the current workflow definition)
</action>
<verify>
python -c "
from pipeline.worker.scripts.literature_pipeline import merge_base_views, build_base_views, STANDARD_VIEW_NAMES
# Simulate: user added a custom view 'My Custom View' to existing base
existing = '''
filters:
  and:
    - file.inFolder(\"骨科\")
views:
  - type: table
    name: \"控制面板\"
    order:
      - file.name
# PAPERFORGE_VIEW: 推荐分析
  - type: table
    name: \"推荐分析\"
    order:
      - year
    filter: 'analyze = true'
  - type: table
    name: \"My Custom View\"
    order:
      - title
'''
views = build_base_views('骨科')
result = merge_base_views(existing, views)
# Custom view should be preserved
assert 'My Custom View' in result, 'User view was lost'
# PaperForge views should be fresh
assert '推荐分析' in result
assert result.count('type: table') >= 8, f'Expected at least 8 views, got {result.count(\"type: table\")}'
print('Merge test passed. Views:', result.count('type: table'))
"
</verify>
<done>merge_base_views() correctly preserves user-defined views while updating standard PaperForge views.</done>

<task type="auto">
<name>Task 4: Update ensure_base_views() — incremental refresh with placeholder substitution</name>
<read_first>
- pipeline/worker/scripts/literature_pipeline.py (lines 244-254: current ensure_base_views)
</read_first>
<acceptance_criteria>
- grep "def ensure_base_views" pipeline/worker/scripts/literature_pipeline.py returns match
- ${LIBRARY_RECORDS} placeholder appears in Base YAML before substitution
- After substitution, paths are vault-relative and contain no placeholders
- PaperForge.base and Literature Hub.base are also generated
- Each standard PaperForge view has PAPERFORGE_VIEW_PREFIX marker
</acceptance_criteria>
<action>
Replace the existing `ensure_base_views()` function (lines 244-254) with:

```python
PAPERFORGE_VIEW_PREFIX = "# PAPERFORGE_VIEW: "  # already defined in Task 3

PROPERTIES_YAML = """properties:
  zotero_key:
    displayName: "Zotero Key"
  title:
    displayName: "Title"
  year:
    displayName: "Year"
  has_pdf:
    displayName: "PDF"
  do_ocr:
    displayName: "OCR"
  analyze:
    displayName: "Analyze"
  ocr_status:
    displayName: "OCR Status"
  deep_reading_status:
    displayName: "Deep Reading"
  pdf_path:
    displayName: "PDF Path"
  fulltext_md_path:
    displayName: "Fulltext"
"""

def _build_base_yaml(folder_filter: str, views: list[dict]) -> str:
    """Build complete .base YAML with PAPERFORGE_VIEW_PREFIX markers on each view."""
    views_yaml = ""
    for v in views:
        views_yaml += f"{PAPERFORGE_VIEW_PREFIX}{v['name']}\n"
        views_yaml += f"  - type: table\n"
        views_yaml += f'    name: "{v["name"]}"\n'
        views_yaml += f"    order:\n"
        for col in v["order"]:
            views_yaml += f"      - {col}\n"
        if v["filter"]:
            views_yaml += f'    filter: \'{v["filter"]}\'\n'
    return f"""filters:
  and:
    - file.inFolder("{folder_filter}")
{PROPERTIES_YAML}
views:
{views_yaml}"""


def ensure_base_views(vault: Path, paths: dict[str, Path], config: dict) -> None:
    """Generate/refresh Domain Base files with incremental merge (preserves user views).

    Each PaperForge standard view is marked with PAPERFORGE_VIEW_PREFIX. On refresh,
    only PaperForge views are replaced; user-defined views are preserved.
    """
    paths["bases"].mkdir(parents=True, exist_ok=True)

    def refresh_base(base_path: Path, folder_filter: str, views: list[dict]) -> None:
        """Refresh a single .base file: merge PaperForge views, preserve user views."""
        if base_path.exists():
            existing = base_path.read_text(encoding="utf-8")
        else:
            existing = ""
        # Merge: replace PaperForge views, preserve user views
        merged = merge_base_views(existing, views)
        # Substitute placeholders
        merged = substitute_config_placeholders(merged, paths)
        base_path.write_text(merged, encoding="utf-8")

    # Domain Bases
    seen_domains = set()
    for entry in config.get("domains", []):
        domain = str(entry.get("domain", "") or "").strip()
        if not domain or domain in seen_domains:
            continue
        seen_domains.add(domain)

        domain_views = build_base_views(domain)
        folder_filter = f"${{LIBRARY_RECORDS}}/{domain}"
        base_path = paths["bases"] / f"{slugify_filename(domain)}.base"
        refresh_base(base_path, folder_filter, domain_views)

    # Cross-domain Literature Hub Base
    hub_views = build_base_views("Literature Hub")
    hub_path = paths["bases"] / "Literature Hub.base"
    refresh_base(hub_path, "${LIBRARY_RECORDS}", hub_views)

    # Legacy PaperForge.base (all records)
    all_views = build_base_views("All Records")
    pf_base = paths["bases"] / "PaperForge.base"
    refresh_base(pf_base, "${LIBRARY_RECORDS}", all_views)
```

Key implementation notes:
- `merge_base_views()` is called to handle the incremental merge (Task 3)
- `substitute_config_placeholders()` replaces ${LIBRARY_RECORDS} etc. with actual vault-relative paths
- User views are identified as views WITHOUT PAPERFORGE_VIEW_PREFIX — they are never modified
- PaperForge views are ALWAYS replaced with fresh content on each refresh (filters/columns always current)
- The `folder_filter` parameter uses placeholder format and is substituted after merge
</action>
<verify>
python -c "
from pipeline.worker.scripts.literature_pipeline import ensure_base_views, merge_base_views, build_base_views, substitute_config_placeholders, PAPERFORGE_VIEW_PREFIX
from pathlib import Path
import tempfile
with tempfile.TemporaryDirectory() as tmpdir:
    vault = Path(tmpdir) / 'Vault'
    vault.mkdir()
    resources = vault / '03_Resources'
    resources.mkdir()
    control = resources / 'LiteratureControl'
    control.mkdir()
    literature = resources / 'Literature'
    literature.mkdir()
    library_records = control / 'library-records'
    library_records.mkdir()
    bases = vault / 'bases'
    bases.mkdir()
    (library_records / '骨科').mkdir()

    paths = {
        'vault': vault,
        'resources': resources,
        'control': control,
        'literature': literature,
        'library_records': library_records,
        'bases': bases,
    }
    config = {'domains': [{'domain': '骨科'}]}

    # First generation
    ensure_base_views(vault, paths, config)
    guke_base = bases / 'guke.base'
    assert guke_base.exists(), 'guke.base not created'
    content = guke_base.read_text(encoding='utf-8')

    # Check PAPERFORGE_VIEW_PREFIX markers are present
    assert content.count(PAPERFORGE_VIEW_PREFIX) == 8, f'Expected 8 markers, got {content.count(PAPERFORGE_VIEW_PREFIX)}'
    # Check placeholders substituted
    assert '\${LIBRARY_RECORDS}' not in content, 'Placeholder not substituted'
    assert '03_Resources/LiteratureControl/library-records/骨科' in content, 'Real path not substituted'
    # Check 8 views present
    assert content.count('type: table') == 8, f'Expected 8 views, got {content.count(\"type: table\")}'

    # Second refresh: add a user view and refresh
    user_content = content + '''
  - type: table
    name: \"My Custom Dashboard\"
    order:
      - title
      - year
'''
    guke_base.write_text(user_content, encoding='utf-8')

    # Refresh
    ensure_base_views(vault, paths, config)
    refreshed = guke_base.read_text(encoding='utf-8')

    # User view should be preserved
    assert 'My Custom Dashboard' in refreshed, 'User view was lost on refresh'
    # Still 9 views (8 standard + 1 user)
    assert refreshed.count('type: table') == 9, f'Expected 9 views after refresh, got {refreshed.count(\"type: table\")}'
    # Standard views still present and substituted
    assert '\${LIBRARY_RECORDS}' not in refreshed, 'Placeholder reintroduced on refresh'

    print('Incremental refresh test passed: user views preserved, standard views updated, paths substituted.')
"
</verify>
<done>ensure_base_views() uses incremental merge. User views preserved. Standard PaperForge views always fresh. Paths substituted correctly.</done>

</tasks>

<verification>
- `grep -n "def build_base_views" pipeline/worker/scripts/literature_pipeline.py` returns 1 match
- `grep -n "def merge_base_views" pipeline/worker/scripts/literature_pipeline.py` returns 1 match
- `grep -n "PAPERFORGE_VIEW_PREFIX" pipeline/worker/scripts/literature_pipeline.py` returns at least 1 match
- `grep -c "type: table" literature_pipeline.py` after changes >= 8 (for the hardcoded YAML)
- All inline verification Python commands above exit without error
</verification>

<success_criteria>
- Domain Bases contain exactly 8 views each with correct filter expressions
- ${LIBRARY_RECORDS} style placeholders are substituted to real vault-relative paths after merge
- PaperForge.base and Literature Hub.base are generated alongside domain Bases
- User-defined views (not in STANDARD_VIEW_NAMES) are PRESERVED across refreshes
- PaperForge standard views are always replaced with fresh content (current workflow definition)
</success_criteria>

<output>
After completion, create `.planning/phases/03-obsidian-bases-config-aware/03-01-SUMMARY.md`
</output>
