---
phase: 03-obsidian-bases-config-aware
plan: 02
type: execute
wave: 2
depends_on:
  - 03-01
files_modified:
  - paperforge/cli.py
  - tests/test_base_views.py
  - tests/test_base_preservation.py
 autonomous: true
requirements:
  - BASE-03
  - BASE-04

must_haves:
  truths:
    - "paperforge base-refresh does incremental merge: user views preserved, standard PaperForge views updated"
    - "paperforge base-refresh --force does full regeneration (bypasses merge, replaces everything)"
    - "Literature Hub.base is generated alongside domain Bases with the same 8-view structure"
    - "merge_base_views() preserves any view NOT in STANDARD_VIEW_NAMES across refreshes"
  artifacts:
    - path: "paperforge/cli.py"
      provides: "paperforge base-refresh subcommand with --force flag"
      contains: "base_refresh"
    - path: "tests/test_base_views.py"
      provides: "Unit tests for 8-view generation and placeholder substitution"
    - path: "tests/test_base_preservation.py"
      provides: "Unit tests for incremental merge and --force full regeneration"
---

<objective>
Add the `paperforge base-refresh` CLI command with `--force` flag and write pytest tests for the incremental merge Base system.

Purpose: `paperforge base-refresh` performs incremental merge (user views preserved, standard views updated). `--force` does full regeneration. Provide comprehensive tests for the 8-view generation and incremental merge behavior.
Output: CLI command + 2 test files.
</objective>

<read_first>
- paperforge/cli.py (existing CLI structure)
- pipeline/worker/scripts/literature_pipeline.py (ensure_base_views, build_base_views, substitute_config_placeholders)
- .planning/phases/03-obsidian-bases-config-aware/03-CONTEXT.md
</read_first>

<tasks>

<task type="auto">
<name>Task 1: Add paperforge base-refresh CLI command</name>
<read_first>
- paperforge/cli.py
</read_first>
<acceptance_criteria>
- grep "base-refresh" paperforge/cli.py returns a match
- paperforge base-refresh --help shows force flag and vault positional arg
- Exit code 0 on success
- ensure_base_views is called with force parameter from CLI args
</acceptance_criteria>
<action>
In `paperforge/cli.py`, add a `base-refresh` subcommand to the main parser.

Add after the existing subparser setup (look for `subparsers.add_parser` calls):

```python
# Base refresh subcommand
parser_base = subparsers.add_parser("base-refresh", help="Refresh Obsidian Base view files")
parser_base.add_argument(
    "--force", "-f", action="store_true",
    help="Force full regeneration (bypasses incremental merge, replaces all views including user views)"
)
```

Add the dispatch case in the `run()` function:
```python
elif command == "base-refresh":
    from pipeline.worker.scripts.literature_pipeline import ensure_base_views
    from paperforge.config import load_vault_config, paperforge_paths

    vault = Path(args.vault) if args.vault else Path.cwd()
    config = load_vault_config(vault)
    paths = paperforge_paths(vault, config)
    force = getattr(args, "force", False)

    logger.info(f"Refreshing Base views in {paths['bases']}")
    ensure_base_views(vault, paths, config, force=force)
    logger.info("Base refresh complete")
    return 0
```

Note: `ensure_base_views` already accepts `force: bool = False` from Plan 01 Task 4:
- `force=False` (default): incremental merge — PaperForge views updated, user views preserved
- `force=True`: full regeneration — replaces ALL views (including user views) with fresh PaperForge views
</action>
<verify>
python -c "
from paperforge.cli import build_parser
parser = build_parser()
import sys, io
old_stderr = sys.stderr
sys.stderr = io.StringIO()
try:
    parser.parse_args(['base-refresh', '--help'])
except SystemExit:
    pass
sys.stderr = old_stderr
"
</verify>
<done>paperforge base-refresh --help works and accepts --force flag.</done>

<task type="auto">
<name>Task 2: Write tests/test_base_views.py</name>
<read_first>
- tests/ directory structure (existing test files)
- pipeline/worker/scripts/literature_pipeline.py (build_base_views, write_base_file)
</read_first>
<acceptance_criteria>
- tests/test_base_views.py exists
- pytest tests/test_base_views.py exits 0
- Tests cover: build_base_views returns 8 views, each view has correct name/order/filter, write_base_file generates correct YAML, placeholder substitution works
</acceptance_criteria>
<action>
Create `tests/test_base_views.py`:

```python
"""Tests for the 8-view Base generation system (Phase 3, Plan 01)."""
import pytest
from pathlib import Path
from pipeline.worker.scripts.literature_pipeline import (
    build_base_views,
    write_base_file,
    substitute_config_placeholders,
)


class TestBuildBaseViews:
    def test_returns_exactly_8_views(self):
        views = build_base_views("骨科")
        assert len(views) == 8

    def test_all_view_names_present(self):
        views = build_base_views("骨科")
        names = [v["name"] for v in views]
        expected = [
            "控制面板", "推荐分析", "待 OCR", "OCR 完成",
            "待深度阅读", "深度阅读完成", "正式卡片", "全记录"
        ]
        assert names == expected

    def test_each_view_has_required_keys(self):
        views = build_base_views("骨科")
        for v in views:
            assert "name" in v
            assert "order" in v
            assert "filter" in v
            assert isinstance(v["order"], list)
            assert len(v["order"]) > 0

    def test_control_panel_has_no_filter(self):
        views = build_base_views("骨科")
        cp = next(v for v in views if v["name"] == "控制面板")
        assert cp["filter"] is None

    def test_pending_ocr_filter(self):
        views = build_base_views("骨科")
        pending = next(v for v in views if v["name"] == "待 OCR")
        assert 'do_ocr = true' in pending["filter"]
        assert 'ocr_status = "pending"' in pending["filter"]

    def test_ocr_done_filter(self):
        views = build_base_views("骨科")
        done = next(v for v in views if v["name"] == "OCR 完成")
        assert 'ocr_status = "done"' in done["filter"]

    def test_deep_reading_pending_filter(self):
        views = build_base_views("骨科")
        pending = next(v for v in views if v["name"] == "待深度阅读")
        assert "analyze = true" in pending["filter"]
        assert 'ocr_status = "done"' in pending["filter"]
        assert 'deep_reading_status = "pending"' in pending["filter"]


class TestWriteBaseFile:
    def test_generates_yaml_with_all_8_views(self, tmp_path):
        views = build_base_views("骨科")
        out = tmp_path / "test.base"
        write_base_file(out, "${LIBRARY_RECORDS}/骨科", views)
        content = out.read_text(encoding="utf-8")
        assert content.count("type: table") == 8

    def test_properties_section_present(self, tmp_path):
        views = build_base_views("骨科")
        out = tmp_path / "test.base"
        write_base_file(out, "${LIBRARY_RECORDS}/骨科", views)
        content = out.read_text(encoding="utf-8")
        assert "zotero_key:" in content
        assert "title:" in content
        assert "year:" in content
        assert "has_pdf:" in content
        assert "ocr_status:" in content
        assert "deep_reading_status:" in content

    def test_folder_filter_placeholder_preserved(self, tmp_path):
        views = build_base_views("骨科")
        out = tmp_path / "test.base"
        write_base_file(out, "${LIBRARY_RECORDS}/骨科", views)
        content = out.read_text(encoding="utf-8")
        assert 'file.inFolder("${LIBRARY_RECORDS}/骨科")' in content


class TestSubstituteConfigPlaceholders:
    def test_substitutes_library_records(self, tmp_path):
        content = "${LIBRARY_RECORDS}/骨科"
        vault = tmp_path / "Vault"
        vault.mkdir()
        lib_rec = vault / "03_Resources" / "LiteratureControl" / "library-records"
        lib_rec.mkdir(parents=True)
        result = substitute_config_placeholders(
            content,
            {"library_records": lib_rec, "vault": vault}
        )
        assert "${LIBRARY_RECORDS}" not in result
        assert "03_Resources/LiteratureControl/library-records" in result

    def test_substitutes_multiple_placeholders(self, tmp_path):
        content = "${LIBRARY_RECORDS} and ${LITERATURE}"
        vault = tmp_path / "Vault"
        vault.mkdir()
        lib_rec = vault / "LR"
        lit = vault / "LIT"
        lib_rec.mkdir()
        lit.mkdir()
        result = substitute_config_placeholders(
            content,
            {"library_records": lib_rec, "literature": lit, "vault": vault}
        )
        assert "${LIBRARY_RECORDS}" not in result
        assert "${LITERATURE}" not in result

    def test_unknown_placeholder_unchanged(self):
        content = "${UNKNOWN_PLACEHOLDER}/骨科"
        result = substitute_config_placeholders(content, {})
        assert "${UNKNOWN_PLACEHOLDER}" in result

    def test_backslash_conversion(self, tmp_path):
        content = "${LIBRARY_RECORDS}"
        vault = tmp_path / "Vault"
        vault.mkdir()
        lib_rec = vault / "LR"
        lib_rec.mkdir()
        result = substitute_config_placeholders(
            content,
            {"library_records": lib_rec, "vault": vault}
        )
        assert "\\" not in result  # Should use forward slash
```
</action>
<verify>
pytest tests/test_base_views.py -v
</verify>
<done>All tests in test_base_views.py pass.</done>

<task type="auto">
<name>Task 3: Write tests/test_base_preservation.py — incremental merge tests</name>
<read_first>
- tests/ directory structure
- pipeline/worker/scripts/literature_pipeline.py (ensure_base_views, merge_base_views, PAPERFORGE_VIEW_PREFIX, STANDARD_VIEW_NAMES)
</read_first>
<acceptance_criteria>
- tests/test_base_preservation.py exists
- pytest tests/test_base_preservation.py exits 0
- Tests cover: user views preserved on refresh, standard PaperForge views updated on refresh, force=True does full regeneration, Literature Hub Base created
</acceptance_criteria>
<action>
Create `tests/test_base_preservation.py` with tests for incremental merge behavior:

```python
"""Tests for incremental merge and user-view preservation (Phase 3, Plan 02)."""
import pytest
from pathlib import Path
from pipeline.worker.scripts.literature_pipeline import (
    ensure_base_views,
    merge_base_views,
    build_base_views,
    substitute_config_placeholders,
    PAPERFORGE_VIEW_PREFIX,
    STANDARD_VIEW_NAMES,
)


class TestIncrementalMerge:
    def setup_method(self, tmp_path):
        self.vault = tmp_path / "Vault"
        self.vault.mkdir()
        resources = self.vault / "03_Resources"
        resources.mkdir()
        control = resources / "LiteratureControl"
        control.mkdir()
        literature = resources / "Literature"
        literature.mkdir()
        lr = control / "library-records"
        lr.mkdir()
        (lr / "骨科").mkdir()
        self.bases = self.vault / "bases"
        self.bases.mkdir()
        self.paths = {
            "vault": self.vault,
            "resources": resources,
            "control": control,
            "literature": literature,
            "library_records": lr,
            "bases": self.bases,
        }
        self.config = {"domains": [{"domain": "骨科"}]}

    def test_user_custom_view_is_preserved_after_refresh(self):
        """User adds a custom view, then refresh — custom view must be preserved."""
        domain_base = self.bases / "guke.base"

        # First generate a valid base
        ensure_base_views(self.vault, self.paths, self.config, force=False)
        content = domain_base.read_text(encoding="utf-8")
        assert content.count("type: table") == 8

        # User adds a custom view (no PAPERFORGE_VIEW_PREFIX)
        user_custom = content + '''
  - type: table
    name: "My Custom Dashboard"
    order:
      - title
      - year
'''
        domain_base.write_text(user_custom, encoding="utf-8")

        # Refresh (force=False, incremental merge)
        ensure_base_views(self.vault, self.paths, self.config, force=False)
        refreshed = domain_base.read_text(encoding="utf-8")

        # User view is preserved
        assert "My Custom Dashboard" in refreshed, "User custom view was lost on incremental refresh"
        # Still 9 views (8 standard + 1 user)
        assert refreshed.count("type: table") == 9
        # Standard PaperForge views still present
        assert "控制面板" in refreshed
        assert "OCR 完成" in refreshed

    def test_standard_paperforge_views_are_updated_on_refresh(self):
        """Standard views are replaced with fresh content on each refresh."""
        domain_base = self.bases / "guke.base"
        ensure_base_views(self.vault, self.paths, self.config, force=False)

        # Get initial content
        content1 = domain_base.read_text(encoding="utf-8")
        # Confirm PAPERFORGE_VIEW_PREFIX markers present
        assert content1.count(PAPERFORGE_VIEW_PREFIX) == 8

        # Refresh
        ensure_base_views(self.vault, self.paths, self.config, force=False)
        content2 = domain_base.read_text(encoding="utf-8")

        # All 8 PaperForge views still present
        assert content2.count("type: table") == 8
        # Paths substituted
        assert "${LIBRARY_RECORDS}" not in content2

    def test_force_flag_does_full_regeneration(self):
        """--force=True bypasses merge and does full regeneration (all views replaced)."""
        domain_base = self.bases / "guke.base"

        # First generate
        ensure_base_views(self.vault, self.paths, self.config, force=False)
        content1 = domain_base.read_text(encoding="utf-8")

        # User adds a custom view
        user_custom = content1 + '''
  - type: table
    name: "My Custom View"
    order:
      - title
'''
        domain_base.write_text(user_custom, encoding="utf-8")

        # Refresh with force=True
        ensure_base_views(self.vault, self.paths, self.config, force=True)
        refreshed = domain_base.read_text(encoding="utf-8")

        # Custom view is LOST because force=True does full regeneration
        assert "My Custom View" not in refreshed, "force=True should have replaced all views"
        # Still 8 views (standard only, no user view)
        assert refreshed.count("type: table") == 8
        # PAPERFORGE_VIEW_PREFIX markers present
        assert refreshed.count(PAPERFORGE_VIEW_PREFIX) == 8

    def test_user_modified_filter_on_standard_view_is_overwritten(self):
        """User changes filter on a standard PaperForge view — on refresh it reverts to template."""
        domain_base = self.bases / "guke.base"
        ensure_base_views(self.vault, self.paths, self.config, force=False)
        content1 = domain_base.read_text(encoding="utf-8")

        # User modifies the OCR Done view's filter to something custom
        modified = content1.replace(
            'filter: \'ocr_status = "done"\'',
            'filter: \'ocr_status = "done" AND has_pdf = true\''
        )
        domain_base.write_text(modified, encoding="utf-8")

        # Refresh (incremental merge)
        ensure_base_views(self.vault, self.paths, self.config, force=False)
        refreshed = domain_base.read_text(encoding="utf-8")

        # User modification is OVERWRITTEN (reverted to standard filter)
        assert 'filter: \'ocr_status = "done"\'' in refreshed, \
            "Standard PaperForge view filter should be reverted on refresh"
        assert "has_pdf = true" not in refreshed, \
            "User filter modification should have been overwritten by incremental merge"

    def test_new_domain_base_is_created_on_first_run(self, tmp_path):
        """First run creates domain base if none exists."""
        # Setup with a fresh empty bases dir
        vault = tmp_path / "V2"
        vault.mkdir()
        resources = vault / "03_Resources"
        resources.mkdir()
        control = resources / "LiteratureControl"
        control.mkdir()
        literature = resources / "Literature"
        literature.mkdir()
        lr = control / "library-records"
        lr.mkdir()
        bases = vault / "bases"
        bases.mkdir()
        (lr / "骨科").mkdir()
        paths = {
            "vault": vault, "resources": resources, "control": control,
            "literature": literature, "library_records": lr, "bases": bases,
        }
        config = {"domains": [{"domain": "骨科"}]}

        domain_base = bases / "guke.base"
        assert not domain_base.exists()

        ensure_base_views(vault, paths, config, force=False)

        assert domain_base.exists()
        content = domain_base.read_text(encoding="utf-8")
        assert content.count("type: table") == 8
        assert PAPERFORGE_VIEW_PREFIX in content


class TestLiteratureHubBase:
    def test_literature_hub_base_created(self, tmp_path):
        vault = tmp_path / "Vault"
        vault.mkdir()
        resources = vault / "03_Resources"
        resources.mkdir()
        control = resources / "LiteratureControl"
        control.mkdir()
        literature = resources / "Literature"
        literature.mkdir()
        library_records = control / "library-records"
        library_records.mkdir()
        bases = vault / "bases"
        bases.mkdir()

        paths = {
            "vault": vault,
            "resources": resources,
            "control": control,
            "literature": literature,
            "library_records": library_records,
            "bases": bases,
        }
        config = {"domains": [{"domain": "骨科"}, {"domain": "运动医学"}]}

        ensure_base_views(vault, paths, config, force=False)

        hub_base = bases / "Literature Hub.base"
        assert hub_base.exists(), "Literature Hub.base not created"
        content = hub_base.read_text(encoding="utf-8")
        assert content.count("type: table") == 8, f"Expected 8 views, got {content.count('type: table')}"
        assert PAPERFORGE_VIEW_PREFIX in content
        assert "${LIBRARY_RECORDS}" not in content, "Placeholder should be substituted"
```

Note: `ensure_base_views` in Plan 01 Task 4 needs a `force: bool = False` parameter added. When `force=True`, the `refresh_base` inner function should use `_build_base_yaml` directly instead of `merge_base_views`. Update the function signature and `refresh_base` inner function accordingly.
</action>
<verify>
pytest tests/test_base_preservation.py -v
</verify>
<done>All tests in test_base_preservation.py pass — incremental merge preserves user views, standard views updated, force=True full regeneration works.</done>


class TestLiteratureHubBase:
    def test_literature_hub_base_created(self, tmp_path):
        vault = tmp_path / "Vault"
        vault.mkdir()
        resources = vault / "03_Resources"
        resources.mkdir()
        control = resources / "LiteratureControl"
        control.mkdir()
        literature = resources / "Literature"
        literature.mkdir()
        library_records = control / "library-records"
        library_records.mkdir()
        bases = vault / "bases"
        bases.mkdir()

        paths = {
            "vault": vault,
            "resources": resources,
            "control": control,
            "literature": literature,
            "library_records": library_records,
            "bases": bases,
        }
        config = {"domains": [{"domain": "骨科"}, {"domain": "运动医学"}]}

        ensure_base_views(vault, paths, config, force=False)

        hub_base = bases / "Literature Hub.base"
        assert hub_base.exists(), "Literature Hub.base not created"
        content = hub_base.read_text(encoding="utf-8")
        assert content.count("type: table") == 8, f"Expected 8 views, got {content.count('type: table')}"
        # Literature Hub should filter on root library_records (all domains)
        assert "file.inFolder(" in content
        # Should have substituted path
        assert "${LIBRARY_RECORDS}" not in content or "03_Resources" in content
```

Note: `ensure_base_views` must be updated to accept `force` parameter:
- `force=False` (default): incremental merge — calls `merge_base_views` to update only PaperForge views
- `force=True`: full regeneration — calls `_build_base_yaml` directly (no merge), replacing ALL views

Add to `ensure_base_views` signature and modify the `refresh_base` inner function:

```python
def ensure_base_views(vault: Path, paths: dict[str, Path], config: dict, force: bool = False) -> None:
    # ...
    def refresh_base(base_path: Path, folder_filter: str, views: list[dict]) -> None:
        if base_path.exists() and not force:
            existing = base_path.read_text(encoding="utf-8")
            merged = merge_base_views(existing, views)
        else:
            # force=True or new file: full regeneration
            merged = _build_base_yaml(folder_filter, views)
        merged = substitute_config_placeholders(merged, paths)
        base_path.write_text(merged, encoding="utf-8")
```
</action>
<verify>
pytest tests/test_base_preservation.py -v
</verify>
<done>All tests in test_base_preservation.py pass.</done>

</tasks>

<verification>
- `pytest tests/test_base_views.py -v` exits 0
- `pytest tests/test_base_preservation.py -v` exits 0
- `python -c "from paperforge.cli import build_parser; p = build_parser(); p.parse_args(['base-refresh', '--help'])"` exits 0
</verification>

<success_criteria>
- paperforge base-refresh --help works
- ensure_base_views accepts force=False (incremental merge) and force=True (full regeneration)
- All pytest tests pass (test_base_views.py, test_base_preservation.py)
- User custom views are preserved after refresh (incremental merge)
</success_criteria>

<output>
After completion, create `.planning/phases/03-obsidian-bases-config-aware/03-02-SUMMARY.md`
</output>
