---
phase: 58-service-extraction
plan: 03
type: execute
wave: 2
depends_on: ["058-01"]
files_modified:
  - paperforge/adapters/obsidian_frontmatter.py
  - tests/unit/adapters/test_obsidian_frontmatter.py
  - paperforge/worker/sync.py
autonomous: true
requirements: [SYNC-03, SYNC-05]

must_haves:
  truths:
    - "paperforge/adapters/obsidian_frontmatter.py can be imported independently of sync.py"
    - "Frontmatter read functions use YAML parser (PyYAML) for bulk field reading, fall back to regex for single-field reads"
    - "update_frontmatter_field() updates an existing field value in frontmatter or appends if missing"
    - "load_control_actions() scans literature notes for do_ocr/analyze flags without requiring sync index"
    - "has_deep_reading_content() validates 3-anchor completeness (Clarity, Figure 导读, 遗留问题)"
    - "extract_preserved_deep_reading() preserves the deep reading section during note regeneration"
    - "All deep-reading related tests pass against extracted module"
  artifacts:
    - path: paperforge/adapters/obsidian_frontmatter.py
      provides: "Frontmatter read/write/update operations extracted from sync.py"
      exports: ["_read_frontmatter_bool_from_text", "_read_frontmatter_optional_bool_from_text", "_legacy_control_flags", "_add_missing_frontmatter_fields", "update_frontmatter_field", "load_control_actions", "extract_preserved_deep_reading", "has_deep_reading_content", "_extract_section"]
      min_lines: 140
    - path: tests/unit/adapters/test_obsidian_frontmatter.py
      provides: "Unit tests for frontmatter operations covering all edge cases"
      min_lines: 150
  key_links:
    - from: paperforge/adapters/obsidian_frontmatter.py
      to: paperforge/worker/_utils.py
      via: "import yaml_quote"
      pattern: "from paperforge.worker._utils import yaml_quote"
    - from: paperforge/worker/sync.py
      to: paperforge/adapters/obsidian_frontmatter.py
      via: "import alias for backward compat"
      pattern: "from paperforge.adapters.obsidian_frontmatter import"
    - from: paperforge/worker/deep_reading.py
      to: paperforge/adapters/obsidian_frontmatter.py
      via: "import has_deep_reading_content"
---
<objective>
Extract all frontmatter read/write/update operations from sync.py into a focused `adapters/obsidian_frontmatter.py` module, replacing regex-based frontmatter parsing with YAML parser (`yaml.safe_load()`) where practical, while preserving regex for surgical single-field reads.

**Purpose:** Frontmatter operations were scattered across sync.py with regex-only parsing. Consolidating into one module makes them independently testable and allows replacing fragile regex with proper YAML parsing for bulk operations. The deep reading validation functions (has_deep_reading_content) are also included since they operate on frontmatter text.

**Output:** adapter module + comprehensive test file. sync.py retains backward-compatible import aliases.

**Extracted functions:**
- `read_frontmatter_dict(text)` — NEW: parse full frontmatter block with `yaml.safe_load()`
- `_read_frontmatter_bool_from_text(text, key, default)` — regex-based field read
- `_read_frontmatter_optional_bool_from_text(text, key)` — regex-based optional bool
- `_legacy_control_flags(paths, zotero_key)` — scans library-records for control flags
- `_add_missing_frontmatter_fields(existing_content, new_fields)` — surgically appends fields
- `update_frontmatter_field(content, key, value)` — update or add a frontmatter field
- `load_control_actions(paths)` — scans all literature notes for do_ocr/analyze
- `extract_preserved_deep_reading(text)` — extracts `## 🔍 精读` section
- `has_deep_reading_content(text)` — 3-anchor completeness check
- `_extract_section(body, section_header)` — section extraction utility

**Design decision:** `read_frontmatter_dict()` uses `yaml.safe_load()` for bulk reads but individual field read functions (`_read_frontmatter_bool*`) keep regex for speed and tolerance of malformed frontmatter (per D-09: surgical parsing for partial data).
</objective>

<execution_context>
@file:///C:/Users/Lin/.opencode/get-shit-done/workflows/execute-plan.md
@file:///C:/Users/Lin/.opencode/get-shit-done/templates/summary.md
</execution_context>

<context>
@file:///D:/L/Med/Research/99_System/LiteraturePipeline/github-release/paperforge/worker/sync.py
@file:///D:/L/Med/Research/99_System/LiteraturePipeline/github-release/paperforge/worker/_utils.py
@file:///D:/L/Med/Research/99_System/LiteraturePipeline/github-release/paperforge/worker/deep_reading.py
@file:///D:/L/Med/Research/99_System/LiteraturePipeline/github-release/paperforge/worker/repair.py

<interfaces>
From paperforge/worker/_utils.py (YAML helpers used by writing functions only — NOT by reading functions):
```python
def yaml_quote(value: str) -> str: ...
def yaml_block(value: str) -> list[str]: ...
```

Note: `read_frontmatter_dict()` will use `import yaml` (PyYAML) directly. PyYAML is already a project dependency (verified: `python -c "import yaml; print(yaml.__version__)"`).
</interfaces>
</context>

<tasks>

<task type="auto" tdd="false">
  <name>Task 1: Create adapters/obsidian_frontmatter.py</name>
  <files>
    paperforge/adapters/obsidian_frontmatter.py
  </files>
  <action>
    Step 1.1: Create paperforge/adapters/obsidian_frontmatter.py.

    **NEW function — add before extracted functions:**
    ```python
    from __future__ import annotations

    import logging
    import re
    from pathlib import Path

    import yaml

    from paperforge.worker._utils import yaml_quote

    logger = logging.getLogger(__name__)


    def read_frontmatter_dict(text: str) -> dict:
        """Parse YAML frontmatter block from markdown text into a dict.

        Extracts text between leading ``---`` and closing ``---``, then
        parses with ``yaml.safe_load()``. Returns empty dict if no valid
        frontmatter is found.

        This is the *preferred* function for bulk field access. Use
        ``_read_frontmatter_bool_from_text`` only when you need a surgical
        single-field read without pulling in the YAML parser.
        """
        if not text or not text.startswith("---"):
            return {}
        parts = text.split("---", 2)
        if len(parts) < 3:
            return {}
        frontmatter_text = parts[1]
        try:
            data = yaml.safe_load(frontmatter_text)
            if isinstance(data, dict):
                return data
            return {}
        except yaml.YAMLError:
            return {}
    ```

    **Extracted functions from sync.py (verbatim copy — regex-based reads preserved for backward compat and speed):**

    **Function 2: `_read_frontmatter_bool_from_text(text: str, key: str, default: bool = False) -> bool`**
    - Lines 36-40 from sync.py verbatim

    **Function 3: `_read_frontmatter_optional_bool_from_text(text: str, key: str) -> Optional[bool]`**
    - Lines 43-47 from sync.py verbatim
    - Import `Optional` from typing

    **Function 4: `_legacy_control_flags(paths: dict[str, Path], zotero_key: str) -> dict[str, Optional[bool]]`**
    - Lines 50-63 from sync.py verbatim
    - Depends on `_read_frontmatter_optional_bool_from_text`

    **Function 5: `_add_missing_frontmatter_fields(existing_content: str, new_fields: dict[str, str]) -> str`**
    - Lines 679-696 from sync.py verbatim
    - Depends on `yaml_quote` from _utils.py

    **Function 6: `update_frontmatter_field(content: str, key: str, value: str) -> str`**
    - Lines 699-708 from sync.py verbatim
    - Calls `_add_missing_frontmatter_fields`

    **Function 7: `load_control_actions(paths: dict[str, Path]) -> dict[str, dict]`**
    - Lines 711-736 from sync.py verbatim
    - Optional refactor: replace nested regex reads with `read_frontmatter_dict()` call
    - **Preferred implementation**: Replace the inner loop with:
      ```python
      def load_control_actions(paths: dict[str, Path]) -> dict[str, dict]:
          actions = {}
          lit_root = paths.get("literature")
          if not lit_root or not lit_root.exists():
              return actions
          for note_file in lit_root.rglob("*.md"):
              if note_file.name in ("fulltext.md", "deep-reading.md", "discussion.md"):
                  continue
              try:
                  text = note_file.read_text(encoding="utf-8")
              except Exception:
                  continue
              fm = read_frontmatter_dict(text)
              if not fm:
                  continue
              zotero_key = str(fm.get("zotero_key", "") or "").strip()
              if not zotero_key:
                  continue
              actions[zotero_key] = {
                  "analyze": bool(fm.get("analyze", False)),
                  "do_ocr": bool(fm.get("do_ocr", False)),
              }
          return actions
      ```
      This replaces 20 lines of regex with a single `read_frontmatter_dict()` call. Verify it produces identical output to the regex version by running the existing OCR tests.

    **Function 8: `extract_preserved_deep_reading(text: str) -> str`**
    - Lines 595-611 from sync.py verbatim (DEEP_READING_HEADER constant stays here)

    **Function 9: `has_deep_reading_content(text: str) -> bool`**
    - Lines 614-660 from sync.py verbatim
    - Calls both `extract_preserved_deep_reading` and `_extract_section`

    **Function 10: `_extract_section(body: str, section_header: str) -> str | None`**
    - Lines 663-674 from sync.py verbatim

    Step 1.2: In paperforge/worker/sync.py, add backward-compatible import aliases (after existing adapter imports):
    ```python
    # Adapter re-exports (backward compat) — Phase 58 Service Extraction
    from paperforge.adapters.obsidian_frontmatter import (
        _read_frontmatter_bool_from_text,
        _read_frontmatter_optional_bool_from_text,
        _legacy_control_flags,
        _add_missing_frontmatter_fields,
        update_frontmatter_field,
        load_control_actions,
        extract_preserved_deep_reading,
        has_deep_reading_content,
        _extract_section,
    )
    ```

    Step 1.3: Verify importability:
    ```
    python -c "from paperforge.adapters.obsidian_frontmatter import read_frontmatter_dict, update_frontmatter_field, load_control_actions, has_deep_reading_content, extract_preserved_deep_reading; print('Frontmatter adapter import OK')"
    ```
  </action>
  <verify>
    <automated>
      python -c "from paperforge.adapters.obsidian_frontmatter import read_frontmatter_dict, _read_frontmatter_bool_from_text, update_frontmatter_field, load_control_actions, has_deep_reading_content, extract_preserved_deep_reading, _extract_section; print('Frontmatter adapter import OK')"
    </automated>
  </verify>
  <done>
    adapters/obsidian_frontmatter.py exists with all extracted functions + new read_frontmatter_dict(). Backward-compat aliases added to sync.py.
  </done>
</task>

<task type="auto" tdd="true">
  <name>Task 2: Create comprehensive frontmatter unit tests</name>
  <files>
    tests/unit/adapters/test_obsidian_frontmatter.py
  </files>
  <behavior>
    Test read_frontmatter_dict with:
    - Valid frontmatter -> correct dict
    - No frontmatter -> empty dict
    - Malformed YAML -> empty dict (no crash)
    - Empty string -> empty dict

    Test _read_frontmatter_bool_from_text with:
    - true value -> True
    - false value -> False
    - quoted true -> True
    - missing key -> default
    - different default

    Test update_frontmatter_field with:
    - Existing field -> value updated
    - Missing field -> field appended
    - No frontmatter -> content unchanged

    Test load_control_actions (with mocked Path) with:
    - Note with analyze=true -> captured
    - Note with do_ocr=true -> captured
    - Note without frontmatter -> skipped
    - Non-literature files excluded (fulltext.md, deep-reading.md, discussion.md)

    Test extract_preserved_deep_reading with:
    - Deep reading section present -> extracted
    - No deep reading -> empty string
    - Deep reading with multiple sections -> full section preserved

    Test has_deep_reading_content with:
    - All 3 anchors complete -> True
    - Missing Clarity -> False
    - Missing Figure -> False
    - Missing 遗留问题 -> False
    - Empty text -> False

    Test _legacy_control_flags (mocked) with:
    - Record found with do_ocr=true -> correct flags
    - Record found with analyze=true -> correct flags
    - No record found -> all None
  </behavior>
  <action>
    Step 2.1 (RED): Write test file at tests/unit/adapters/test_obsidian_frontmatter.py:

    ```python
    from __future__ import annotations

    from pathlib import Path
    from unittest.mock import MagicMock, patch

    from paperforge.adapters.obsidian_frontmatter import (
        _add_missing_frontmatter_fields,
        _legacy_control_flags,
        _read_frontmatter_bool_from_text,
        _read_frontmatter_optional_bool_from_text,
        extract_preserved_deep_reading,
        has_deep_reading_content,
        load_control_actions,
        read_frontmatter_dict,
        update_frontmatter_field,
    )


    SAMPLE_FRONTMATTER = """---
    title: Test Paper
    zotero_key: ABC123
    analyze: true
    do_ocr: false
    ocr_status: done
    deep_reading_status: pending
    ---
    """

    DEEP_READING_SECTION = """
    ## \U0001f50d 精读
    - **Clarity**（清晰度）：Good clarity in methods
    - **Figure 导读**：Figure 1 shows X
    - **遗留问题**：Need more data on Y
    """

    NO_CLARITY_SECTION = """
    ## \U0001f50d 精读
    - **Clarity**（清晰度）：
    - **Figure 导读**：Figure 1 shows X
    - **遗留问题**：Need more data on Y
    """

    NO_FIGURE_SECTION = """
    ## \U0001f50d 精读
    - **Clarity**（清晰度）：Good clarity
    - **Figure 导读**：
    - **遗留问题**：Need more data
    """

    NO_ISSUE_SECTION = """
    ## \U0001f50d 精读
    - **Clarity**（清晰度）：Good clarity
    - **Figure 导读**：Figure 1 shows X
    - **遗留问题**：
    """


    class TestReadFrontmatterDict:
        def test_valid_frontmatter(self) -> None:
            result = read_frontmatter_dict(SAMPLE_FRONTMATTER)
            assert result.get("title") == "Test Paper"
            assert result.get("zotero_key") == "ABC123"

        def test_no_frontmatter(self) -> None:
            assert read_frontmatter_dict("Just text") == {}

        def test_malformed_yaml(self) -> None:
            assert read_frontmatter_dict("---\n:invalid yaml::\n---") == {}

        def test_empty_string(self) -> None:
            assert read_frontmatter_dict("") == {}

        def test_incomplete_frontmatter(self) -> None:
            assert read_frontmatter_dict("---\ntitle: only") == {}


    class TestReadFrontmatterBool:
        def test_true_value(self) -> None:
            assert _read_frontmatter_bool_from_text("analyze: true\n", "analyze") is True

        def test_false_value(self) -> None:
            assert _read_frontmatter_bool_from_text("analyze: false\n", "analyze") is False

        def test_quoted_true(self) -> None:
            assert _read_frontmatter_bool_from_text('analyze: "true"\n', "analyze") is True

        def test_missing_key_returns_default(self) -> None:
            assert _read_frontmatter_bool_from_text("other: true\n", "analyze") is False

        def test_custom_default(self) -> None:
            assert _read_frontmatter_bool_from_text("other: true\n", "analyze", default=True) is True

        def test_optional_bool_present_true(self) -> None:
            assert _read_frontmatter_optional_bool_from_text("analyze: true\n", "analyze") is True

        def test_optional_bool_present_false(self) -> None:
            assert _read_frontmatter_optional_bool_from_text("analyze: false\n", "analyze") is False

        def test_optional_bool_missing(self) -> None:
            assert _read_frontmatter_optional_bool_from_text("other: true\n", "analyze") is None


    class TestUpdateFrontmatterField:
        def test_update_existing_field(self) -> None:
            content = "---\nkey: old\n---\nbody"
            result = update_frontmatter_field(content, "key", "new")
            assert "key: \"new\"" in result or "key: new" in result

        def test_add_missing_field(self) -> None:
            content = "---\nother: val\n---\nbody"
            result = update_frontmatter_field(content, "new_field", "value")
            assert "new_field:" in result

        def test_no_frontmatter_no_change(self) -> None:
            content = "body only"
            result = update_frontmatter_field(content, "key", "val")
            assert result == "body only"

        def test_add_missing_frontmatter_fields(self) -> None:
            content = "---\nexisting: val\n---\nbody"
            result = _add_missing_frontmatter_fields(content, {"new1": "v1", "new2": "v2"})
            assert "new1:" in result
            assert "new2:" in result

        def test_skips_existing_fields(self) -> None:
            content = "---\nexisting: old\n---\nbody"
            result = _add_missing_frontmatter_fields(content, {"existing": "new"})
            # Should NOT add a second "existing:" line
            assert content == result


    class TestLoadControlActions:
        def _make_mock_paths(self, root: Path) -> dict:
            return {"literature": root / "Literature"}

        def test_captures_analyze_true(self, tmp_path: Path) -> None:
            lit = tmp_path / "Literature" / "Ortho"
            lit.mkdir(parents=True)
            note = lit / "ABC123 - Test.md"
            note.write_text(SAMPLE_FRONTMATTER + "\ncontent", encoding="utf-8")
            actions = load_control_actions(self._make_mock_paths(tmp_path))
            assert "ABC123" in actions
            assert actions["ABC123"]["analyze"] is True
            assert actions["ABC123"]["do_ocr"] is False

        def test_skips_system_files(self, tmp_path: Path) -> None:
            lit = tmp_path / "Literature" / "Ortho"
            lit.mkdir(parents=True)
            for name in ("fulltext.md", "deep-reading.md", "discussion.md"):
                (lit / name).write_text(SAMPLE_FRONTMATTER, encoding="utf-8")
            actions = load_control_actions(self._make_mock_paths(tmp_path))
            assert len(actions) == 0


    class TestDeepReadingExtraction:
        def test_extracts_deep_reading_section(self) -> None:
            text = "---\n---\n\ncontent\n" + DEEP_READING_SECTION
            result = extract_preserved_deep_reading(text)
            assert "精读" in result

        def test_no_deep_reading_returns_empty(self) -> None:
            assert extract_preserved_deep_reading("no section here") == ""

        def test_empty_text(self) -> None:
            assert extract_preserved_deep_reading("") == ""

        def test_has_all_anchors(self) -> None:
            text = SAMPLE_FRONTMATTER + DEEP_READING_SECTION
            assert has_deep_reading_content(text) is True

        def test_missing_clarity(self) -> None:
            text = SAMPLE_FRONTMATTER + NO_CLARITY_SECTION
            assert has_deep_reading_content(text) is False

        def test_missing_figure(self) -> None:
            text = SAMPLE_FRONTMATTER + NO_FIGURE_SECTION
            assert has_deep_reading_content(text) is False

        def test_missing_issues(self) -> None:
            text = SAMPLE_FRONTMATTER + NO_ISSUE_SECTION
            assert has_deep_reading_content(text) is False

        def test_no_content_false(self) -> None:
            assert has_deep_reading_content("") is False
    ```

    Step 2.2 (GREEN): Run tests:
    ```
    python -m pytest tests/unit/adapters/test_obsidian_frontmatter.py -v --tb=short
    ```

    Step 2.3: Fix any failures. Common issues:
    - YAML parser might not preserve boolean values as expected (YAML `true` is `True` in Python)
    - Unicode emoji in deep reading section header might need careful handling
    - `has_deep_reading_content` uses the exact emoji `\U0001f50d` — ensure test matches
  </action>
  <verify>
    <automated>python -m pytest tests/unit/adapters/test_obsidian_frontmatter.py -v --tb=short 2>&1</automated>
  </verify>
  <done>
    15+ test cases pass covering frontmatter dict parsing, boolean reads, field updates, control action scanning, and deep reading validation.
  </done>
</task>

<task type="auto">
  <name>Task 3: Verify backward compatibility and downstream imports</name>
  <files>
    paperforge/worker/sync.py
  </files>
  <action>
    Step 3.1: Verify all modules that import frontmatter/deep-reading functions from sync.py still work:

    Critical downstream consumers:
    - `paperforge.worker.repair` imports `update_frontmatter_field` from sync.py
    - `paperforge.worker.ocr` imports `load_control_actions` from sync.py
    - `paperforge.worker.deep_reading` imports `has_deep_reading_content` from sync.py
    - `paperforge.worker.asset_index` imports `has_deep_reading_content` from sync.py (lazy import)

    ```
    python -c "
    from paperforge.adapters.obsidian_frontmatter import (
        _read_frontmatter_bool_from_text, update_frontmatter_field, 
        load_control_actions, has_deep_reading_content,
        extract_preserved_deep_reading
    )
    from paperforge.worker.sync import (
        _read_frontmatter_bool_from_text, update_frontmatter_field,
        load_control_actions, has_deep_reading_content
    )
    print('All import paths OK')
    "
    ```

    Step 3.2: Run deep-reading tests (most sensitive to has_deep_reading_content extraction):
    ```
    python -m pytest tests/test_ld_deep_skel.py tests/test_ld_deep_config.py -v --tb=short 2>&1 | tail -30
    ```

    Step 3.3: Run repair tests (use update_frontmatter_field from sync.py):
    ```
    python -m pytest tests/test_repair.py -v --tb=short 2>&1 | tail -30
    ```

    Step 3.4: Full worker import check:
    ```
    python -c "
    import paperforge.worker.sync
    import paperforge.worker.ocr
    import paperforge.worker.repair
    import paperforge.worker.deep_reading
    import paperforge.worker.asset_index
    print('All downstream imports OK')
    "
    ```
  </action>
  <verify>
    <automated>
      python -m pytest tests/test_repair.py tests/test_ld_deep_skel.py -v --tb=short 2>&1 | tail -30
    </automated>
  </verify>
  <done>
    All downstream modules import cleanly. deep_reading.py, repair.py, ocr.py, and asset_index.py all work with the backward-compat aliases.
  </done>
</task>

</tasks>

<verification>
1. `python -m pytest tests/unit/adapters/test_obsidian_frontmatter.py -v` — all frontmatter tests pass
2. `python -m pytest tests/test_repair.py tests/test_ld_deep_skel.py -v --tb=short` — downstream tests pass
3. `python -c "from paperforge.adapters.obsidian_frontmatter import read_frontmatter_dict, update_frontmatter_field, load_control_actions, has_deep_reading_content; print('adapter import OK')"`
4. `python -c "from paperforge.worker.sync import update_frontmatter_field, load_control_actions, has_deep_reading_content; print('backward compat OK')"`
5. `python -c "import paperforge.worker.deep_reading; import paperforge.worker.repair; import paperforge.worker.ocr; import paperforge.worker.asset_index; print('downstream OK')"`
</verification>

<success_criteria>
1. paperforge/adapters/obsidian_frontmatter.py exists with all 10 functions importable
2. New `read_frontmatter_dict()` function using `yaml.safe_load()` available for bulk frontmatter reads
3. paperforge/worker/sync.py has backward-compatible aliases for all extracted frontmatter functions
4. tests/unit/adapters/test_obsidian_frontmatter.py covers dict parsing, bool reads, field updates, control actions, deep reading validation
5. All downstream modules (deep_reading.py, repair.py, ocr.py, asset_index.py) continue to work
6. No behavioral changes to any frontmatter/deep-reading function
</success_criteria>

<output>
After completion, create `.planning/phases/058-service-extraction/058-03-SUMMARY.md`
</output>
