---
phase: 22-configuration-truth-compatibility
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - paperforge/config.py
  - paperforge/commands/sync.py
autonomous: true
requirements:
  - CONF-01
  - CONF-02

must_haves:
  truths:
    - "paperforge.json carries a `schema_version` field that downstream code can check"
    - "An older vault with legacy top-level path keys can be auto-migrated: the file is backed up as .bak, then rewritten with only vault_config + schema_version"
    - "`paperforge sync` calls the migration function at startup so existing vaults upgrade transparently"
    - "Python's load_vault_config() still reads both top-level and vault_config for backward compat during the transition window"
  artifacts:
    - path: "paperforge/config.py"
      provides: "DEFAULT_CONFIG with schema_version + migration engine"
      contains: "def migrate_paperforge_json"
    - path: "paperforge/commands/sync.py"
      provides: "Migration trigger at sync start"
      contains: "migrate_paperforge_json"
  key_links:
    - from: "commands/sync.py :: run()"
      to: "config.py :: migrate_paperforge_json()"
      via: "call before sync operations"
    - from: "config.py :: DEFAULT_CONFIG"
      to: "config.py :: CONFIG_KEYS"
      via: "schema_version key"

---

<objective>
Add `schema_version` to the PaperForge config, create a safe migration engine that rewrites legacy top-level keys into the `vault_config` canonical block with a `.bak` backup, and wire migration into `paperforge sync`.

Purpose: Eliminate the dual-path-key write pattern (top-level + vault_config) that causes drift. Let brownfield vaults upgrade silently. Give every paperforge.json a schema marker for future migrations.

Output: `config.py` with migration engine + `sync.py` with migration trigger.
</objective>

<execution_context>
@C:/Users/Lin/.opencode/get-shit-done/workflows/execute-plan.md
</execution_context>

<context>
@.planning/STATE.md
@.planning/ROADMAP.md (Phase 22)
@.planning/phases/22-configuration-truth-compatibility/22-CONTEXT.md (D-06..D-12)
@paperforge/config.py (full file, 309 lines)
@paperforge/commands/sync.py (full file, 95 lines)
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Add schema_version to DEFAULT_CONFIG, CONFIG_KEYS, and paperforge.json read/write</name>
  <files>paperforge/config.py</files>

  <read_first>
  @paperforge/config.py (lines 48-71 -- DEFAULT_CONFIG, ENV_KEYS, CONFIG_KEYS)
  @paperforge/config.py (lines 79-95 -- read_paperforge_json)
  @paperforge/config.py (lines 149-205 -- load_vault_config merge logic)
  </read_first>

  <behavior>
  - DEFAULT_CONFIG now includes `schema_version: 2`
  - CONFIG_KEYS now includes `schema_version`
  - `read_paperforge_json()` returns the raw dict including `schema_version` if present
  - `load_vault_config()` does NOT merge schema_version (it is metadata, not a path config value); it is only accessible via read_paperforge_json()
  - A public function `get_paperforge_schema_version(vault)` returns the schema_version from paperforge.json, defaulting to 1 if absent (meaning legacy format)
  </behavior>

  <action>
  Apply the following changes to `paperforge/config.py`:

  1. In `DEFAULT_CONFIG` dict (currently line 48), add the new entry as the first key:
     ```python
     DEFAULT_CONFIG: dict[str, str] = {
         "schema_version": "2",
         "system_dir": "99_System",
         "resources_dir": "03_Resources",
         "literature_dir": "Literature",
         "control_dir": "LiteratureControl",
         "base_dir": "05_Bases",
         "skill_dir": ".opencode/skills",
         "command_dir": ".opencode/command",
     }
     ```

  2. The ENV_KEYS map (line 59) stays unchanged -- schema_version is never set via env var. Do NOT add it there.

  3. The CONFIG_KEYS set (line 71) auto-derives from DEFAULT_CONFIG keys via `set(DEFAULT_CONFIG.keys())`, so it picks up schema_version automatically. No change needed.

  4. Add a new public function AFTER `read_paperforge_json()` (after line 95) and BEFORE `resolve_vault()`:
     ```python
     def get_paperforge_schema_version(vault: Path) -> int:
         """Return the schema_version from paperforge.json, defaulting to 1 if absent."""
         data = read_paperforge_json(vault)
         raw = data.get("schema_version", "1")
         try:
             return int(raw)
         except (ValueError, TypeError):
             return 1
     ```

  5. In `load_vault_config()` (line 149): After the init `config: dict[str, str] = dict(DEFAULT_CONFIG)` at line 175, remove schema_version from the config dict so it does not pollute path config output:
     ```python
     config.pop("schema_version", None)
     ```

  6. Update the load_vault_config() docstring to note: "schema_version is not a path config key and is excluded from the merged output (use get_paperforge_schema_version() instead)."
  </action>

  <verify>
  <automated>python -c "from paperforge.config import DEFAULT_CONFIG, CONFIG_KEYS, get_paperforge_schema_version; assert 'schema_version' in DEFAULT_CONFIG; assert 'schema_version' in CONFIG_KEYS; print('OK')"</automated>
  <automated>python -c "from paperforge.config import get_paperforge_schema_version; from pathlib import Path; import tempfile, json; d=tempfile.mkdtemp(); Path(d,'paperforge.json').write_text('{}'); assert get_paperforge_schema_version(Path(d))==1; print('default=1 OK'); Path(d,'paperforge.json').write_text(json.dumps({'schema_version':'2'})); assert get_paperforge_schema_version(Path(d))==2; print('schema=2 OK')"</automated>
  <automated>python -c "from paperforge.config import load_vault_config; from pathlib import Path; import tempfile; d=tempfile.mkdtemp(); Path(d,'paperforge.json').write_text('{}'); c=load_vault_config(Path(d)); assert 'schema_version' not in c; print('not_in_output OK')"</automated>
  </verify>

  <acceptance_criteria>
  - DEFAULT_CONFIG contains key `"schema_version"` with value `"2"`
  - File contains `def get_paperforge_schema_version(vault)` defined after `read_paperforge_json`
  - `get_paperforge_schema_version()` returns 1 when no schema_version key in paperforge.json
  - `get_paperforge_schema_version()` returns 2 when `"schema_version": "2"` in paperforge.json
  - `load_vault_config()` output dict does NOT contain `schema_version` key
  - All 3 automated verify commands pass
  </acceptance_criteria>

  <done>
  DEFAULT_CONFIG includes schema_version: "2". get_paperforge_schema_version() resolves correctly. load_vault_config() excludes schema_version.
  </done>
</task>

<task type="auto" tdd="true">
  <name>Task 2: Implement config migration engine in config.py</name>
  <files>paperforge/config.py</files>

  <read_first>
  @paperforge/config.py (lines 79-95 -- read_paperforge_json)
  @paperforge/config.py (lines 149-205 -- load_vault_config merge logic)
  @paperforge/config.py (lines 1-18 -- module imports, docstring)
  </read_first>

  <behavior>
  - `migrate_paperforge_json(vault)` detects legacy top-level path keys (system_dir, resources_dir, literature_dir, control_dir, base_dir)
  - If top-level path keys exist, it:
    1. Reads the full paperforge.json
    2. Separates non-path top-level keys (version, agent_platform, etc.) from path keys
    3. Merges top-level path values into vault_config block (top-level fills gaps where vault_config misses a key)
    4. Backs up old file as `paperforge.json.bak` (only on first migration)
    5. Writes new file with only vault_config + non-path keys + schema_version: "2"
  - Idempotent: if no top-level path keys exist, skip (returns False)
  - Safe: if paperforge.json does not exist, skip silently
  - Non-path top-level keys are preserved in the output root
  </behavior>

  <action>
  1. Add `import shutil` to the existing imports at the top:

     Current (lines 16-19):
     ```python
     import json
     import os
     from pathlib import Path
     from typing import Any
     ```

     Change to:
     ```python
     import json
     import os
     import shutil
     from pathlib import Path
     from typing import Any
     ```

  2. Add the following at the END of `paperforge/config.py` (after the `paths_as_strings` function):

     ```python
     # ---------------------------------------------------------------------------
     # Migration: legacy top-level keys to vault_config block
     # ---------------------------------------------------------------------------

     CONFIG_PATH_KEYS: tuple[str, ...] = (
         "system_dir",
         "resources_dir",
         "literature_dir",
         "control_dir",
         "base_dir",
     )


     def migrate_paperforge_json(vault: Path) -> bool:
         """Migrate paperforge.json from legacy top-level keys to vault_config block.

         If the file has top-level path keys (system_dir, resources_dir, etc.),
         this function:
           1. Reads the full existing content.
           2. Merges top-level path values into the ``vault_config`` block (top-level
              values fill gaps when vault_config is missing a key).
           3. Removes those top-level path keys from the root.
           4. Sets ``schema_version`` to ``"2"``.
           5. Backs up the original as ``paperforge.json.bak`` (first time only).
           6. Writes the cleaned file.

         Returns:
             True if migration was performed, False if no migration was needed.

         The function is idempotent -- calling it on an already-migrated file is a
         no-op (returns False).
         """
         path = vault / "paperforge.json"
         if not path.exists():
             return False

         try:
             data = json.loads(path.read_text(encoding="utf-8"))
         except (json.JSONDecodeError, OSError):
             return False
         if not isinstance(data, dict):
             return False

         # Detect legacy top-level path keys
         has_top_level_keys = any(k in data for k in CONFIG_PATH_KEYS)
         if not has_top_level_keys:
             return False  # Already migrated or fresh install

         # --- Build the cleaned output ---

         # 1. Non-path top-level keys to preserve (everything except path keys and vault_config)
         non_path_keys = {
             k: v for k, v in data.items()
             if k not in CONFIG_PATH_KEYS and k != "vault_config"
         }

         # 2. Build vault_config: start with existing vault_config, fill gaps from top-level
         vault_config = dict(data.get("vault_config", {}) or {})
         for key in CONFIG_PATH_KEYS:
             if key not in vault_config and key in data and data[key]:
                 vault_config[key] = data[key]

         # 3. Assemble output
         output = dict(non_path_keys)
         output["vault_config"] = vault_config
         output["schema_version"] = "2"

         # 4. Backup original (first time only)
         backup_path = path.with_suffix(".json.bak")
         if not backup_path.exists():
             shutil.copy2(str(path), str(backup_path))

         # 5. Write
         path.write_text(
             json.dumps(output, indent=2, ensure_ascii=False),
             encoding="utf-8",
         )
         return True
     ```
  </action>

  <verify>
  <automated>python -c "
import tempfile, json
from pathlib import Path
from paperforge.config import migrate_paperforge_json

# Test 1: legacy top-level keys - migrate
d = tempfile.mkdtemp()
data = {'system_dir': 'OldSystem', 'resources_dir': 'OldRes', 'version': '1.4', 'vault_config': {'system_dir': '99_System', 'resources_dir': '03_Resources'}}
Path(d, 'paperforge.json').write_text(json.dumps(data))
assert migrate_paperforge_json(Path(d)) == True, 'should migrate'
result = json.loads(Path(d, 'paperforge.json').read_text())
assert 'system_dir' not in result, 'top-level key removed'
assert result['vault_config']['system_dir'] == 'OldSystem', 'top-level fills vault_config gap'
assert result['vault_config']['resources_dir'] == '03_Resources', 'existing vault_config preserved'
assert result['schema_version'] == '2'
assert Path(d, 'paperforge.json.bak').exists(), 'backup created'
print('Test 1 OK: legacy migration works')

# Test 2: already migrated - no-op
Path(d, 'paperforge.json').write_text(json.dumps({'vault_config': {'system_dir': '99_System'}, 'schema_version': '2'}))
assert migrate_paperforge_json(Path(d)) == False, 'should be no-op'
print('Test 2 OK: idempotent')

# Test 3: no file - no-op
d2 = tempfile.mkdtemp()
assert migrate_paperforge_json(Path(d2)) == False, 'no file = no-op'
print('Test 3 OK: no file safe')
"</automated>
  </verify>

  <acceptance_criteria>
  - `import shutil` present in config.py imports
  - `def migrate_paperforge_json(vault)` present at module level
  - `migrate_paperforge_json()` returns True after migrating legacy keys
  - `migrate_paperforge_json()` returns False for already-migrated files (idempotent)
  - `migrate_paperforge_json()` returns False if paperforge.json does not exist
  - Migrated file has NO top-level path keys, only `vault_config` block
  - Migrated file has `schema_version: "2"`
  - Backup file `paperforge.json.bak` is created
  - Non-path top-level keys (version, etc.) survive in output root
  - All automated verify commands pass
  </acceptance_criteria>

  <done>
  Migration engine implemented and tested: legacy top-level path keys migrated to vault_config block with .bak backup.
  </done>
</task>

<task type="auto">
  <name>Task 3: Wire migration call into sync command</name>
  <files>paperforge/commands/sync.py</files>

  <read_first>
  @paperforge/commands/sync.py (lines 1-95 -- full file)
  @paperforge/config.py (migrate_paperforge_json function)
  </read_first>

  <action>
  In `paperforge/commands/sync.py`:

  1. Add import at top (after line 5 `from pathlib import Path`):
     ```python
     from paperforge.config import migrate_paperforge_json
     ```

  2. In the `run()` function, after the vault-path resolution block (after line 61 where vault is resolved), add migration call BEFORE any sync operations:

     Current line 61:
     ```python
         vault = resolve_vault(cli_vault=getattr(args, "vault", None))
     ```

     After line 61, add:
     ```python
         verbose = getattr(args, "verbose", False)
         # Auto-migrate paperforge.json from legacy top-level keys to vault_config block
         migrated = migrate_paperforge_json(vault)
         if migrated:
             logger.info("Migrated paperforge.json to vault_config canonical format")
             if verbose:
                 print("[INFO] paperforge.json migrated to canonical format (backup: paperforge.json.bak)")
     ```

     Note: `logger` is already defined at line 4 as `logger = logging.getLogger(__name__)`. No additional import needed.
  </action>

  <verify>
  <automated>python -c "
import tempfile, json
from pathlib import Path
from paperforge.config import migrate_paperforge_json

d = tempfile.mkdtemp()
Path(d, 'paperforge.json').write_text(json.dumps({
    'system_dir': 'OldSys', 'version': '1.4', 'vault_config': {}}))
result = migrate_paperforge_json(Path(d))
assert result == True
data = json.loads(Path(d, 'paperforge.json').read_text())
assert 'system_dir' not in data
assert data['vault_config']['system_dir'] == 'OldSys'
print('Sync migration wire test: OK')
"</automated>
  <automated>python -c "import ast; tree=ast.parse(open('paperforge/commands/sync.py').read()); calls=[n for n in ast.walk(tree) if isinstance(n,ast.Call) and getattr(n.func,'attr',None)=='migrate_paperforge_json']; assert len(calls)>0, 'migrate_paperforge_json must be called'; print('sync.py calls migrate_paperforge_json: OK')"</automated>
  </verify>

  <acceptance_criteria>
  - `paperforge/commands/sync.py` imports `migrate_paperforge_json` from `paperforge.config`
  - `sync.py` `run()` function contains a call to `migrate_paperforge_json(vault)` before sync operations
  - `sync.py` `run()` has `verbose = getattr(args, "verbose", False)` available
  - Migration info log message emitted when migration occurs
  - Console message printed when migration occurs AND verbose is True
  - Both automated verify commands pass
  </acceptance_criteria>

  <done>
  Migration engine wired into sync command. Every `paperforge sync` call auto-migrates legacy configs before any sync operation.
  </done>
</task>

</tasks>

<verification>

### Plan 01 Verification

1. **schema_version in DEFAULT_CONFIG:** `DEFAULT_CONFIG` dict contains key `schema_version` with value `"2"`
2. **get_paperforge_schema_version function:** Exists at module level, returns int, defaults to 1 when key absent
3. **load_vault_config excludes schema_version:** Output dict does NOT contain `schema_version` key
4. **migrate_paperforge_json function:** Exists at module level, accepts `Path`, returns `bool`
5. **Migration test:** Legacy top-level keys convert to vault_config block with .bak backup
6. **Idempotent test:** Already-migrated file returns False
7. **sync.py integration:** Import + call in `run()` present
8. **Full integration:** Create legacy paperforge.json -> run sync test -> verify migration
</verification>

<success_criteria>

- [ ] `DEFAULT_CONFIG` has `"schema_version": "2"` (verified via `'schema_version' in DEFAULT_CONFIG`)
- [ ] `get_paperforge_schema_version()` returns correct int from paperforge.json, defaults to 1
- [ ] `load_vault_config()` output excludes `schema_version`
- [ ] `migrate_paperforge_json()` migrates legacy files correctly (all test cases pass)
- [ ] `paperforge sync` triggers migration automatically
- [ ] Legacy top-level keys are backed up as `paperforge.json.bak` before rewrite
- [ ] Non-path top-level keys (version, agent_platform, etc.) survive migration
- [ ] All automated verify commands pass
</success_criteria>

<output>
After completion, create `.planning/phases/22-configuration-truth-compatibility/22-01-SUMMARY.md`
</output>
