From 8657bc53b34784199e160dcbad38acd69f776def Mon Sep 17 00:00:00 2001 From: WiseGuru <42100212+WiseGuru@users.noreply.github.com> Date: Wed, 8 Jul 2026 21:38:30 -0700 Subject: [PATCH] 1.2.0 Real-time dictation (AssemblyAI/Deepgram, own provider/key/model), auto-ingest folders, record-in-background, per-invocation diarization, manage built-in templates (enable/track), local whisper.cpp auto-start/idle-stop, and dev tooling (local review + release-checklist skill). Voxtral realtime was reverse-engineered but pulled (not WebView-reachable); its adapter is kept on disk, unwired. Co-Authored-By: Claude Opus 4.8 --- .claude/skills/release-checklist/CHECKLIST.md | 29 +- .claude/skills/release-checklist/SKILL.md | 45 +- CLAUDE.md | 96 ++++- README.md | 1 + dev-tools.config.example.json | 4 +- docs/DEV_TOOLING.md | 16 +- docs/DIARIZATION.md | 11 +- docs/RELEASING.md | 4 +- docs/ROADMAP.md | 104 ++--- docs/WHISPER_HOST.md | 11 +- local-review.mjs | 143 ++++++- manifest.json | 2 +- package-lock.json | 4 +- package.json | 3 +- src/audio-persist.ts | 51 ++- src/ingest.ts | 236 +++++++++++ src/main.ts | 105 ++++- src/pipeline.ts | 18 +- src/realtime/assemblyai.ts | 104 +++++ src/realtime/deepgram.ts | 81 ++++ src/realtime/index.ts | 96 +++++ src/realtime/pcm.ts | 137 ++++++ src/realtime/voxtral.ts | 133 ++++++ src/settings/index.ts | 46 +++ src/settings/tab.ts | 389 ++++++++++++++++-- src/templates-folder.ts | 108 ++++- src/transcription/whisper-local.ts | 71 ++-- src/types.ts | 59 ++- src/ui/confirm-modal.ts | 9 + src/ui/ingest-rule-modal.ts | 115 ++++++ src/ui/modal.ts | 90 ++++ src/ui/quick-record.ts | 50 ++- src/ui/realtime.ts | 214 ++++++++++ src/whisper-host.ts | 59 +++ styles.css | 108 +++++ test/ingest.test.ts | 116 ++++++ test/local-review.test.ts | 79 +++- test/mocks/obsidian.ts | 1 + test/realtime-pcm.test.ts | 60 +++ test/settings-merge.test.ts | 61 +++ test/templates-folder.test.ts | 35 ++ test/whisper-host.test.ts | 42 +- versions.json | 5 +- wiki/Commands-and-Menus.md | 8 + wiki/Creating-Templates.md | 18 +- wiki/Home.md | 1 + wiki/Mobile.md | 2 + wiki/Providers.md | 43 +- wiki/Self-Hosting-Whisper.md | 2 + wiki/Settings-Reference.md | 41 +- wiki/Voxtral-Realtime.md | 78 ++++ wiki/_Sidebar.md | 1 + 52 files changed, 3088 insertions(+), 257 deletions(-) create mode 100644 src/ingest.ts create mode 100644 src/realtime/assemblyai.ts create mode 100644 src/realtime/deepgram.ts create mode 100644 src/realtime/index.ts create mode 100644 src/realtime/pcm.ts create mode 100644 src/realtime/voxtral.ts create mode 100644 src/ui/ingest-rule-modal.ts create mode 100644 src/ui/realtime.ts create mode 100644 test/ingest.test.ts create mode 100644 test/realtime-pcm.test.ts create mode 100644 wiki/Voxtral-Realtime.md diff --git a/.claude/skills/release-checklist/CHECKLIST.md b/.claude/skills/release-checklist/CHECKLIST.md index d33180b..31343d6 100644 --- a/.claude/skills/release-checklist/CHECKLIST.md +++ b/.claude/skills/release-checklist/CHECKLIST.md @@ -34,6 +34,7 @@ Keep this file current: when a feature is added or changed, add or update its it - [ ] **Destination** override: changing insert mode / folder / filename affects this run only; "Reset to template default" clears it; the template file on disk is unchanged. Result: ____ - [ ] **Context hint**: the `
` appears only for a template with `enableContextHint`; the typed hint reaches the LLM (visible in the result); resets on template change. Result: ____ - [ ] Setup card blocks Record/Paste when the active profile is unconfigured, with the right voice-vs-text messaging. Result: ____ +- [ ] **Record in background** (desktop): the checkbox shows on the Record tab, persists across reopen; ticking it and pressing Record closes the modal and continues in the Quick Record floater carrying the template/destination/context; hidden on mobile. Result: ____ ## 3. Templates @@ -44,7 +45,8 @@ Keep this file current: when a feature is added or changed, add or update its it - [ ] **Note properties**: a template with `noteProperties` writes the declared keys into the new note's frontmatter (newFile only). Result: ____ - [ ] **Note title** (`titleFromContent`): the LLM-generated title names the new file via `{{title}}` / whole-name; falls back to the static name when unusable. Result: ____ - [ ] `disableSharedCore: true` runs a template without the shared-core preface (and settings shows the warning line naming it). Result: ____ -- [ ] `diarize: true` on a template forces speaker labels on a capable provider. Result: ____ +- [ ] **Diarization is per-invocation**: the modal shows an "Identify speakers" checkbox only on a capable provider (AssemblyAI/Deepgram/Rev.ai), defaulting to the template's `diarize` flag (on for Meeting transcript); it can be unticked per run; there is no "Identify speakers" toggle in Settings; a non-diarize template (e.g. Daily note) produces no `Speaker` labels. Result: ____ +- [ ] **Manage built-in templates**: each row has a clearly-labelled Enabled switch (caption reads "Enabled" when on, "Disabled" when off) and a Tracked checkbox with its label to the LEFT of the box (reads "Tracked" when checked, "Untracked" when unchecked); turning Enabled off (confirm modal) removes the file and Populate/Update no longer re-add it; turning it back on restores it; unchecking Tracked writes `managed: false` and Update then leaves it untouched (reported as untracked). Result: ____ ## 4. Transcription providers @@ -68,6 +70,8 @@ Keep this file current: when a feature is added or changed, add or update its it - [ ] A non-loopback `--host` in Extra args is refused before spawn. Result: ____ - [ ] Restarting Obsidian adopts the orphaned server (status shows "adopted from previous session"); an externally-started server shows "external" and Stop is disabled. Result: ____ - [ ] Status-bar item is hidden on mobile / when the active profile isn't whisper-local. Result: ____ +- [ ] **Start automatically**: with the toggle on and the profile using whisper-local, the server starts on Obsidian launch (a still-running one is adopted, not doubled). Result: ____ +- [ ] **Stop when idle**: with a small minutes value set, the server stops after that idle time; it never stops mid-transcription or an externally-started server. Result: ____ ## 7. Known nouns / assistant prompt (wake name) @@ -102,13 +106,32 @@ Keep this file current: when a feature is added or changed, add or update its it - [ ] Template popover: opens, is keyboard-navigable (Arrow/Escape), dismisses on selection / outside click / when processing starts. Result: ____ - [ ] Cancel works both during capture and during post-capture processing (aborts the pipeline). Result: ____ -## 12. Mobile +## 12. Real-time transcription + +- [ ] `Real-time transcription (start/stop)` requires a **Real-time provider** set (AssemblyAI/Deepgram, independent of the batch transcription provider; errors with a steer otherwise), its real-time key set, and an open Markdown note. Verify the batch provider and real-time provider can differ (e.g. batch Voxtral + real-time AssemblyAI). Result: ____ +- [ ] Settings shows a self-contained **Real-time transcription** section: a Real-time provider dropdown (None + realtime-capable only, so **only AssemblyAI/Deepgram, NOT Voxtral**) that re-renders on change, plus key + adaptive model field when a provider is picked; the model field is a dropdown with Refresh for Deepgram and a text field for AssemblyAI; the key persists/hydrates across reload under its own secret. Result: ____ +- [ ] AssemblyAI/Deepgram: starting shows the floating bar (dot + rolling interim text + Stop); spoken words appear at the cursor as finalized segments; interim text is never inserted. Result: ____ +- [ ] **Voxtral real-time is NOT offered** (pulled: not WebView-reachable). Confirm it is absent from the Real-time provider dropdown. (The adapter stays on disk, unwired.) Result: ____ +- [ ] Stop flushes the last segment and closes the connection; running the command again while active stops it. Result: ____ + +## 13. Auto-ingest folders + +- [ ] Settings "Auto-ingest folders": Add opens the rule popup; the template dropdown lists only newFile templates; rules list with enable toggle / Edit / Delete. Result: ____ +- [ ] `Process auto-ingest folders` processes each audio file in an enabled folder with that rule's template, one at a time, behind a sticky Notice with Cancel. Result: ____ +- [ ] On success each recording is moved to the attachments location and its note's `![[embed]]` still resolves; a re-run does not reprocess it. Result: ____ +- [ ] A file that fails stays in the ingest folder and is retried next run; the summary Notice reports processed/failed counts **and** a second sticky Notice lists the concrete failure reason per file (first 5, then "…and N more"). Result: ____ +- [ ] **Ingest folder == recordings/attachments folder** is caught UP FRONT: setting a rule's folder to the same folder recordings are stored in (either the ReWrite Attachments folder, or Obsidian's own attachment folder when ReWrite's is blank) skips the rule with a clear "this folder is also where recordings are stored; point the rule at a different folder" Notice BEFORE any note is created. Result: ____ +- [ ] **Move-after-success failure**: if the note is created but the recording can't be moved out, the error says the note was created and to move/delete the source manually (so it isn't reprocessed). Result: ____ + +## 14. Mobile - [ ] On mobile, whisper-local and other desktop-only options are absent from dropdowns. Result: ____ - [ ] Modals pin to the top so fields stay above the soft keyboard; Paste textarea is shorter (rows=4). Result: ____ - [ ] Record + Paste paths work on mobile. Result: ____ +- [ ] While recording on mobile, a yellow "keep Obsidian in the foreground" caution shows in the Record tab and the Quick Record floater; it's absent on desktop and hidden once processing starts. Result: ____ +- [ ] Auto-ingest and real-time transcription work on mobile (both are cross-platform). Result: ____ -## 13. Misc +## 15. Misc - [ ] Errors surface as provider-attributed Notices; no secret leaks into a Notice or log (query strings redacted). Result: ____ - [ ] Cancelling a run (modal / Quick Record) leaves the persisted audio file as the recovery path and writes nothing further. Result: ____ diff --git a/.claude/skills/release-checklist/SKILL.md b/.claude/skills/release-checklist/SKILL.md index 3808c50..ecda427 100644 --- a/.claude/skills/release-checklist/SKILL.md +++ b/.claude/skills/release-checklist/SKILL.md @@ -1,39 +1,54 @@ --- name: release-checklist -description: Run the ReWrite (Voice Notes) plugin's pre-release verification. Use before tagging a release, or when the user asks to smoke-test / verify the plugin in a real vault. Automates the build-and-install-into-a-scratch-vault step, then walks the feature-by-feature CHECKLIST.md so the human effort goes into clicking through features rather than copying files. +description: Run the ReWrite (Voice Notes) plugin's pre-release verification. Use before tagging a release, or when the user asks to smoke-test / verify the plugin in a real vault. Runs the automated pre-checks (build, lint, test, and the advisory local reviews), installs a fresh build into a test vault, then walks the feature-by-feature CHECKLIST.md so the human effort goes into clicking through features rather than copying files. --- # Release checklist -This plugin has no headless UI test harness, so event-wiring and UI-lifecycle bugs are invisible to `npm run build` / `npm run lint` / `npm test`. This skill closes that gap: it installs a fresh build into a scratch Obsidian vault, then drives a human through a feature-by-feature manual pass. +This plugin has no headless UI test harness, so event-wiring and UI-lifecycle bugs are invisible to `npm run build` / `npm run lint` / `npm test`. This skill closes that gap: it runs every automated check available, installs a fresh build into a test Obsidian vault, then drives a human through a feature-by-feature manual pass. -It is the successor to the stale "Testing Checklist" in `obsidian-voice-notes-spec.md` and the token "manual smoke test" step in `docs/RELEASING.md`. See `docs/DEV_TOOLING.md` for how the automated portion works. +It is the successor to the stale "Testing Checklist" in `obsidian-voice-notes-spec.md` and the token "manual smoke test" step in `docs/RELEASING.md`. The mechanics of each tool are in `docs/DEV_TOOLING.md`; the tag/push release steps are in `docs/RELEASING.md`. This skill is the front door that sequences them. -## Phase 1 — Setup check +## Phase 1 — Automated pre-checks -1. Confirm `dev-tools.config.json` exists at the repo root with a non-empty `releaseVault.vaultPath`. If not, tell the user to copy `dev-tools.config.example.json` to `dev-tools.config.json` and fill in `releaseVault.vaultPath` (a **scratch** vault, never a real personal one). Do not proceed without it. -2. Confirm the working tree is in the state the user intends to ship (everything they want tested is saved/committed). +Run all of these first; they are cheap and catch the machine-checkable problems before a human spends time clicking: -## Phase 2 — Build and install (automated) +```bash +npm run build && npm run lint && npm test # CI parity — must all pass +npm run review # advisory local code review of the working-tree diff +npm run review:docs # advisory local doc-consistency review +``` -Run the release-prep script via Bash: +- `build`/`lint`/`test` are release blockers if they fail. +- The two `review` runs are **advisory** (always exit 0). Read the reports in `docs/claude-scratch/local-review--report.md` and surface anything real. They will not block, and they are imperfect (local, quantized model), so treat findings as a first-pass filter, not a gate. +- **If a review report says "(no findings returned)" or looks empty/garbled**, it is almost always a model/tooling issue, not a clean bill of health — see `docs/DEV_TOOLING.md` Gotchas (reasoning models need thinking disabled; the context must fit the prompt; a huge diff truncates). Fix the tooling or note it; do not read empty output as "no problems." + +## Phase 2 — Setup check + +1. Confirm `dev-tools.config.json` exists at the repo root with a non-empty `releaseVault.vaultPath`. If not, tell the user to copy `dev-tools.config.example.json` to `dev-tools.config.json` and fill it in. A dedicated **scratch** vault is ideal; a real vault is acceptable because `release:prep` only overwrites the three build artifacts (see Phase 3), but never point it anywhere you'd mind a plugin reload. +2. Confirm the working tree holds exactly what the user intends to ship. For a real release you also want the version already bumped (`npm version --no-git-tag-version`) so the manual pass exercises the actual versioned artifact **before** you commit and tag — do not tag first. + +## Phase 3 — Build and install ```bash npm run release:prep ``` -This runs `npm run build`, then copies `main.js` / `manifest.json` / `styles.css` into `/.obsidian/plugins/rewrite-voice-notes/`. It fails loudly (non-zero exit) on a build error or a bad vault path; surface any failure to the user and stop. +This runs `npm run build`, then copies `main.js` / `manifest.json` / `styles.css` into `/.obsidian/plugins/rewrite-voice-notes/`. It fails loudly (non-zero exit) on a build error or a bad vault path; surface any failure and stop. It **only overwrites those three files** — `data.json` and `secrets.json.nosync` in the target folder are left untouched, so a re-install keeps the user's settings and keys (this is why running against a vault with real data is safe). -Then tell the user to open that scratch vault in Obsidian and reload the plugin (Settings -> Community plugins -> toggle it off and on, or reload Obsidian), and confirm it loads with no errors in the developer console. **A clean load is itself the first checklist item** — it is the exact step that would have caught the regression that motivated this tooling. +Then tell the user to open the vault and reload the plugin (Settings -> Community plugins -> toggle it off and on, or reload Obsidian), and confirm it loads with no errors in the developer console. **A clean load is itself the first checklist item** — it is the exact step that would have caught the regression that motivated this tooling. -## Phase 3 — Feature pass (human-driven) +## Phase 4 — Feature pass (human-driven) -Open `CHECKLIST.md` (in this skill folder) and walk it area by area. For each item, ask the user to perform the action in the scratch vault and report the result; record PASS / FAIL / SKIP and any notes on the `Result: ____` line. +Open `CHECKLIST.md` (in this skill folder) and walk it area by area. For each item, ask the user to perform the action in the vault and report the result; record PASS / FAIL / SKIP and any notes on the `Result: ____` line. -- Scope the pass to what changed when the user only touched part of the plugin, but always include the "Core" area and a clean-load check. +- Scope the pass to what changed when the user only touched part of the plugin, but always include the "Core" area and a clean-load check. Give the newest, least-tested features the most attention. - On any FAIL, capture the exact symptom (and console output if any) — that is the finding, not "it broke." -- `CHECKLIST.md` is also runnable standalone by a human with no Claude Code; keep it self-contained prose. +- Mobile-only items need a phone; SKIP is fine if the user isn't testing mobile that round. +- `CHECKLIST.md` is also runnable standalone by a human with no Claude Code; keep it self-contained prose, and keep it in sync with the feature set (a feature change updates it in the same commit, like CLAUDE.md and the wiki). -## Phase 4 — Summary +## Phase 5 — Summary and hand-off to release Save the filled-out run to `docs/claude-scratch/release-checklist-.md` (read `` from `manifest.json`; that folder is gitignored). Then give a clear **go / no-go**: list every FAIL with its symptom, note any SKIP and why, and state plainly whether the build is releasable. Do not soften a FAIL into a pass. + +On a **go**, the actual release (commit the bump + rolled `docs/ROADMAP.md`, tag the bare version, push, let CI build the attested assets) follows `docs/RELEASING.md` — this skill stops at the go/no-go decision; it does not tag or push. diff --git a/CLAUDE.md b/CLAUDE.md index 0ffa6e2..6a00d67 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -35,6 +35,7 @@ Page map (keep `_Sidebar.md` and `Home.md`'s contents list in sync when adding/r - `Commands-and-Menus.md`: command palette, ribbon, editor/file menus, Quick Record UI ([src/main.ts](src/main.ts)). - `Creating-Templates.md`: template file format + authoring guide. This is the canonical user-facing template guide (it replaced the old in-vault `Template guide.md`); keep it in sync with the `NoteTemplate` schema. - `Providers.md`: provider tables, model selection, base-URL conventions, diarization, context hint, known nouns, recording limits. +- `Voxtral-Realtime.md`: the reverse-engineered Voxtral realtime WebSocket protocol, the browser-auth caveat (why it is disabled/unwired, not shipped), how a contributor would re-enable and test it, and a contributor ask. Keep in sync with `src/realtime/voxtral.ts`. - `Self-Hosting-Whisper.md`: local whisper.cpp host (mirrors user-facing parts of [docs/WHISPER_HOST.md](docs/WHISPER_HOST.md)). - `Self-Hosting-LLMs.md`: Ollama/llama.cpp local and remote, plus low-spec model recommendations (refresh these as models age). - `Secrets-and-Sync.md`: encryption modes summary + the 7 sync-exclusion recipes. @@ -64,7 +65,7 @@ CI ([.github/workflows/lint.yml](.github/workflows/lint.yml)) runs `npm ci`, `np ### Testing -[vitest.config.ts](vitest.config.ts) drives a small [Vitest](https://vitest.dev) suite under [test/](test/) covering pure/isolable logic: `whisper-host` (`getHostArgs`/`isLoopbackHost`), `pipeline.extractFromBlock`, `secrets` (passphrase round trip, corrupt-envelope recovery), `settings mergeSettings`, `templates-folder` (parse/render round trip, `mergeTemplate`), `transcription/limits`, and the `version-bump.mjs` guard. There is no UI/integration test layer; UI behavior is still verified manually per the spec checklist. +[vitest.config.ts](vitest.config.ts) drives a small [Vitest](https://vitest.dev) suite under [test/](test/) covering pure/isolable logic: `whisper-host` (`getHostArgs`/`isLoopbackHost`/`shouldStopWhenIdle`), `pipeline.extractFromBlock`, `secrets` (passphrase round trip, corrupt-envelope recovery), `settings mergeSettings` (incl. `disabledDefaultTemplateIds` / `ingestRules` sanitization), `templates-folder` (parse/render round trip, `mergeTemplate`, `managed` tri-state), `realtime/pcm` (`downsampleBuffer`/`floatTo16BitPcm`), `ingest` (`isIngestTemplate`/`collectIngestFiles`), `transcription/limits`, and the `version-bump.mjs` guard. There is no UI/integration test layer; UI behavior is still verified manually per the spec checklist. Two things make `src/` runnable under Vitest even though it targets the Obsidian runtime, not Node: - **The `obsidian` npm package ships only `.d.ts` files, no runtime JS.** [test/mocks/obsidian.ts](test/mocks/obsidian.ts) is a minimal runtime stand-in (just the classes/functions the tested modules actually import: `Notice`, `Plugin`, `TFile`, `parseYaml`/`stringifyYaml` via the `yaml` package, `normalizePath`, etc.), aliased in via `resolve.alias` in [vitest.config.ts](vitest.config.ts). It is not a full API shim; extend it only as new modules come under test. @@ -96,7 +97,8 @@ src/ ├── audio-transcode.ts # WebAudio decode + resample to 16 kHz mono PCM WAV (shared by whisper-local and mistral-voxtral) ├── pipeline.ts # transcribe → cleanup → insert orchestrator ├── insert.ts # cursor/newFile/append + {{date}}/{{time}} expansion -├── whisper-host.ts # Spawns/stops a user-supplied whisper-server child process (desktop only) +├── ingest.ts # Auto-ingest batch runner: scan rule folders, pipeline each audio file, move on success +├── whisper-host.ts # Spawns/stops a user-supplied whisper-server child process (desktop only; Phase B auto-start + idle-stop) ├── templates-folder.ts # Load templates from a vault folder + populate it with the 10 defaults ├── template-guide.ts # Writes the "Template update report.md" (the Update button's worklist) next to the templates folder; never loaded by the plugin. (The in-vault user guide was removed; the template guide now lives in the wiki.) ├── shared-core.ts # Load the shared cleanup preface from a vault Markdown file + populate default (prepended to every template prompt) @@ -109,13 +111,20 @@ src/ │ ├── tab.ts # PluginSettingTab: active profile, two profile sections, templates folder, recording │ └── default-templates.ts # The 10 default templates used by the populate button (General cleanup, Todo list, Daily note, Meeting notes, Meeting transcript, Idea capture, Lecture, Podcast, Guides, Book log); each prompt = per-template rules only (the shared preface is prepended at runtime from shared-core.ts) ├── ui/ -│ ├── modal.ts # Main modal: template select + Record/Paste/From note tabs + setup-card injection +│ ├── modal.ts # Main modal: template select + Record/Paste/From note tabs + setup-card injection + record-in-background handoff │ ├── setup-card.ts # Inline blocker when active profile is unconfigured (voice vs text purpose) -│ ├── quick-record.ts # QuickRecordController + floating mini-UI for the Quick Record command +│ ├── quick-record.ts # QuickRecordController + floating mini-UI for the Quick Record command (also hosts modal handoffs) +│ ├── realtime.ts # RealtimeController + floating bar for live dictation at the cursor │ ├── template-picker.ts # Lightweight modal for picking a template (used by Process text command and editor menu) +│ ├── ingest-rule-modal.ts # Popup editor for one auto-ingest rule (folder + newFile template + enabled) │ ├── text-source.ts # resolveActiveTextSource + runTextPipeline helpers for text-source flows │ ├── pipeline-progress.ts # runBackgroundPipeline: shared sticky-Notice-with-Cancel wrapper for detached pipeline runs; stageLabel/formatDuration │ └── whisper-status-bar.ts # Status-bar dot for whisper-host start/stop (desktop + whisper-local profile only) +├── realtime/ +│ ├── index.ts # RealtimeProvider/RealtimeSession interfaces + factory + transcriptionProviderSupportsRealtime() +│ ├── pcm.ts # PcmCapture (ScriptProcessor mic tap) + pure downsample/quantize helpers +│ ├── deepgram.ts # Deepgram live WS adapter (auth via ['token', key] subprotocol) +│ └── assemblyai.ts # AssemblyAI Universal-Streaming WS adapter (ephemeral temp token) ├── transcription/ │ ├── index.ts # TranscriptionProvider interface + createTranscriptionProvider() │ ├── limits.ts # Per-provider maxBytes/maxDurationMs + validateRecording (called from pipeline) @@ -145,6 +154,8 @@ The pipeline accepts an `AbortSignal` (forwarded to providers) and is consumed b The modal has two pipeline-running paths. The **Record** tab closes the modal the moment recording stops and runs the pipeline detached via `startRecordingPipeline` ([src/ui/modal.ts](src/ui/modal.ts)), which delegates to the shared `runBackgroundPipeline` helper in [src/ui/pipeline-progress.ts](src/ui/pipeline-progress.ts), mirroring `runAudioFilePipeline` in [src/ui/audio-source.ts](src/ui/audio-source.ts) and `runTextPipeline` in [src/ui/text-source.ts](src/ui/text-source.ts), all three of which now go through the same helper. `runBackgroundPipeline` opens a sticky `Notice` with a **Cancel** button (`setMessage` per stage), wires an `AbortController` into `PipelineParams.signal`, and on completion saves `lastUsedTemplateId` + shows a completion `Notice`; on error (including a user cancel, which surfaces as an `AbortError`) it shows the failure message and calls an optional `onError` hook. This is safe because the `persist-audio` stage saved the recording before transcription, so the vault file is the recovery path even if a cancel/error happens after that point. The **Paste** and **From note** tabs instead use `execute`, which keeps the modal open with an inline `.rewrite-progress` line and renders an inline **Retry** button on error (they have no persisted recovery, so closing would lose the input); `execute` is guarded against re-entrancy (`if (this.running) return;`) and the Paste/From-note buttons disable themselves for the duration, since nothing previously stopped a double-click from launching two concurrent pipeline runs. Destination override and context hint are captured into locals before `startRecordingPipeline` closes the modal, so a recorded run still honors them. +**Record in background** (desktop only): the Record tab shows a checkbox backed by `GlobalSettings.recordInBackground` (persisted; default off, hidden on mobile where the WebView suspends backgrounded `MediaRecorder`). When on, pressing Record does not capture in the modal at all: `startBackgroundHandoff` ([src/ui/modal.ts](src/ui/modal.ts)) captures the template / destination override / context hint into locals, closes the modal, and calls `plugin.startBackgroundRecording(opts)` ([src/main.ts](src/main.ts)), which routes into the SAME `activeQuickRecord` slot + `quickRecordStarting` guard as the two Quick Record commands (single-recording guarantee, one `onunload` cleanup path). `QuickRecordController` carries the handed-off `destinationOverride` / `contextHint` into `runPipeline`; picking a *different* template from the floater's popover clears both (they were set up against the original template). When the checkbox is off, capture stays in-modal, and Stop still closes the modal + runs detached as described above, so processing never blocks Obsidian either way. + Note on cancellation: Obsidian's `requestUrl` (used by every provider adapter, see [src/http.ts](src/http.ts)) accepts no `AbortSignal`, so Cancel only takes effect between network calls (e.g. before the next poll in `assemblyai.ts`/`revai.ts`), not mid-transfer. It still recovers a pipeline stuck at a stage that would otherwise never resolve. The `PipelineSource` union has three variants: `audio` (recorded blob, optional `sourcePath` for reprocess flows), `paste` (textarea input), `text` (input from an existing note via selection or whole body). Text-source flows skip transcription entirely and only require the LLM half of the profile. The `audio` variant's `sourcePath` is set by the reprocess flow ([src/ui/audio-source.ts](src/ui/audio-source.ts)) to point at an existing vault file; when present, the persist stage is skipped and that path is reused for the `![[]]\n\n` prepend. @@ -153,29 +164,31 @@ The `PipelineSource` union has three variants: `audio` (recorded blob, optional [src/transcription/index.ts](src/transcription/index.ts) and [src/llm/index.ts](src/llm/index.ts) each define a small interface plus a `create...Provider(id)` factory. Provider families share one adapter file where the API shapes match: OpenAI Whisper, `openai-compatible`, and Groq all dispatch into [src/transcription/openai.ts](src/transcription/openai.ts) with different base URLs; OpenAI GPT, `openai-compatible`, and Mistral all dispatch into [src/llm/openai.ts](src/llm/openai.ts). Mistral Voxtral does NOT share with `openai.ts` (see [src/transcription/mistral-voxtral.ts](src/transcription/mistral-voxtral.ts)) because Voxtral's response is JSON-only (no `response_format=text`) and it rejects WebM input (so the blob is always transcoded to 16 kHz mono WAV via [src/audio-transcode.ts](src/audio-transcode.ts)). -API keys are stored per profile on `EnvironmentProfile.transcriptionConfig.apiKey` / `llmConfig.apiKey`. Two slots per profile, one for transcription and one for the LLM. No global by-family map; the desktop and mobile profiles each carry their own keys even when both use the same provider (deliberate: per-profile keys make per-function usage tracking easier). Persistence is in [src/secrets.ts](src/secrets.ts) using the key IDs `profile-desktop-transcription`, `profile-desktop-llm`, `profile-mobile-transcription`, `profile-mobile-llm`. These ids (and the namespace prefix `secrets.ts` prepends for `secretStorage`) are dash-joined, NOT colon-joined: Obsidian's `app.secretStorage.setSecret` throws on any id that is not lowercase-alphanumeric + dashes, so a colon/underscore id silently fails the availability probe on every platform. Do not reintroduce colons or underscores into a secret id. +API keys are stored per profile on `EnvironmentProfile.transcriptionConfig.apiKey` / `llmConfig.apiKey` / `realtimeConfig.apiKey`. Three slots per profile: transcription, LLM, and real-time (the last a separate key + model because streaming models/keys differ from batch; see Real-time transcription). No global by-family map; the desktop and mobile profiles each carry their own keys even when both use the same provider (deliberate: per-profile keys make per-function usage tracking easier). Persistence is in [src/secrets.ts](src/secrets.ts) via [src/settings/index.ts](src/settings/index.ts) using the key IDs `profile-desktop-transcription`, `profile-desktop-llm`, `profile-desktop-realtime`, and the three `profile-mobile-*` equivalents. These ids (and the namespace prefix `secrets.ts` prepends for `secretStorage`) are dash-joined, NOT colon-joined: Obsidian's `app.secretStorage.setSecret` throws on any id that is not lowercase-alphanumeric + dashes, so a colon/underscore id silently fails the availability probe on every platform. Do not reintroduce colons or underscores into a secret id. Providers may optionally implement `listModels(config, signal)` returning a string array of model IDs the configured API key can access. Implemented by: OpenAI / Groq / Mistral (via `openai.ts` shared adapter), Mistral Voxtral (own adapter, filters Mistral's `/v1/models` catalog by ID substring `voxtral`), Anthropic, Gemini, Deepgram. Not implemented for `openai-compatible` (URL-specific, list-shape varies), AssemblyAI, Rev.ai. The settings tab caches results to `GlobalSettings.modelCache` per side and provider ID; the Refresh button in the model field triggers `listModels` and updates the cache. The model field is a single adaptive control (`populateModelField` in [src/settings/tab.ts](src/settings/tab.ts), shared by both sides via a `side: 'transcription' | 'llm'` arg): a **dropdown** when the provider supports `listModels` and the cache is non-empty, otherwise a **plain text field**. There is never both at once. The dropdown carries a trailing "Custom..." option (sentinel `CUSTOM_MODEL_OPTION`, never written to `config.model`) that toggles the same control into the text field (the `forceText` arg) so a model ID not in the catalog can still be typed; a "Back to list" button returns to the dropdown. A custom value not in the cache is shown as a selected ` (custom)` dropdown option. The Refresh button accompanies the dropdown (and the empty-cache text field, so the first fetch can flip it to a dropdown); on success the field re-renders in dropdown mode. `openai-compatible` / AssemblyAI / Rev.ai (no `listModels`) always show a plain text field with no Refresh. For AssemblyAI and Rev.ai the field description appends a "list of models" external link to the provider's docs (`transcriptionModelDocsUrl` + `applyModelFieldDesc` in [src/settings/tab.ts](src/settings/tab.ts)); `openai-compatible` has none (the model list is the user's own server). Whichever control is active, the canonical source is `profile.config.model`. ## Speaker diarization -Opt-in `Speaker X:` labels, supported only on `assemblyai` / `deepgram` / `revai`. Two switches: a per-profile `TranscriptionConfig.diarize` toggle and a per-template `NoteTemplate.diarize` override (frontmatter `diarize: true`, raises the effective setting only; the Meeting transcript default ships with it). Capability is gated by `transcriptionProviderSupportsDiarization(id)` ([src/transcription/index.ts](src/transcription/index.ts)); on a non-capable provider the flag is a documented no-op. Labels survive cleanup via a `DEFAULT_SHARED_CORE` clause. +Opt-in `Speaker X:` labels, supported only on `assemblyai` / `deepgram` / `revai`. Chosen **per invocation**, not as a persisted setting (a profile-wide toggle would wrongly diarize e.g. daily notes, producing stray `Speaker A` labels). Two inputs: the per-template `NoteTemplate.diarize` flag (frontmatter `diarize: true`; the Meeting transcript default ships with it) and the main modal's per-run **"Identify speakers"** checkbox (`renderDiarizeToggle` in [src/ui/modal.ts](src/ui/modal.ts), shown only when the active provider supports diarization, defaulting to the active template's flag and reset on template change). The pipeline ([src/pipeline.ts](src/pipeline.ts) `collectTranscript`) computes `effectiveDiarize = (template.diarize || params.diarize) && transcriptionProviderSupportsDiarization(id)` and always sets `transcriptionConfig.diarize` explicitly (so a stale value from an older `data.json` can't leak through). `PipelineParams.diarize` carries the modal toggle; it is threaded through the Record tab, `startRecordingPipeline`, and the "Record in background" handoff (`startBackgroundRecording` -> `startQuickRecord` -> `QuickRecordController`), and cleared when the Quick Record floater switches templates. There is **no** settings-tab diarization toggle and `TranscriptionConfig.diarize` is now purely the internal pipeline->adapter transport. Capability is gated by `transcriptionProviderSupportsDiarization(id)` ([src/transcription/index.ts](src/transcription/index.ts)); on a non-capable provider the flag is a documented no-op. Labels survive cleanup via a `DEFAULT_SHARED_CORE` clause. Full detail (per-adapter formatting, pipeline merge rules, label survival) lives in [docs/DIARIZATION.md](docs/DIARIZATION.md). ## Local whisper.cpp host (desktop) -`WhisperHost` ([src/whisper-host.ts](src/whisper-host.ts), instantiated in `onload`, stopped in `onunload`) manages a user-supplied whisper-server child process. Config lives at `GlobalSettings.localWhisper = { binaryPath, modelPath, port, extraArgs }`: no auto-discovery, no auto-download, no baked defaults. Desktop-only (Node modules lazy-required behind `Platform.isDesktop`; the option is filtered out of mobile dropdowns). **Loopback binding is enforced** (a non-loopback `--host` throws before spawn) because whisper-server has no auth/TLS; `getHostArgs` collects EVERY `--host` occurrence in `extraArgs` (not just the first), since whisper-server honors the last one and checking only the first let a duplicate `--host 127.0.0.1 --host 0.0.0.0` bypass the guard. The `whisper-local` transcription provider ([src/transcription/whisper-local.ts](src/transcription/whisper-local.ts)) is a thin shim POSTing transcoded 16 kHz mono WAV to `http://127.0.0.1:/inference`; no API key. +`WhisperHost` ([src/whisper-host.ts](src/whisper-host.ts), instantiated in `onload`, stopped in `onunload`) manages a user-supplied whisper-server child process. Config lives at `GlobalSettings.localWhisper = { binaryPath, modelPath, port, extraArgs, autoStart, idleStopMinutes }`: no auto-discovery, no auto-download, no baked defaults. Desktop-only (Node modules lazy-required behind `Platform.isDesktop`; the option is filtered out of mobile dropdowns). **Loopback binding is enforced** (a non-loopback `--host` throws before spawn) because whisper-server has no auth/TLS; `getHostArgs` collects EVERY `--host` occurrence in `extraArgs` (not just the first), since whisper-server honors the last one and checking only the first let a duplicate `--host 127.0.0.1 --host 0.0.0.0` bypass the guard. The `whisper-local` transcription provider ([src/transcription/whisper-local.ts](src/transcription/whisper-local.ts)) is a thin shim POSTing transcoded 16 kHz mono WAV to `http://127.0.0.1:/inference`; no API key. -Full detail (start/stop/probe lifecycle, the spawned/adopted/external ownership model, PID sidecar, Auto-detect, the build script, and all whisper-host gotchas) lives in [docs/WHISPER_HOST.md](docs/WHISPER_HOST.md). +Phase B lifecycle (both knobs default off): `autoStart` starts the server after `onload`'s probe settles (only when the active profile uses `whisper-local` and the probe found nothing to adopt); `idleStopMinutes > 0` stops a ReWrite-owned (spawned/adopted, NEVER external) server after that many minutes without a transcription, via a 60 s interval calling `whisperHost.stopIfIdle()`. The decision logic is the pure `shouldStopWhenIdle(...)` (unit-tested); the `whisper-local` adapter brackets each transcription with `beginUse()`/`endUse()` so an in-flight job blocks the stop and resets the idle clock. + +Full detail (start/stop/probe lifecycle, the spawned/adopted/external ownership model, PID sidecar, Auto-detect, the build script, the Phase B auto-start/idle-stop wiring, and all whisper-host gotchas) lives in [docs/WHISPER_HOST.md](docs/WHISPER_HOST.md). ## Settings `GlobalSettings` (defined in [src/types.ts](src/types.ts)) is the shape of `data.json`. Loading flow: 1. `plugin.loadData()` returns `Partial | null`. -2. `mergeSettings(DEFAULT_SETTINGS, stored)` deep-merges, preferring stored values. Enum fields (`recordingFormat`, `activeProfileOverride`, `newFileCollisionMode`, `transcriptionProvider`, `llmProvider`) are validated against an allowlist and fall back to the base value on an unrecognized string; nested config objects (`localWhisper`, `modelCache`, `transcriptionConfig`, `llmConfig`) are only spread when they're actually plain objects. Both guard against a corrupt/hand-edited `data.json` leaking garbage into runtime state (a non-object partial field would otherwise spread its characters into the nested config). -3. `hydrateSecrets()` reads keys from `secrets.json.nosync` and writes them into each profile's `transcriptionConfig.apiKey` / `llmConfig.apiKey`. +2. `mergeSettings(DEFAULT_SETTINGS, stored)` deep-merges, preferring stored values. Enum fields (`recordingFormat`, `activeProfileOverride`, `newFileCollisionMode`, `transcriptionProvider`, `llmProvider`) are validated against an allowlist and fall back to the base value on an unrecognized string; nested config objects (`localWhisper`, `modelCache`, `transcriptionConfig`, `llmConfig`) are only spread when they're actually plain objects. The two array fields (`disabledDefaultTemplateIds`, `ingestRules`) go through `sanitizeDisabledIds` / `sanitizeIngestRules` (exported, tested): a non-array falls back to base, and malformed elements are dropped (a rule needs a non-blank `folderPath` and `templateId`; `enabled` is coerced to a strict boolean). All of this guards against a corrupt/hand-edited `data.json` leaking garbage into runtime state (a non-object partial field would otherwise spread its characters into the nested config). +3. `hydrateSecrets()` reads keys from `secrets.json.nosync` and writes them into each profile's `transcriptionConfig.apiKey` / `llmConfig.apiKey` / `realtimeConfig.apiKey`. Saving flow strips secrets out of `data.json` and writes them to `secrets.json.nosync` instead (see [src/secrets.ts](src/secrets.ts)). Never persist API keys to `data.json`. @@ -196,9 +209,9 @@ Full detail (envelope schema, KDF params, lock/unlock flow, opportunistic PBKDF2 ## Templates -Templates are Markdown files in a vault folder, not entries in `data.json`. The folder path lives on `GlobalSettings.templatesFolderPath` (default `ReWrite/Templates`). Each `.md` file in the folder is one template: YAML frontmatter holds `id`, `name`, `insertMode`, `newFileFolder`, `newFileNameTemplate`, plus the three optional boolean flags `disableSharedCore`, `enableContextHint`, and `diarize`; the file body is the LLM prompt. Files are sorted by basename in the modal/picker so users can prefix names (`01-...`, `02-...`) to control order. +Templates are Markdown files in a vault folder, not entries in `data.json`. The folder path lives on `GlobalSettings.templatesFolderPath` (default `ReWrite/Templates`). Each `.md` file in the folder is one template: YAML frontmatter holds `id`, `name`, `insertMode`, `newFileFolder`, `newFileNameTemplate`, the optional boolean flags `disableSharedCore`, `enableContextHint`, `diarize`, and `titleFromContent`, plus the tri-state `managed` ownership marker (see Managing individual defaults below); the file body is the LLM prompt. Files are sorted by basename in the modal/picker so users can prefix names (`01-...`, `02-...`) to control order. -[src/templates-folder.ts](src/templates-folder.ts) exports `loadTemplatesFromFolder(app, folderPath)`, `populateDefaultTemplates(app, folderPath)`, and `updateDefaultTemplates(app, folderPath)` (see Updating defaults below). The plugin keeps a cache on `plugin.templates: NoteTemplate[]`, refreshed in [src/main.ts](src/main.ts) on: +[src/templates-folder.ts](src/templates-folder.ts) exports `loadTemplatesFromFolder(app, folderPath)`, `populateDefaultTemplates(app, folderPath, disabledIds?)`, and `updateDefaultTemplates(app, folderPath, disabledIds?)` (see Updating defaults below; `disabledIds` = `new Set(settings.disabledDefaultTemplateIds)` at both call sites). The plugin keeps a cache on `plugin.templates: NoteTemplate[]`, refreshed in [src/main.ts](src/main.ts) on: - `workspace.onLayoutReady` after `onload` (initial load, after the vault is ready) - vault `create` / `modify` / `delete` events scoped to the templates folder via `isPathInTemplatesFolder` - vault `rename` (checks both old and new path) @@ -213,6 +226,13 @@ The 10 defaults ([src/settings/default-templates.ts](src/settings/default-templa `NoteTemplate.disableSharedCore?: boolean` (frontmatter `disableSharedCore: true`) opts a single template out of the shared-core prepend. `renderTemplateFile` ALWAYS emits the key so the knob is discoverable: empty (`disableSharedCore:`, parses to null = not disabled) by default, `disableSharedCore: true` when set. `parseTemplateFile` treats it as disabled only when the value is boolean `true` or the string `"true"` (case-insensitive) — the string form is tolerated because Obsidian's Properties UI may store an edited value as text; any other value (null/empty/false) means not disabled. The three opt-in boolean flags `enableContextHint` (see Context hint), `diarize` (see Speaker diarization), and `titleFromContent` (see Note title) follow the exact same render/parse convention (always-emitted empty key, boolean-or-`"true"` tolerance); their polarity is positive (set to turn ON), the reverse of `disableSharedCore`. +### Managing individual defaults (managed flag + disable list) + +Two per-default controls, surfaced as the "Manage built-in templates" `
` checklist in the Templates settings section (`renderManageDefaults` in [src/settings/tab.ts](src/settings/tab.ts), expand state on `manageDefaultsExpanded`; the list is driven off `freshDefaultTemplates()` so it tracks the default set). The two controls are deliberately different affordances so their roles read without a hover: **Enabled** is a toggle switch with an inline caption that reads "Enabled" when on and "Disabled" when off (the whole tab re-renders on toggle, so the caption reflects the current state); **Tracked** is a real labelled checkbox (a manual `` appended to the row's `controlEl`, with its caption span placed BEFORE the input so the label reads left-to-right; the caption reads "Tracked" when checked and "Untracked" when unchecked, mirroring the Enabled/Disabled switch; not a second `addToggle`, which would look identical to the Enabled switch). A legend above the list bolds both terms, and the `.rewrite-manage-*` CSS lays the row out: + +- **Enabled / disabled (existence).** Disabling removes the on-disk file (matched by frontmatter `id` via `findTemplateFileById`, deleted with `app.fileManager.trashFile` behind a `ConfirmModal`) and records the id in `GlobalSettings.disabledDefaultTemplateIds`, which Populate and Update both receive as `disabledIds`: neither ever (re)creates a disabled default, including a future plugin version re-shipping it. Decision (documented): the ignore list is keyed by `id` only, NOT also by name — the `managed` flag already prevents a name-colliding user template from being touched, so name suppression would add ambiguity without protection. Re-enabling removes the id from the list and immediately recreates the file via `restoreDefaultTemplate` (same name/path-collision skip as Populate). The `ConfirmModal` gained an optional `onCancel` hook ([src/ui/confirm-modal.ts](src/ui/confirm-modal.ts), fired from `onClose` when not confirmed) so the settings tab can re-render and reset the optimistically-flipped toggle when the user backs out. +- **Tracked / untracked (updates), via `NoteTemplate.managed`.** `managed` is TRI-state, unlike the other flags: `true` (plugin-managed; Populate/Update stamp it on every file they create or reconcile), `false` (untracked: Update skips the file entirely, counted in `UpdateResult.untracked`), `undefined` (key absent/empty — treated as tracked when the id matches a built-in, preserving pre-flag behavior). `parseTemplateContent` uses `parseTriStateFlag` (boolean or string `"true"`/`"false"`); `renderTemplateFile` emits `managed: true` / `managed: false` / the empty `managed:` stub for undefined. Because Update's file-matching is now "id matches a built-in AND the file is not `managed: false`", and because only Populate/Update ever write `managed: true`, a user's own hand-authored template that coincidentally shares a name with a built-in is never clobbered. The untrack toggle edits just the frontmatter key via `app.fileManager.processFrontMatter` (preserves the body byte-for-byte) rather than re-rendering the whole file. + The Templates "Populate" button also seeds SharedCore.md when missing (non-destructive), because the shared core is load-bearing for the default templates' quality (it carries their guardrail + output discipline). Populating templates without it would yield prompts with no guardrail. (Populate no longer seeds an in-vault help file; the user-facing template guide lives in the wiki.) **Keep `wiki/Creating-Templates.md` in sync with the template structure.** Whenever you add or change a `NoteTemplate` field / frontmatter property, change the prompt-assembly order, alter what the shared core carries, or change the default-template set or count, update that wiki page in the SAME change: add the property to its frontmatter table AND give the behavior its own `##` section. The settings tab links users to it from the Templates section ("Creating templates"); it is the user-facing contract for the template format, so treat updating it as part of the task, not a follow-up. ### Updating defaults @@ -221,9 +241,9 @@ The Templates section ships three buttons sharing one `Setting` row: **Populate* **Default history.** [src/settings/template-history.ts](src/settings/template-history.ts) holds `TEMPLATE_HISTORY: Record` — the plugin's in-code memory of what each default *used to be*, keyed by template id, each snapshot carrying `{ version, template }`. It is the base for the merge and the source for Load prior versions. **It starts EMPTY.** MAINTENANCE RULE (treat like the keep-the-guide-in-sync rule): whenever you change a default in [src/settings/default-templates.ts](src/settings/default-templates.ts), append the OUTGOING template to `TEMPLATE_HISTORY[id]` under the manifest version it shipped in. Without the snapshot, Update cannot tell an unedited old prompt from a user edit and falls back to a body conflict. `priorVersionsForId(id)` and `allPriorVersions()` return clones so callers can't mutate the registry. -**Per-field 3-way merge.** `mergeTemplate(onDisk, def, priors)` (pure, synchronous; `priors` = `priorVersionsForId(id)`) treats a field as *pristine* (user never touched it) when its on-disk value equals the current default OR any prior shipped default; a pristine value is brought forward to the current default, a genuine edit is kept. The **prompt body** is the only field whose kept edit becomes a report `body` conflict — an unedited old body is silently updated (recorded in `changes`). Scalars + flags adopt-or-keep silently (a real adoption is noted in `changes`). `noteProperties` union by name: missing defaults appended; an instruction matching a prior default is brought forward; a genuinely edited instruction is kept + `changedInstruction`; a property the default dropped is always kept + `removedProperty` (with `wasShippedDefault` set when the user's value matches a prior default, so the report can say it is safe to delete). Because `renderTemplateFile` always emits the four flag stubs and emits `noteProperties` only when non-empty, re-rendering a pristine current default is byte-identical to Populate's output, so an already-current folder is a clean no-op. +**Per-field 3-way merge.** `mergeTemplate(onDisk, def, priors)` (pure, synchronous; `priors` = `priorVersionsForId(id)`) treats a field as *pristine* (user never touched it) when its on-disk value equals the current default OR any prior shipped default; a pristine value is brought forward to the current default, a genuine edit is kept. The **prompt body** is the only field whose kept edit becomes a report `body` conflict — an unedited old body is silently updated (recorded in `changes`). Scalars + flags adopt-or-keep silently (a real adoption is noted in `changes`). `noteProperties` union by name: missing defaults appended; an instruction matching a prior default is brought forward; a genuinely edited instruction is kept + `changedInstruction`; a property the default dropped is always kept + `removedProperty` (with `wasShippedDefault` set when the user's value matches a prior default, so the report can say it is safe to delete). `mergeTemplate` stamps `managed: true` on its result (it only runs on tracked default-derived files, so this also back-fills the flag onto pre-flag files, at the cost of one rewrite). Because `renderTemplateFile` always emits the flag stubs and emits `noteProperties` only when non-empty, re-rendering a pristine current default is byte-identical to Populate's output, so an already-current folder is a clean no-op. -`updateDefaultTemplates` walks the folder, matches files to defaults by frontmatter `id` (via `readTemplateId`, also now backing `collectExistingIds`; non-default-derived files skipped), merges, and `vault.modify`s in place only when the render differs (CRLF-normalized compare; never renames, even with an ordering prefix). It then **recreates any default missing from disk entirely** (superset top-up, same name + path-collision skip as Populate) but does NOT seed SharedCore.md (Populate's job). Counters are mutually exclusive: `created` + `updated` + `unchanged` + `conflicts` + `parseFailed` = files seen. A default-derived file that fails to parse is left untouched and reported as `parseFailed`. +`updateDefaultTemplates` walks the folder, matches files to defaults by frontmatter `id` (via `readTemplateId`, also now backing `collectExistingIds`; non-default-derived files skipped, disabled ids skipped, `managed: false` files skipped as `untracked`), merges, and `vault.modify`s in place only when the render differs (CRLF-normalized compare; never renames, even with an ordering prefix). It then **recreates any default missing from disk entirely** (superset top-up, same name + path-collision skip as Populate, disabled ids excluded) but does NOT seed SharedCore.md (Populate's job). Counters are mutually exclusive: `created` + `updated` + `unchanged` + `conflicts` + `parseFailed` + `untracked` = files seen. A default-derived file that fails to parse is left untouched and reported as `parseFailed`; an untracked (`managed: false`) file is counted only in the `untracked` tally and gets no per-template entry in `Template update report.md` (freezing it is the user's explicit choice, not a conflict). **Load prior versions** writes each `allPriorVersions()` snapshot as a standalone template with a distinct id (`@`) and versioned name (` `), so it appears in the picker for comparison, never collides with the live template's identity, and is left untouched by Update (its id is not a current default) and Populate. Non-destructive (skips by id/path); reports `available` so the button can say "no prior versions yet" when the registry is empty. @@ -276,6 +296,29 @@ Two halves, deliberately decoupled: UI surfaces, both a collapsed `
` so the frictionless path is untouched: the main modal ([src/ui/modal.ts](src/ui/modal.ts) `renderContextSelector`, shown only when the active template has the flag; `contextHint` / `contextExpanded` instance state reset on template change, survives re-renders like `destinationExpanded`) and the reprocess-audio picker ([src/ui/template-picker.ts](src/ui/template-picker.ts), via `showContext` which [src/main.ts](src/main.ts) sets to `templates.some(t => t.enableContextHint)`; the typed value is forwarded to `runAudioFilePipeline` only when the picked template has the flag). Quick Record and the process-text command intentionally skip it. +## Real-time transcription + +Opt-in live dictation, deliberately minimal: it streams the mic to the provider over a WebSocket and types finalized segments at the editor cursor as they arrive. No template, no LLM cleanup, no audio persistence, no main-modal entry — just the `rewrite-voice-notes:realtime-transcribe` command (see Commands). Lower-value than the rest of the pipeline (a raw transcript with no cleanup is what Obsidian's existing voice plugins already do), shipped because the primitives were cheap once the streaming adapters existed. + +Provider support is gated by `transcriptionProviderSupportsRealtime(id)` ([src/realtime/index.ts](src/realtime/index.ts)): only `deepgram` and `assemblyai` expose a browser-reachable streaming WebSocket. Every other provider (OpenAI/Groq whisper-shape, Rev.ai async, whisper.cpp server) accepts only whole-file uploads, so the command errors with a "use AssemblyAI or Deepgram" Notice on them. **Voxtral realtime was tried and pulled** (see the Voxtral bullet below): its adapter is kept on disk but is deliberately NOT in the gate or the factory, so it is neither bundled nor selectable. + +**Realtime is configured ENTIRELY independently of batch transcription** — its own provider, key, and model — so a profile can run e.g. Voxtral for batch, AssemblyAI for realtime, and Anthropic for cleanup. `EnvironmentProfile.realtimeProvider` (a `TranscriptionProviderID`; `'none'` = off, else a realtime-capable id) plus `realtimeConfig` (a `TranscriptionConfig` shape; only `apiKey`/`model` used). Reason: streaming providers/models/keys differ from batch. The key is persisted under secret id `profile-{kind}-realtime` (see Provider system / docs/SECRETS.md), hydrated/stripped/merged alongside the transcription and LLM slots; `realtimeProvider` is validated by `pickEnum` in `mergeProfile`. The settings tab renders an independent "Real-time transcription" section — a **Real-time provider** dropdown (None + realtime-capable only) that on change re-renders, plus key + model when a provider is picked (`renderRealtimeSection` in [src/settings/tab.ts](src/settings/tab.ts)). The model field is the same adaptive control as the batch fields (`populateModelField(..., 'realtime')`: dropdown + Refresh where the provider supports `listModels`, e.g. Deepgram; a plain text field otherwise, e.g. AssemblyAI), reading `profile.realtimeProvider` / `realtimeConfig` and sharing the transcription model cache. `startRealtimeTranscription` and `RealtimeController.begin()` gate on `realtimeProvider` (not `transcriptionProvider`) and `realtimeConfig.apiKey`. + +- **Capture** ([src/realtime/pcm.ts](src/realtime/pcm.ts)): `PcmCapture` taps the mic with a `ScriptProcessorNode` (deprecated but universally available; `AudioWorklet` would need a separate module file a single-bundle plugin can't ship under the app CSP — the deprecation is `eslint-disable`d with that rationale inline), downsamples the context rate to 16 kHz via the pure `downsampleBuffer` (linear interpolation), quantizes to signed-16 PCM via the pure `floatTo16BitPcm`, and emits `ArrayBuffer` chunks. Both helpers and `REALTIME_SAMPLE_RATE` are exported and unit-tested. The node is routed through a zero-gain node to `destination` (a ScriptProcessor only runs while connected) so nothing is audible. +- **Session interface** ([src/realtime/index.ts](src/realtime/index.ts)): `RealtimeProvider.start(config, sampleRate, callbacks) -> RealtimeSession`, where `RealtimeSession` is `{ sendAudio(chunk), stop() }` and callbacks are `onFinal` / `onInterim` / `onError` / `onUnexpectedClose`. `waitForOpen` / `waitForClose` are shared WS plumbing. +- **Deepgram** ([src/realtime/deepgram.ts](src/realtime/deepgram.ts)): auth rides the `['token', ]` WebSocket subprotocol (browser WebSockets can't set headers), so the key never touches the URL. `is_final` marks a finalized segment; the profile's batch model (nova family) generally works live, empty falls to Deepgram's default. `stop()` sends `{ type: 'CloseStream' }` to flush. +- **AssemblyAI** ([src/realtime/assemblyai.ts](src/realtime/assemblyai.ts)): browser WS can't send the API key as a header and AssemblyAI has no subprotocol auth, so the real key first buys a short-lived (60 s) single-use token over a normal header-authed `jsonGet`, and only that ephemeral token rides the WS URL query. **This is the one sanctioned exception to the "auth never goes in the query" rule** (documented in the HTTP gotcha too): the token expires fast, can't mint more, and no error path echoes the URL. `format_turns: true` re-sends each turn once formatted; only the formatted end-of-turn is treated as final so a turn isn't inserted twice. `stop()` sends `{ type: 'Terminate' }`. +- **Voxtral** ([src/realtime/voxtral.ts](src/realtime/voxtral.ts)): **UNWIRED as of 1.2.0.** The adapter exists (protocol reverse-engineered from the mistralai Python SDK `src/mistralai/extra/realtime/`: `wss://api.mistral.ai/v1/audio/transcriptions/realtime?model=`, `session.update` then base64 `input_audio.append` split at the 262144-decoded-byte cap, `flush`/`end` on stop, `transcription.text.delta`/`transcription.done` back) but live testing confirmed the AUTH CAVEAT: the browser WebSocket can only send the key via the `['token', ]` subprotocol (no `Authorization: Bearer` header from a WebView), and Mistral's realtime endpoint rejects that handshake, so Voxtral realtime is not WebView-reachable. The file is kept on disk, documented, as a starting point for a contributor who finds a working browser-auth path; it is NOT referenced by `createRealtimeProvider` or `transcriptionProviderSupportsRealtime`, so it is not bundled or selectable. To re-enable, wire it back into both. See `wiki/Voxtral-Realtime.md`. +- **Controller** ([src/ui/realtime.ts](src/ui/realtime.ts)): `RealtimeController` owns the capture + session + a floating status bar (reuses the `.rewrite-quick-floater` shell + `.rewrite-realtime-interim` line). `insertFinal` uses `editor.replaceSelection(text + ' ')` so segments chain like typing. `finish()` is a graceful stop (end capture, let trailing finals flush, dispose); `cancel()` is a hard teardown for errors/unload. One session at a time, owned by the plugin via `activeRealtime` + `realtimeStarting` (mirrors `activeQuickRecord`), cancelled in `onunload`. `startRealtimeTranscription` gates on unlocked secrets, a realtime-capable provider, a key, capture availability, and an open Markdown editor (the transcript needs a cursor). + +## Auto-ingest folders + +Semi-automatic batch ingestion for audio captured outside Obsidian and dropped into a vault folder. A user-run command (NOT a live watcher — a `vault.on('create')` watcher would fire for every existing file on vault load, a "startup storm"), so the user controls when a batch runs. Config is `GlobalSettings.ingestRules: { folderPath, templateId, enabled }[]`. + +[src/ingest.ts](src/ingest.ts) `runIngestBatch(plugin)` gates on enabled rules existing, unlocked secrets, and a fully-configured voice profile (same checks as manual reprocess), then `collectWork` (async) resolves each enabled rule's folder + template. A rule is skipped (with a Notice) when its template is missing or is not `insertMode: 'newFile'` — `isIngestTemplate` (exported, tested) enforces newFile-only because unattended ingest has no active editor, so `cursor`/`append` would silently cascade to `newFile` anyway; requiring it makes the destination explicit. A rule is ALSO skipped up front when its folder is where recordings would be stored (see the same-folder guard below). `collectIngestFiles` (exported, tested) takes only direct `TFile` children filtered to `AUDIO_EXTENSIONS`, sorted by name (no recursion, so organized subfolders aren't swept up); a path seen by two rules is queued once. The batch runs **serialized** (one pipeline at a time) behind a sticky `Notice` with a Cancel button (`AbortController`), reporting a per-batch "N processed, M failed" summary. + +Each file runs the pipeline with `source: { kind: 'audio', audio, sourcePath: file.path }` (skips `persist-audio`, reuses the path for the `![[embed]]` prepend, exactly like reprocess). **Move-on-success is the dedupe** (and the requested last step): after a fully successful run, `moveIngestedFile` moves the recording to the attachments location via `app.fileManager.renameFile` (which rewrites the just-created note's embed to the new path automatically), so a re-run never reprocesses it — no processed-path bookkeeping in `data.json`. A file that fails stays in the ingest folder and is retried next run (failures don't move). **Same-folder guard, two layers:** the move would be a pointless in-place de-collision rename (and the file would be re-ingested every run) whenever the ingest folder IS the attachments/recordings location. `collectWork` catches this UP FRONT, before creating any note: it resolves where recordings would land for the rule's template via the side-effect-free `resolveAttachmentFolder` ([src/audio-persist.ts](src/audio-persist.ts), the folder-only sibling of `resolveAttachmentPath` that does NOT create folders) and skips the rule with a Notice when that folder equals the rule folder (`normalizeIngestFolder`, exported/tested, normalizes root + trailing-slash variants before comparing). `moveIngestedFile` keeps the original per-file backstop (`ingestTargetIsSameFolder`) for the case only knowable at move time (empty attachments setting + note-relative Obsidian setting), throwing a reported error rather than churning the file. Ships on all platforms including mobile-only (a user-run file-move command, no cross-device double-processing hazard, so no `Platform.isDesktop` gate). Rule creation/editing is a popup ([src/ui/ingest-rule-modal.ts](src/ui/ingest-rule-modal.ts), template dropdown filtered to newFile templates); the settings "Auto-ingest folders" section lists rules with enable toggle / Edit / Delete plus an Add button. `resolveAttachmentPath` was extracted from `persistAudio` in [src/audio-persist.ts](src/audio-persist.ts) so the mover resolves the same destination a live recording would. + ## Commands Registered in [src/main.ts](src/main.ts): @@ -285,6 +328,8 @@ Registered in [src/main.ts](src/main.ts): - **`rewrite-voice-notes:quick-record-fixed`** ("Quick record (set template)"): same flow, but records with the template chosen in `GlobalSettings.quickRecordTemplateId` (Settings dropdown beside Default template). If that id is unset or no longer resolves, it shows a "choose a Quick record template in settings" `Notice` and does not start (no `templates[0]` fallback, unlike the last-used command). Both commands share `toggleQuickRecord(opts?)` and the single `activeQuickRecord` ref, so either one stops an in-flight recording. The floating UI shows a stop-hotkey hint ("Press or click Stop") read live from the command's binding; see the Gotcha on `hotkeyManager`. - **`rewrite-voice-notes:process-text`** ("Process text with template"): runs a template over the active editor's selection (or the whole note body if there's no selection). Opens a template quick-picker, then runs the pipeline in the background with progress shown via `Notice`. Gates on LLM-only configuration; opens the main modal's setup card when not configured. Bails with a Notice when no Markdown editor is active. - **`rewrite-voice-notes:reprocess-audio`** ("Reprocess audio file with template"): reruns the pipeline over an audio file already in the vault. Opens an `AudioFilePickerModal` (`FuzzySuggestModal` filtered to `AUDIO_EXTENSIONS` from [src/audio-persist.ts](src/audio-persist.ts)) then the template quick-picker, then calls `runAudioFilePipeline` in [src/ui/audio-source.ts](src/ui/audio-source.ts). The pipeline skips its `persist-audio` stage because the `audio` source variant carries a `sourcePath` (the existing vault path is reused for the `![[]]\n\n` prepend). Gates on the full voice profile (`isProfileConfigured`). +- **`rewrite-voice-notes:process-ingest-folders`** ("Process auto-ingest folders"): runs the auto-ingest batch (`runIngestBatch` in [src/ingest.ts](src/ingest.ts)). See Auto-ingest folders above. Plain `callback` (available everywhere, including mobile-only vaults). +- **`rewrite-voice-notes:realtime-transcribe`** ("Real-time transcription (start/stop)"): toggles live dictation (`toggleRealtime` → `startRealtimeTranscription` in [src/ui/realtime.ts](src/ui/realtime.ts)). Second press (or the floating bar's Stop) ends the session. Routed through the single `activeRealtime` slot + `realtimeStarting` guard. See Real-time transcription above. - **`rewrite-voice-notes:start-whisper-host`** / **`rewrite-voice-notes:stop-whisper-host`**: start or stop the local whisper.cpp server. Both use `checkCallback` so the palette only shows them on desktop, when the active profile's transcription provider is `whisper-local` (start) or when the host is currently `running` / `starting` (stop). Errors surface via `Notice`. Same code paths as the settings-tab Start/Stop button. Plus an editor-menu item "ReWrite with template..." registered via `workspace.on('editor-menu', ...)` (and a second "Reprocess audio with template..." item that appears only when the cursor sits inside an `![[