diff --git a/CLAUDE.md b/CLAUDE.md index 4e3c0ec..5a0d1b7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -52,16 +52,22 @@ src/ ├── pipeline.ts # transcribe → cleanup → insert orchestrator ├── insert.ts # cursor/newFile/append + {{date}}/{{time}} expansion ├── whisper-host.ts # Spawns/stops a user-supplied whisper-server child process (desktop only) +├── templates-folder.ts # Load templates from a vault folder + populate it with the 5 defaults +├── assistant-prompt.ts # Load the ad-hoc-instructions assistant prompt from a vault Markdown file + populate default +├── known-nouns.ts # Load a vault Markdown file of known nouns + populate default + build the system-prompt section +├── audio-persist.ts # Write the recorded Blob to an attachments folder, return vault-relative path +├── wake-name.ts # Extract ", " instructions from a transcript ├── settings/ │ ├── index.ts # DEFAULT_SETTINGS, load/save, per-profile secret hydration -│ ├── tab.ts # PluginSettingTab: active profile, two profile sections, templates, recording -│ └── default-templates.ts # The 5 seeded templates (General cleanup, Todo list, etc.) +│ ├── tab.ts # PluginSettingTab: active profile, two profile sections, templates folder, recording +│ └── default-templates.ts # The 5 default templates used by the populate button (General cleanup, Todo list, etc.) ├── ui/ │ ├── modal.ts # Main modal: template select + Record/Paste/From note tabs + setup-card injection │ ├── setup-card.ts # Inline blocker when active profile is unconfigured (voice vs text purpose) │ ├── quick-record.ts # QuickRecordController + floating mini-UI for the Quick Record command │ ├── template-picker.ts # Lightweight modal for picking a template (used by Process text command and editor menu) -│ └── text-source.ts # resolveActiveTextSource + runTextPipeline helpers for text-source flows +│ ├── text-source.ts # resolveActiveTextSource + runTextPipeline helpers for text-source flows +│ └── whisper-status-bar.ts # Status-bar dot for whisper-host start/stop (desktop + whisper-local profile only) ├── transcription/ │ ├── index.ts # TranscriptionProvider interface + createTranscriptionProvider() │ ├── openai.ts # Whisper-shape POST (also used by openai-compatible + groq) @@ -79,15 +85,16 @@ src/ ## Pipeline -[src/pipeline.ts](src/pipeline.ts) runs three stages with `onStage` callbacks for UI: +[src/pipeline.ts](src/pipeline.ts) runs four stages with `onStage` callbacks for UI: -1. **Transcribe**: `audio` → `createTranscriptionProvider(profile.transcriptionProvider).transcribe(blob, config)`. Skipped when the source is `paste` or `text` (input passes through). Short-circuited when the source is `webspeech` (the transcript was already captured live by `src/webspeech.ts` during recording). -2. **Cleanup**: `createLLMProvider(profile.llmProvider).complete(template.prompt, transcript, config)`. On error, the raw transcript is copied to the clipboard before re-throwing, so the user keeps their words. -3. **Insert**: `src/insert.ts` routes to `cursor` / `newFile` / `append` per the template. `cursor` falls back to `append` when no editor is active; `append` falls back to `newFile` when no markdown file exists. `{{date}}` / `{{time}}` in filename templates expand via Obsidian's `moment`. +1. **Persist audio** (`audio` source only): writes the raw `Blob` to the vault via [src/audio-persist.ts](src/audio-persist.ts) before transcription, so the user keeps the recording even if later stages fail. Path resolution: when `settings.attachmentsFolderPath` is set, the file goes under that folder with manual de-collision (`-1`, `-2`, ...); when empty, the path comes from `app.fileManager.getAvailablePathForAttachment(filename)`, which respects Obsidian's own attachments setting. Filename is `ReWrite-YYYY-MM-DD-HHmmss.` with the extension derived from the blob's mime type (`webm` / `m4a` / `ogg` / `wav` / `mp3`, default `webm`). Failure is non-fatal: a Notice fires and transcription proceeds. The resolved path is later prepended to the cleaned output as `![[]]\n\n` before insertion. +2. **Transcribe**: `audio` → `createTranscriptionProvider(profile.transcriptionProvider).transcribe(blob, config)`. Skipped when the source is `paste` or `text` (input passes through). Short-circuited when the source is `webspeech` (the transcript was already captured live by `src/webspeech.ts` during recording). +3. **Cleanup**: `createLLMProvider(profile.llmProvider).complete(systemPrompt, transcript, config)`. The system prompt is the template prompt, optionally augmented with an `## Ad-hoc instructions` block when the wake-name scan ([src/wake-name.ts](src/wake-name.ts)) extracts directives from the transcript, and a `## Known nouns` block when `plugin.knownNouns` is non-empty (see Assistant prompt and Known nouns sections below). On error, the (possibly stripped) transcript is copied to the clipboard before re-throwing, so the user keeps their words. +4. **Insert**: `src/insert.ts` routes to `cursor` / `newFile` / `append` per the template. `cursor` falls back to `append` when no editor is active; `append` falls back to `newFile` when no markdown file exists. `{{date}}` / `{{time}}` in filename templates expand via Obsidian's `moment`. The modal's per-invocation Destination control overrides `insertMode` / `newFileFolder` / `newFileNameTemplate` via `PipelineParams.destinationOverride`; the override is shallow-merged onto a copy of the template before the insert call, so the template file on disk is never mutated. -The pipeline accepts an `AbortSignal` (forwarded to providers) and is consumed by [src/ui/modal.ts](src/ui/modal.ts), [src/ui/quick-record.ts](src/ui/quick-record.ts), and [src/ui/text-source.ts](src/ui/text-source.ts) (the `runTextPipeline` helper for command + editor-menu entry points). +The pipeline accepts an `AbortSignal` (forwarded to providers) and is consumed by [src/ui/modal.ts](src/ui/modal.ts), [src/ui/quick-record.ts](src/ui/quick-record.ts), and [src/ui/text-source.ts](src/ui/text-source.ts) (the `runTextPipeline` helper for command + editor-menu entry points). Every caller passes the plugin itself as `host: PipelineHost`; `PipelineHost` is a narrow interface ({ `assistantPrompt`, `knownNouns` }) so `cleanupTranscript` can read the loaded vault content without importing `ReWritePlugin` (which would form a circular dep through the UI layer). -The `PipelineSource` union has four variants: `audio` (recorded blob), `paste` (textarea input), `webspeech` (live-captured transcript), `text` (input from an existing note via selection or whole body). Text-source flows skip transcription entirely and only require the LLM half of the profile. +The `PipelineSource` union has four variants: `audio` (recorded blob, optional `sourcePath` for reprocess flows), `paste` (textarea input), `webspeech` (live-captured transcript), `text` (input from an existing note via selection or whole body). Text-source flows skip transcription entirely and only require the LLM half of the profile. The `audio` variant's `sourcePath` is set by the reprocess flow ([src/ui/audio-source.ts](src/ui/audio-source.ts)) to point at an existing vault file; when present, the persist stage is skipped and that path is reused for the `![[]]\n\n` prepend. ## Provider system @@ -103,7 +110,7 @@ Providers may optionally implement `listModels(config, signal)` returning a stri `start()` validates the binary and model paths exist via `fs.existsSync`, probes the port via `net.createServer().listen(port)` to detect conflicts, spawns the user's whisper-server with `child_process.spawn`, captures stdout/stderr into a 1 MB ring buffer, then polls `net.createConnection` against the port every 250 ms for up to 5 s before declaring `'running'`. Any failure transitions status to `'crashed'` with the log tail surfaced in the error message. `stop()` sends SIGTERM, waits up to 3 s, then SIGKILL. -The `whisper-local` transcription provider ([src/transcription/whisper-local.ts](src/transcription/whisper-local.ts)) is a thin shim that POSTs to `http://127.0.0.1:/v1/audio/transcriptions` using the same multipart shape as OpenAI Whisper. The shim grabs the host reference via `bindWhisperHost(host)` called from [src/main.ts](src/main.ts) on load. If the host status is anything other than `'running'`, the adapter throws a `ProviderError` with a "start it from settings" message; the pipeline surfaces that as a Notice. No API key is collected for this provider (no auth, no settings field). +The `whisper-local` transcription provider ([src/transcription/whisper-local.ts](src/transcription/whisper-local.ts)) is a thin shim that POSTs to `http://127.0.0.1:/inference` (whisper.cpp's native server route, not OpenAI's `/v1/audio/transcriptions`). Before sending, the recorded blob is transcoded to 16 kHz mono 16-bit PCM WAV via Web Audio API (`AudioContext.decodeAudioData` → `OfflineAudioContext` for resample/downmix → hand-written RIFF header). whisper.cpp's server cannot decode WebM/Opus (the MediaRecorder default) without a custom ffmpeg-enabled build, so the transcode is mandatory — it is *not* applied to any other transcription provider. The shim grabs the host reference via `bindWhisperHost(host)` called from [src/main.ts](src/main.ts) on load. If the host status is anything other than `'running'`, the adapter throws a `ProviderError` with a "start it from settings" message; the pipeline surfaces that as a Notice. No API key is collected for this provider (no auth, no settings field). Mobile compatibility: `WhisperHost` and the `whisper-local` option are guarded everywhere by `Platform.isDesktop`. Node modules (`child_process`, `net`, `fs`) are lazy-required inside the `Platform.isDesktop` branch, mirroring the [src/secrets.ts](src/secrets.ts) `getSafeStorage` pattern; on mobile the host is inert and the provider option is filtered out of dropdowns in [src/settings/tab.ts](src/settings/tab.ts) and [src/ui/setup-card.ts](src/ui/setup-card.ts). @@ -113,11 +120,64 @@ Mobile compatibility: `WhisperHost` and the `whisper-local` option are guarded e 1. `plugin.loadData()` returns `Partial | null`. 2. `mergeSettings(DEFAULT_SETTINGS, stored)` deep-merges, preferring stored values. -3. If `merged.templates.length === 0`, [src/settings/default-templates.ts](src/settings/default-templates.ts) seeds the 5 starter templates and sets `defaultTemplateId` if unset. This handles first launch *and* migrations from any pre-Phase-11 install with an empty templates array. -4. `hydrateSecrets()` reads keys from `secrets.json.nosync` and writes them into each profile's `transcriptionConfig.apiKey` / `llmConfig.apiKey`. +3. `hydrateSecrets()` reads keys from `secrets.json.nosync` and writes them into each profile's `transcriptionConfig.apiKey` / `llmConfig.apiKey`. Saving flow strips secrets out of `data.json` and writes them to `secrets.json.nosync` instead (see [src/secrets.ts](src/secrets.ts)). Never persist API keys to `data.json`. +## Secrets encryption + +[src/secrets.ts](src/secrets.ts) supports three encryption modes for `secrets.json.nosync`, file-wide (not per-key): + +- **`safeStorage`** — Electron's OS keychain (Keychain on macOS, DPAPI on Windows, libsecret/kwallet on Linux). Default when available. Each value is the base64 ciphertext of `safeStorage.encryptString`. Desktop only; chosen automatically on first run when `safeStorage.isEncryptionAvailable()` returns true. Backend name is surfaced via `safeStorage.getSelectedStorageBackend()`. +- **`passphrase`** — WebCrypto AES-GCM-256 with a key derived from a user-supplied passphrase via PBKDF2-SHA256, 600,000 iterations, 16-byte random salt (per-file). Each value is stored as `.` (12-byte random IV per value). A `verifier` field stores an encryption of `VERIFIER_PLAINTEXT` so unlock can validate the passphrase without trying to decrypt user keys. Works on every platform including mobile and Linux-without-keyring. +- **`plaintext`** — no encryption. Only used as the auto-fallback on first run when `safeStorage` isn't available; users must explicitly opt back into it after switching away. + +File envelope (`SECRETS_VERSION = 2`): +```json +{ "version": 2, "mode": "passphrase", + "kdf": { "iterations": 600000, "salt": "" }, + "verifier": ".", + "keys": { "profile:desktop:transcription": ".", ... } } +``` + +The derived AES-GCM key for passphrase mode lives in module-level state (`unlockedKey`); it never touches disk. `lockSecrets()` forgets it. `unlockSecrets(plugin, passphrase)` derives a candidate key and decrypts the `verifier` to check correctness before caching. + +`ReWritePlugin.encryptionStatus` (a snapshot of `{ mode, locked, safeStorageAvailable, safeStorageBackend }`) is loaded on `onload` and refreshed via `plugin.refreshEncryptionStatus()` after every mode change / unlock. UI code reads this synchronously. + +When `mode === 'passphrase'` and not yet unlocked (`encryptionStatus.locked === true`): +- `loadAllKeys` / `loadKey` return empty strings (no error). +- `saveManyKeys` is a no-op (so calls to `saveSettings()` from unrelated UI changes do not clobber the on-disk encrypted values with empties). +- `saveKey` throws. +- All entry points ([src/ui/modal.ts](src/ui/modal.ts), [src/ui/quick-record.ts](src/ui/quick-record.ts), [src/ui/text-source.ts](src/ui/text-source.ts), [src/ui/audio-source.ts](src/ui/audio-source.ts)) check `plugin.encryptionStatus.locked` and call `plugin.promptUnlock()` instead of proceeding. +- The settings tab disables the API key input fields and shows a red "Unlock" banner at the top. + +`changeEncryptionMode(plugin, newMode, newPassphrase?)` decrypts all keys with the current mode, switches the envelope, and re-encrypts them. Requires the current mode to be unlocked (if passphrase). For `passphrase` newMode, `newPassphrase` is required. `changePassphrase(plugin, newPassphrase)` is a thin wrapper that calls `changeEncryptionMode(plugin, 'passphrase', newPassphrase)`. + +## Templates + +Templates are Markdown files in a vault folder, not entries in `data.json`. The folder path lives on `GlobalSettings.templatesFolderPath` (default `ReWrite/Templates`). Each `.md` file in the folder is one template: YAML frontmatter holds `id`, `name`, `insertMode`, `newFileFolder`, `newFileNameTemplate`; the file body is the LLM prompt. Files are sorted by basename in the modal/picker so users can prefix names (`01-...`, `02-...`) to control order. + +[src/templates-folder.ts](src/templates-folder.ts) exports `loadTemplatesFromFolder(app, folderPath)` and `populateDefaultTemplates(app, folderPath)`. The plugin keeps a cache on `plugin.templates: NoteTemplate[]`, refreshed in [src/main.ts](src/main.ts) on: +- `workspace.onLayoutReady` after `onload` (initial load, after the vault is ready) +- vault `create` / `modify` / `delete` events scoped to the templates folder via `isPathInTemplatesFolder` +- vault `rename` (checks both old and new path) +- the Templates folder path field changing in settings +- the populate button completing + +Consumers ([src/main.ts](src/main.ts), [src/ui/modal.ts](src/ui/modal.ts), [src/ui/quick-record.ts](src/ui/quick-record.ts)) read `plugin.templates` directly, never `settings.templates` (there is no such field). The populate button is non-destructive: it skips any default template whose `id` already exists on disk, and skips path collisions. Frontmatter `id` is canonical for identity, so renaming a file does not break the `defaultTemplateId` / `lastUsedTemplateId` reference. The first-launch experience is empty templates plus a setup nudge in the modal; the user clicks Populate to get the defaults. + +## Assistant prompt + +The system-prompt preface inserted above extracted ad-hoc directives lives as a Markdown file in the vault, not a settings textarea. Path: `GlobalSettings.assistantPromptPath` (default `ReWrite/AssistantPrompt.md`). [src/assistant-prompt.ts](src/assistant-prompt.ts) exports `loadAssistantPromptFromFile(app, path)`, `populateDefaultAssistantPrompt(app, path)`, `isPathAssistantPrompt(path, configuredPath)`, and the `DEFAULT_ASSISTANT_PROMPT` constant used as the fallback when the file is missing or empty. The plugin caches the body on `plugin.assistantPrompt: string | null`, refreshed in [src/main.ts](src/main.ts) on the same triggers as templates (`workspace.onLayoutReady`, scoped vault `create`/`modify`/`delete`/`rename`, settings-path change, populate button). The file body is the prompt; frontmatter is currently ignored (the loader tolerates it for future extensions). [src/pipeline.ts](src/pipeline.ts) reads `params.host.assistantPrompt ?? DEFAULT_ASSISTANT_PROMPT` inside `cleanupTranscript`. + +Branding note: the "AI name" / "Agent control prompt" labels in older builds are replaced by "Assistant name" / "Assistant prompt file" everywhere. The wake-name feature itself keeps the term "wake name" (it's the trigger word, not the persona). The setting key is `assistantName`; the persona is "the assistant". + +## Known nouns + +A vault Markdown file of proper nouns the LLM should preserve verbatim. Path: `GlobalSettings.knownNounsPath` (default `ReWrite/KnownNouns.md`). [src/known-nouns.ts](src/known-nouns.ts) exports `loadKnownNounsFromFile`, `populateDefaultKnownNouns`, `isPathKnownNouns`, and `buildKnownNounsSystemPromptSection(nouns)`. File format: YAML frontmatter for human-readable guidance (token-cost warning, format hint), Markdown body with one noun per line, optional `canonical: alt1, alt2` for misheard variants. Lines starting with `#` and blank lines are skipped. Frontmatter is parsed but NOT sent to the LLM in v1; future opt-in is possible but should not become the default. The default file body includes both the guidance frontmatter and two illustrative example nouns. + +Cache: `plugin.knownNouns: KnownNoun[]` (default `[]`), refreshed on the same triggers as the assistant prompt. [src/pipeline.ts](src/pipeline.ts) appends the section returned by `buildKnownNounsSystemPromptSection(host.knownNouns)` to the system prompt when non-empty. Order in the assembled prompt: template prompt, then ad-hoc instructions (if any), then known nouns (if any). + ## Commands Registered in [src/main.ts](src/main.ts): @@ -125,8 +185,10 @@ Registered in [src/main.ts](src/main.ts): - **`rewrite-plugin:open-modal`** ("Open"): opens the main modal with the last-used template selected. - **`rewrite-plugin:quick-record`** ("Quick record"): starts a recording immediately with a floating mini-UI (no modal). Second press toggles to Stop. On unconfigured profile or capture-API unavailability, opens the modal instead. On post-capture pipeline error, opens the modal so the user can retry (LLM-stage failures leave the raw transcript on the clipboard). - **`rewrite-plugin:process-text`** ("Process text with template"): runs a template over the active editor's selection (or the whole note body if there's no selection). Opens a template quick-picker, then runs the pipeline in the background with progress shown via `Notice`. Gates on LLM-only configuration; opens the main modal's setup card when not configured. Bails with a Notice when no Markdown editor is active. +- **`rewrite-plugin:reprocess-audio`** ("Reprocess audio file with template"): reruns the pipeline over an audio file already in the vault. Opens an `AudioFilePickerModal` (`FuzzySuggestModal` filtered to `AUDIO_EXTENSIONS` from [src/audio-persist.ts](src/audio-persist.ts)) then the template quick-picker, then calls `runAudioFilePipeline` in [src/ui/audio-source.ts](src/ui/audio-source.ts). The pipeline skips its `persist-audio` stage because the `audio` source variant carries a `sourcePath` (the existing vault path is reused for the `![[]]\n\n` prepend). Gates on the full voice profile (`isProfileConfigured`). +- **`rewrite-plugin:start-whisper-host`** / **`rewrite-plugin:stop-whisper-host`**: start or stop the local whisper.cpp server. Both use `checkCallback` so the palette only shows them on desktop, when the active profile's transcription provider is `whisper-local` (start) or when the host is currently `running` / `starting` (stop). Errors surface via `Notice`. Same code paths as the settings-tab Start/Stop button. -Plus an editor-menu item "ReWrite with template..." registered via `workspace.on('editor-menu', ...)` and an `addRibbonIcon('mic', 'ReWrite', ...)` that opens the modal. +Plus an editor-menu item "ReWrite with template..." registered via `workspace.on('editor-menu', ...)` (and a second "Reprocess audio with template..." item that appears only when the cursor sits inside an `![[