wiseguru_ReWrite-Voice-Notes/docs/IMPLEMENTATION_PLAN.md

ReWrite (Voice Notes): Implementation Plan

Context

The repository is the unmodified Obsidian sample-plugin scaffold. The real product is described in obsidian-voice-notes-spec.md: an Obsidian plugin that records or accepts pasted speech, transcribes it via a user-configured provider, cleans/structures it via an LLM, and inserts the result per a chosen template. The plugin must run on desktop and mobile, with auto-selected provider profiles.

This plan covers the full v1 implementation from the spec, with the following user-confirmed decisions:

• Display name: "ReWrite (Voice Notes)"; manifest id: rewrite-plugin; commands prefixed ReWrite: ….
• HTTP transport: Obsidian's requestUrl for every provider call (bypasses CORS in renderer and mobile).
• No baked-in model defaults: profiles ship with empty model strings; the modal blocks with an inline setup card until the active profile is configured.
• Scope: full v1 (6 transcription + 5 LLM providers, profiles, secrets, modal, both commands, 5 default templates, settings UI).

The spec was written without referencing CLAUDE.md/AGENTS.md, so several details conflict with the build and Obsidian conventions. Those discrepancies are listed below with the chosen resolution.

---

Spec discrepancies (resolved before coding)

#	Spec says	Issue	Resolution
1	import { safeStorage } from 'electron' at module top	electron is externalized in esbuild.config.mjs; top-level import crashes on mobile load.	Lazy require inside Platform.isDesktop guard. Wrap in try/catch; treat any failure as "encryption unavailable".
2	Buffer.from(value, 'base64') in secrets code	tsconfig.json lib is DOM-only, no Node types.	Use atob/btoa plus Obsidian's arrayBufferToBase64/base64ToArrayBuffer. Stay in browser primitives.
3	Raw POST {baseUrl}/v1/... examples imply fetch	Anthropic blocks browser CORS without a dangerous header; many providers will fail in mobile WebView too.	All adapters call Obsidian requestUrl. Multipart bodies (Whisper, Rev.ai) hand-built as ArrayBuffer with the proper Content-Type: multipart/form-data; boundary=….
4	Default LLM model claude-sonnet-4-20250514	Spec value is stale; user requested no hard default.	Ship profile with model: "". Modal shows an inline setup card if any required field is blank. Settings text field is free-text per spec UI rule, with placeholder hints like claude-sonnet-4-5 / gpt-4o-mini.
5	Record<ProviderID, string> for global keys	openai ID is reused by both transcription and LLM (same key value), but openai-compatible is distinct config (different URL per profile).	Global keys keyed by provider family (openai, anthropic, groq, assemblyai, deepgram, revai, gemini, mistral). openai-compatible has no global key: base URL+key live on the profile only.
6	secrets.json.nosync path voice-notes	id is now rewrite-plugin.	All paths and README sync-exclusion examples use .obsidian/plugins/rewrite-plugin/secrets.json.nosync.
7	Mobile profile default = webspeech	Obsidian iOS uses WKWebView; SpeechRecognition is unavailable there. Spec only mentions a fallback Notice.	Keep webspeech as the mobile default transcription provider (spec is firm on this), but the inline modal blocker detects unavailability up front and points the user to Paste tab + alternate provider. README "Mobile Limitations" section is explicit.
8	recordingFormat: 'webm' | 'mp4'	iOS/Safari only supports mp4; desktop Electron prefers webm/opus. A hard user choice can produce an unrecordable selection.	Setting becomes a preference ranked against MediaRecorder.isTypeSupported(). Implement getBestMimeType(preference) per the spec's Prior-Art notes: falls back through the candidate list and finally '' (browser default).
9	claude-sonnet-4-20250514 and other model strings throughout	The spec confuses provider config defaults with documentation.	Treat all model strings in the spec as examples in placeholder/help text only; never as runtime defaults.
10	Manifest "Voice Notes"	Rebranded.	Display name = "ReWrite (Voice Notes)"; settings tab header = "ReWrite"; commands prefixed ReWrite:; spec's voice-notes:open-modal id stays the same character string for stability (rewrite:open-modal, rewrite:quick-record): picked once at v1, never renamed.
11	"Keep modal open on error so the user can retry" but Quick Record has no modal	Errors during Quick Record have no UI surface to retry from.	Quick Record errors fall back to opening the modal pre-populated with last audio/transcript (held in memory only). If recording itself failed, surface a Notice and stop.
12	Insert mode cursor with no editor	Unspecified.	If no MarkdownView is active when cursor mode runs, fall back to append to the most-recently-edited markdown file; if none, fall back to newFile using a default name. Surfaced as a Notice.

---

Architecture overview

src/
├── main.ts                     # Lifecycle only: load settings, register commands + ribbon + settings tab
├── types.ts                    # Shared interfaces (ProviderID, EnvironmentProfile, NoteTemplate, etc.)
├── settings/
│   ├── index.ts                # GlobalSettings interface + DEFAULT_SETTINGS + (load|save)Settings helpers
│   ├── tab.ts                  # PluginSettingTab with all 6 sections, dynamic provider-field show/hide
│   └── templates-ui.ts         # Templates CRUD + drag-to-reorder (Sortable-free, native HTML5 DnD)
├── secrets.ts                  # safeStorage + plaintext fallback; ArrayBuffer/base64 in browser primitives
├── platform.ts                 # Platform detection + active-profile resolver + webspeech availability probe
├── recorder.ts                 # MediaRecorder state machine + getBestMimeType + 25MB size guard
├── webspeech.ts                # SpeechRecognition wrapper (returns final transcript on stop)
├── pipeline.ts                 # Orchestrator: transcribe → cleanup → insert
├── insert.ts                   # cursor / newFile / append helpers + {{date}}/{{time}} expansion
├── http.ts                     # requestUrl wrappers: jsonPost, jsonGet, multipartPost (hand-built boundary)
├── ui/
│   ├── modal.ts                # Main modal: template select + Record/Paste tabs + inline setup blocker
│   ├── setup-card.ts           # Inline blocker shown when active profile is unconfigured
│   ├── progress.ts             # "Transcribing…" / "Cleaning up…" labels + cancel
│   └── ribbon.ts               # Ribbon icon registration
├── transcription/
│   ├── index.ts                # interface TranscriptionProvider + factory(profile) → instance
│   ├── openai.ts               # OpenAI Whisper (also reused by openai-compatible + groq presets)
│   ├── assemblyai.ts           # upload → request → poll
│   ├── deepgram.ts             # single POST
│   ├── revai.ts                # submit → poll → fetch transcript
│   └── webspeech.ts            # adapter that just returns the live transcript captured by webspeech.ts
└── llm/
    ├── index.ts                # interface LLMProvider + factory
    ├── openai.ts               # /v1/chat/completions (reused by openai-compatible + mistral preset)
    ├── anthropic.ts            # /v1/messages with system + messages[]
    └── gemini.ts               # :generateContent with system_instruction + contents[]

main.ts stays under ~80 lines: settings load, command registration, ribbon, settings tab. All feature code lives in the modules above.

---

Implementation phases

Per-phase deliverables (applies to every phase below)

Every phase ends with the same two doc touches before it counts as done:

1. Update the live tracker: bump the "Updated" date, flip the phase row to committed/uncommitted, list uncommitted files, and trim the "What's left" section, so a future Claude can resume without re-reading the world. (Historical: this was docs/claude-scratch/STATUS.md, now retired; docs/ROADMAP.md is the live lifecycle tracker.)
2. Update CLAUDE.md if the phase changed anything CLAUDE.md describes (architecture, commands, gotchas, conventions). Per CLAUDE.md's own Documentation Maintenance rule. Phase 13 is the dedicated full refresh; phases before it should at minimum keep the doc from going stale (e.g. drop a pointer to STATUS.md if "Project state" is now wrong).

Treat both as part of the phase, not follow-ups. The user has already had to remind once.

Phase 1: Scaffold rename and types (foundations)

• manifest.json: id → rewrite-plugin, name → ReWrite (Voice Notes), version → 0.1.0 (pre-release), minAppVersion → 1.4.0, description → spec wording, drop fundingUrl/authorUrl/author Obsidian defaults, isDesktopOnly: false.
• Replace src/main.ts and src/settings.ts with the lifecycle-only skeleton plus new file layout.
• Create src/types.ts: TranscriptionProviderID, LLMProviderID, TranscriptionConfig, LLMConfig, EnvironmentProfile, NoteTemplate, InsertMode, GlobalSettings.
• Add styles.css placeholder; ensure tabs/UI styles for modal land here (no styled-components, no CSS-in-JS).

Phase 2: Settings storage + secrets

• src/settings/index.ts: loadSettings/saveSettings wrappers. Migration block tolerates missing keys (return DEFAULT merged with stored).
• src/secrets.ts: saveKey(providerFamily, key) / loadKey(providerFamily) / deleteKey(...). Lazy-require electron's safeStorage only when Platform.isDesktop is true; cache the availability check. Base64 in/out via btoa/atob over UTF-8 bytes. File path: ${plugin.manifest.dir}/secrets.json.nosync.
• Verify the file is created lazily: never write on onload.

Phase 3: Provider abstractions + factories

• src/transcription/index.ts defines:

  interface TranscriptionProvider {
    transcribe(audio: Blob, config: TranscriptionConfig, signal?: AbortSignal): Promise<string>;
    requiresAudio: boolean; // false for webspeech, true otherwise
  }
  function createTranscriptionProvider(id: TranscriptionProviderID): TranscriptionProvider;
  

• src/llm/index.ts mirrors the shape for LLMProvider.complete(systemPrompt, userMessage, config, signal?).
• Factories return shared instances per family (so OpenAI/openai-compatible/groq all dispatch into openai.ts with different baseURL + model).
• src/http.ts wraps requestUrl: surfaces non-2xx as ProviderError with status + body; JSON body helpers stringify and set Content-Type. multipartPost(url, headers, parts) builds a Uint8Array with a random boundary and computed Content-Length.

Phase 4: Transcription adapters (one PR-sized commit per group)

1. openai.ts: single multipart POST, used by openai, openai-compatible, groq. Field response_format: text so the body is plain text.
2. deepgram.ts: raw audio body with Content-Type matching the recorded mime. Parse results.channels[0].alternatives[0].transcript.
3. assemblyai.ts: upload (octet-stream), POST /transcript JSON, poll GET with exponential backoff (1s → 2 → 4 → max 8s), abort at 60s with timeout error.
4. revai.ts: multipart submit, poll job, GET /transcript with Accept: text/plain. Same 60s ceiling.
5. webspeech.ts adapter: thin wrapper that pulls the transcript out of the active webspeech.ts session. requiresAudio = false.

Phase 5: LLM adapters

1. openai.ts: /v1/chat/completions. Reused by openai, openai-compatible, mistral. Parse choices[0].message.content.
2. anthropic.ts: /v1/messages with anthropic-version: 2023-06-01. Parse content[0].text. Add anthropic-dangerous-direct-browser-access: true header defensively in case any user runs in an env where requestUrl behaves like fetch.
3. gemini.ts: :generateContent. Parse candidates[0].content.parts[0].text. Tolerate safetyRatings blocks: surface as a "blocked by safety filter" error.

Phase 6: Recorder + Web Speech

• recorder.ts: state machine idle → recording → paused → recording → stopped. Guard every transition. Collect dataavailable chunks in an array; on stop, concatenate to one Blob and check size. >25MB → reject with size error and a Notice suggesting a shorter clip (v1 deferral of chunking per spec).
• getBestMimeType(preference): ordered candidate list, returns first supported, falls back to ''. Preference inflects ordering only (mp4 first if user picked mp4).
• webspeech.ts: wraps window.SpeechRecognition || window.webkitSpeechRecognition. start() returns a controller with stop() that resolves to final transcript. Detects unsupported environments and rejects synchronously.

Phase 7: Pipeline + Insert

• pipeline.ts: run({ source: 'audio' | 'paste' | 'webspeech', payload, template, profile, onStage }). Stages: transcribe (skipped for paste; webspeech short-circuits with pre-captured transcript) → cleanup → insert. Each stage updates onStage(label) for the modal's progress UI. Errors at transcribe halt the pipeline; LLM errors copy the raw transcript to clipboard and surface a Notice.
• insert.ts:
  • cursor: app.workspace.getActiveViewOfType(MarkdownView)?.editor.replaceSelection(text); if missing, fall back to append.
  • newFile: expand {{date}} → moment().format('YYYY-MM-DD'), {{time}} → HHmmss. Create via app.vault.create(path, body); open via app.workspace.openLinkText(path, '', true).
  • append: append to active note (or last-edited markdown), guaranteeing one trailing newline before the inserted block.

Phase 8: Modal + setup card

• ui/modal.ts: top-of-modal template selector (<select> populated from settings.templates). Two tabs: Record / Paste. Record tab renders a big button + timer + pulsing dot; flips to Stop while recording. Paste tab is a textarea + "Clean Up" button.
• ui/setup-card.ts: shown above the tabs when the active profile is missing transcription provider, LLM provider, or any required field (key/model/baseURL). Card has a provider dropdown + model field + "Open settings" button. Saving the inline form persists and re-renders the modal.
• Keyboard nav: Tab order; Enter triggers the primary action of the active tab; Escape closes (but progress runs in background and posts a final Notice).
• Modal stays open on error per spec; "Retry" button re-runs the last pipeline step.

Phase 9: Settings tab

src/settings/tab.ts renders six sections in order:

1. Active Profile: read-out + override dropdown.
2. Desktop Profile: provider dropdowns + provider-conditional fields. Fields auto show/hide based on selection (webspeech hides key field; openai-compatible reveals base URL field).
3. Mobile Profile: same layout.
4. Global API Keys: one password input per provider family. Label: "Shared across profiles unless overridden above."
5. Templates: list with edit/delete/reorder. Editor is a small inline form: name, prompt (textarea min 6 rows), insertMode select, newFileFolder + newFileNameTemplate (visible when insertMode = newFile).
6. Recording: format preference selector.

Drag-to-reorder uses native HTML5 DnD (no extra dep). Each row has a drag handle and draggable: true.

Phase 10: Commands + ribbon

• rewrite:open-modal ("ReWrite: Open"): opens modal with last-used template selected.
• rewrite:quick-record ("ReWrite: Quick Record"): checks active profile is configured; if not, opens modal to setup card. Otherwise starts recording immediately with a floating mini-UI (small Notice with elapsed timer; click to stop). On stop, runs pipeline with default template.
• Ribbon icon registered via addRibbonIcon('microphone', 'ReWrite', …) opening the modal.

Phase 11: Templates seed data

Ship the 5 default templates with prompt text drafted to spec intent (paraphrased here; actual text written in implementation):

• General Cleanup, Todo List, Daily Note, Meeting Notes, Idea Capture. Loaded on first launch only (i.e. when settings.templates.length === 0 after merge with defaults). Users can fully edit/remove afterward.

Phase 12: README

Replace README.md per spec sections: install, sync-exclusion instructions for every listed sync tool (paths use rewrite-plugin), mobile limitations (iOS screen-off, Web Speech availability, secrets-not-encrypted-on-mobile), acknowledgements (Magic Mic, Scribe, sample-plugin), known limitations.

Phase 13: CLAUDE.md update

Per CLAUDE.md's Documentation Maintenance rule, when implementation lands also update:

• Replace the "Project state" paragraph to reflect that the plugin is now in active development with the structure described above.
• Add a "Pipeline" section enumerating the transcribe → cleanup → insert stages and where each lives.
• Add a "Provider system" section pointing at transcription/index.ts and llm/index.ts factories.
• Add gotchas: requestUrl multipart trick, safeStorage lazy-require, no model defaults, webspeech mobile unavailability.

---

Critical files

Create:

• src/types.ts
• src/settings/index.ts
• src/settings/tab.ts
• src/settings/templates-ui.ts
• src/secrets.ts
• src/platform.ts
• src/recorder.ts
• src/webspeech.ts
• src/pipeline.ts
• src/insert.ts
• src/http.ts
• src/ui/modal.ts
• src/ui/setup-card.ts
• src/ui/progress.ts
• src/ui/ribbon.ts
• src/transcription/{index,openai,assemblyai,deepgram,revai,webspeech}.ts
• src/llm/{index,openai,anthropic,gemini}.ts
• README.md (replace sample)

Modify:

• manifest.json: id/name/version/description/minAppVersion
• src/main.ts: gut to lifecycle skeleton
• src/settings.ts: delete (logic moves under src/settings/)
• CLAUDE.md: refresh "Project state" + add Pipeline / Provider system sections
• styles.css: modal styles

Keep:

• esbuild.config.mjs: already correct
• tsconfig.json, eslint.config.mts, .editorconfig
• .github/workflows/lint.yml
• version-bump.mjs

---

Verification

No test runner is configured (CLAUDE.md). Verification is the spec's Testing Checklist plus the following manual procedure on each device class.

Build + lint gate (CI parity):

npm install
npm run build     # tsc -noEmit + esbuild production
npm run lint

Local install (desktop):

1. Symlink main.js, manifest.json, styles.css into <TestVault>/.obsidian/plugins/rewrite-plugin/.
2. Enable in Obsidian.
3. Walk the Testing Checklist (lines 454–476 of the spec): every checkbox manually verified.
4. Verify secrets.json.nosync exists, contains base64 blobs with _encrypted: true, and is unreadable when copied to a different desktop user account.

Mobile smoke (Android + iOS Obsidian):

• Profile auto-switches to Mobile.
• Paste tab → Anthropic happy path (with key entered on device).
• Record tab on Android: webspeech captures speech; on iOS: setup card surfaces "Web Speech unavailable; use Paste or pick another provider" and lets the user switch.
• secrets.json.nosync written with _encrypted: false on mobile.

Smoke per provider: at least one happy-path transcript per provider, validated against the response shape parser. AssemblyAI and Rev.ai also tested for the 60s timeout (point at a fake slow endpoint or a known long clip).

Modal UX: Tab navigation, Enter triggers primary action, error keeps modal open, Retry button re-runs the last failed stage.

Insert modes: all three produce correct placement; {{date}} and {{time}} expand in filenames.

Edge cases: missing key → Notice, no network → Notice, empty transcript → halts before LLM, oversize recording (>25MB) → halts with Notice.