This commit is contained in:
Michael Gorini 2026-04-30 08:43:44 +02:00
parent e7eb786aa5
commit 1520be5185
10 changed files with 182 additions and 113 deletions

221
README.md
View file

@ -1,68 +1,165 @@
# Resonance
Resonance is the clean-slate v2 of the Obsidian recording plugin. The product is now built around a local-first pipeline, settings-first setup, and manifest-backed session storage.
![Resonance banner](assets/banner.png)
## Fastest Setup
Resonance is a local-first AI recorder for Obsidian.
If this machine already has a built `whisper.cpp` and at least one local ggml model:
It is built around:
1. Open the plugin settings page.
2. Work through **Step 1**, **Step 2**, and **Step 3** from top to bottom.
3. Use the Detect buttons to fill paths automatically when possible.
4. Pick a microphone and run **Quick test**.
5. Leave **Ollama** as the provider unless you intentionally want a cloud provider.
- Web Audio capture
- Whisper for transcription
- Ollama or an optional cloud provider for summaries
- Session storage with a built-in library
The settings page is now the full setup flow. Nothing important is hidden behind a separate setup modal.
The product is desktop-only.
## What Changed
## What Resonance Does
- `src/` is the only active application tree for v2.
- The primary entrypoint is now the plugin settings surface instead of a simple start/stop flow.
- Recording state is driven by an explicit session controller with ordered live transcription.
- Sessions are stored as structured manifests with `audio/`, `transcript/`, `summary/`, and `diagnostics.log`.
- The session library is manifest-backed. It no longer scans raw media files blindly.
- Records from one main microphone plus optional extra audio inputs in the same Web Audio graph
- Writes live transcript updates while recording
- Builds a final summary note after the recording stops
- Stores each session with audio, transcript, summary, and diagnostics
- Lets you inspect, recover, clean up, and bulk-delete session artifacts from the Library
## Product Direction
## Screenshots
- Desktop-only Obsidian plugin
- Local-first path: `Web Audio` + `whisper.cpp` + `Ollama`
- Cloud summary providers remain available as secondary adapters
- Native Obsidian settings are now the primary surface, with horizontal tabs for diagnostics, library, and setup
### Recorder
## Current Information Architecture
![Recorder UI](assets/recorder.png)
- Diagnostics
- Health status and blocking issues
- Quick test
- Startup behavior
- Setup guidance when the local path is incomplete
- Setup & Settings
- Horizontal tabs for Diagnostics, Library, Capture, Transcription, Summary, and Output
- Capture tab for microphone, additional sources, and quick test
- Transcription tab for whisper.cpp, CLI, and model
- Summary tab for Ollama or cloud providers
- Output tab for vault, retention, and startup behavior
- Session Library
- Filters for `done` and `failed`
- Artifact availability
- Transcript, diagnostics, audio, and summary actions
### Library
## Repository Layout
![Library UI](assets/library.png)
```text
src/ Active Resonance v2 source
tests/ Pure unit tests for v2
dist/ Build output
## First Successful Session
1. Install the plugin in your vault.
2. Open the Resonance settings page.
3. In `Capture`, choose your microphone.
4. Optionally add loopback or monitor inputs under `Additional sources` if you want call or desktop audio in the same recording.
5. In `Transcription`, set the `whisper.cpp` repo, CLI, and model paths.
6. In `Summary`, keep `Ollama` or choose a cloud provider.
7. Run `Quick test`.
8. Open the recorder and make a short recording.
9. Stop the session and confirm that the transcript and summary appear in the Library.
## Setup Overview
### Capture
- `Microphone device`: the voice input you speak into
- `Additional sources`: optional extra audio inputs such as loopback or monitor devices
- `Segment seconds`: how often live transcript updates are committed
- `Quick test`: verifies microphone access and the local pipeline
### Transcription
Resonance expects a local `whisper.cpp` build plus a readable ggml model.
Typical setup:
```bash
git clone https://github.com/ggerganov/whisper.cpp
cd whisper.cpp
cmake -S . -B build
cmake --build build -j
```
Then download a model, for example:
```bash
cd /path/to/whisper.cpp/models
./download-ggml-model.sh small
```
Set:
- `whisper.cpp repo`
- `whisper.cpp CLI`
- `Model path`
### Summary
Recommended local path:
- provider: `Ollama`
- endpoint: `http://localhost:11434`
- model: `gemma3`
Cloud providers are also supported, but they require API credentials on the machine where the plugin runs.
### Library
The Library is the operational workspace for finished and failed sessions.
It supports:
- previewing transcript and diagnostics
- opening transcript and summary notes
- audio playback and export
- recovery actions when transcript or summary is missing
- bulk cleanup with storage estimates
- storage stats for visible sessions
## Runtime Artifacts
Each supported session persists:
- `session.json`
- `audio/recording.wav`
- `audio/segments/`
- `transcript/live-transcript.txt`
- `summary/summary.md`
- `diagnostics.log`
The session Library reads these manifests rather than scanning arbitrary files.
## Common Failures
### Microphone access denied
Symptoms:
- Quick test fails before recording starts
- Capture diagnostics show microphone access denied
Fix:
- Re-enable microphone access for Obsidian in the operating system settings
- Return to Resonance and run `Quick test` again
### whisper.cpp or model missing
Symptoms:
- Diagnostics block recording
- Quick test captures audio but transcription fails
Fix:
- Point Resonance to a working `whisper.cpp` CLI
- Set a readable ggml model file such as `ggml-small.bin`
### Additional source unavailable
Symptoms:
- A saved loopback or monitor input no longer appears
- Diagnostics warn that extra sources will be skipped
Fix:
- Re-select the source in `Capture`
- Remove stale additional inputs you no longer use
## Local Development
Requirements:
- Obsidian desktop
- Node.js
- whisper.cpp with a local model
- Ollama if you want the full tier-1 local path
- `whisper.cpp` with a local ggml model
- Ollama if you want the default local summary path
Commands:
@ -72,39 +169,11 @@ npm test
npm run build
```
## Install In Obsidian
## Manual Install in Obsidian
1. Build the plugin with `npm run build`.
1. Run `npm run build`.
2. Copy `dist/main.js`, `dist/manifest.json`, and `dist/styles.css` into your vault plugin folder.
3. Enable `Resonance` in Obsidian community plugins.
4. Open the plugin settings page.
5. Use the `Capture` tab for microphone and additional sources, then `Transcription` and `Summary`.
6. When setup is done, use the `Diagnostics` tab for health and the `Library` tab for saved session artifacts.
## Minimum Local Requirements
To record and summarize locally, Resonance needs:
- `whisper.cpp` CLI
- a readable local ggml model file
- `Ollama` running locally for summaries
If auto-detect fails, the manual fallback is:
1. Point Step 3 in plugin settings at your `whisper.cpp` repo.
2. Detect or set the `whisper.cpp` CLI path.
3. Detect or set the ggml model path.
4. Return to Step 2 and run the quick test.
## Runtime Model
Each session persists:
- `session.json`
- `audio/recording.wav`
- `audio/segments/`
- `transcript/live-transcript.txt`
- `summary/summary.md`
- `diagnostics.log`
The live transcription queue commits segments strictly in order and the stop flow waits for the queue to flush before summary generation starts.
4. Open the Resonance settings page.
5. Work through `Capture`, `Transcription`, and `Summary`.
6. Use `Diagnostics` for health checks and `Library` for saved sessions.

BIN
assets/banner.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

BIN
assets/library.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 235 KiB

BIN
assets/recorder.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 330 KiB

View file

@ -3,9 +3,9 @@
"name": "Resonance",
"version": "0.1.0",
"minAppVersion": "1.5.0",
"description": "A local first AI Obsidian recorder with resilient live transcription, diagnostics, and session based notes.",
"description": "A local-first AI Obsidian recorder with live transcription, diagnostics, and session-based notes.",
"author": "Michael Gorini",
"authorUrl": "https://buymeacoffee.com/michaelgorini",
"fundingUrl": "https://buymeacoffee.com/michaelgorini",
"isDesktopOnly": true
}
}

View file

@ -1,9 +1,9 @@
{
"name": "resonance-next",
"version": "2.0.0",
"version": "0.1.0",
"private": true,
"type": "module",
"description": "Obsidian plugin v2: local-first recording, live transcription, diagnostics, and resilient note generation.",
"description": "A local-first AI Obsidian recorder with live transcription, diagnostics, and session-based notes.",
"scripts": {
"dev": "node ./node_modules/vite/bin/vite.js build --watch",
"build": "node ./node_modules/vite/bin/vite.js build && node scripts/postbuild.mjs",

View file

@ -51,7 +51,7 @@ export class DiagnosticsService {
severity: isCoreConfigured(settings) ? "ok" : "error",
detail: isCoreConfigured(settings)
? "Core local-first path is configured."
: "Web Audio capture is ready, but transcription or summary configuration is still incomplete.",
: "Recording is ready, but transcription or summary setup is still incomplete.",
remediation: "Open Setup & Settings in the plugin settings tab and complete the missing dependency step.",
});
@ -69,7 +69,7 @@ export class DiagnosticsService {
label: "Web Audio capture",
severity: capability.hasGetUserMedia && capability.hasEnumerateDevices ? "ok" : "error",
detail: capability.hasGetUserMedia && capability.hasEnumerateDevices
? `Web Audio APIs are available${permissionState === "granted" ? " and microphone permission is granted." : permissionState === "prompt" ? ". Microphone permission will be requested on first recording." : permissionState === "denied" ? ", but microphone permission is denied." : "."}`
? `Web Audio APIs are available${permissionState === "granted" ? " and microphone access is granted." : permissionState === "prompt" ? ". Microphone access will be requested on the first recording or quick test." : permissionState === "denied" ? ", but microphone access is denied." : "."}`
: "Required browser audio APIs are unavailable in this runtime.",
remediation:
permissionState === "denied"
@ -82,7 +82,7 @@ export class DiagnosticsService {
label: "Audio input devices",
severity: deviceSnapshot.devices.length > 0 ? "ok" : "warning",
detail: deviceSnapshot.devices.length > 0
? `${deviceSnapshot.devices.length} audio input${deviceSnapshot.devices.length === 1 ? "" : "s"} discovered via Web Audio.${deviceSnapshot.labelsAvailable ? "" : " Device labels will become more descriptive after microphone permission is granted."}`
? `${deviceSnapshot.devices.length} audio input${deviceSnapshot.devices.length === 1 ? "" : "s"} discovered.${deviceSnapshot.labelsAvailable ? "" : " Labels become more descriptive after microphone access is granted."}`
: "No audio inputs were discovered via Web Audio.",
remediation: "Grant microphone permission and check the OS input devices if the list stays empty.",
});
@ -96,8 +96,8 @@ export class DiagnosticsService {
detail: selectedMic
? selectedMicPresent
? "Selected microphone is available."
: "Selected microphone is unavailable. Recording will fall back to the system default input."
: "No specific microphone selected. Recording will use the system default input.",
: "Selected microphone is unavailable. Resonance will use the current system default input instead."
: "No specific microphone selected. Resonance will use the current system default input.",
remediation: "Choose a microphone device if you want to pin one instead of following the OS default.",
});
@ -124,7 +124,7 @@ export class DiagnosticsService {
: "ok",
detail:
additionalSources.length === 0
? "No additional sources selected. Recording will use only the microphone."
? "No additional sources selected. Recordings will use only the microphone."
: duplicateIds.size > 0
? "Some additional sources are duplicated and will be ignored."
: sameAsMicSources.length > 0
@ -133,7 +133,7 @@ export class DiagnosticsService {
? `${missingSources.length} selected additional source${missingSources.length === 1 ? "" : "s"} are unavailable and will be skipped.`
: `${additionalSources.length} additional source${additionalSources.length === 1 ? "" : "s"} selected for the recording mix.`,
remediation:
"Keep only the extra audioinput devices you actually want to mix in, such as loopback or monitor inputs.",
"Keep only the extra audio inputs you actually want to mix in, such as loopback or monitor sources.",
});
addCheck({
@ -244,7 +244,7 @@ export class DiagnosticsService {
const transcriptionAdapter = new WhisperTranscriptionAdapter(settings.transcription);
const transcript = await transcriptionAdapter.transcribeFile(audioPath);
if (!transcript.trim()) {
return { ok: false, detail: "Quick test recorded audio, but whisper.cpp returned an empty transcript." };
return { ok: false, detail: "Quick test captured audio, but whisper.cpp returned an empty transcript." };
}
}

View file

@ -275,7 +275,7 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
const intro = this.createGuideSection(container, {
badge: "Workspace",
title: "Diagnostics",
intro: "Check what is blocking the local pipeline and fix it in the setup tabs.",
intro: "Check permissions, devices, and local dependencies before you record.",
});
const actions = intro.createDiv({ cls: "rxn-action-bar" });
@ -323,7 +323,7 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
const preferences = this.createGuideSection(container, {
badge: "Preferences",
title: "Quick test and startup",
intro: "Use these to tune the smoke test and choose which tab opens first.",
intro: "Adjust the quick test length and choose which tab opens first.",
});
new Setting(preferences)
@ -343,7 +343,7 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
new Setting(preferences)
.setName("Open setup on startup")
.setDesc("Open Capture on launch.")
.setDesc("Open Capture when Obsidian starts.")
.addToggle((toggle) =>
toggle.setValue(settings.ui.showSetupWizardOnStartup).onChange(async (value) => {
await this.options.saveSettings((current) => ({
@ -355,7 +355,7 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
new Setting(preferences)
.setName("Open diagnostics on startup")
.setDesc("Open Diagnostics on launch.")
.setDesc("Open Diagnostics when Obsidian starts.")
.addToggle((toggle) =>
toggle.setValue(settings.ui.showDiagnosticsOnStartup).onChange(async (value) => {
await this.options.saveSettings((current) => ({
@ -367,7 +367,7 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
preferences.createEl("p", {
cls: "rxn-muted",
text: "Session logs are available in Library with Preview diagnostics.",
text: "Session logs remain available in Library under Preview diagnostics.",
});
}
@ -394,12 +394,12 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
cls: `rxn-status-pill is-${this.getWebPermissionTone(deviceSnapshot.permissionState)}`,
});
meta.createEl("span", {
text: deviceSnapshot.devices.length > 0 ? `${deviceSnapshot.devices.length} mic inputs` : "No mic inputs found",
text: deviceSnapshot.devices.length > 0 ? `${deviceSnapshot.devices.length} audio inputs` : "No audio inputs found",
cls: "rxn-pill",
});
if (!deviceSnapshot.labelsAvailable) {
meta.createEl("span", {
text: "Labels improve after granting mic permission",
text: "Labels improve after microphone access is granted",
cls: "rxn-pill",
});
}
@ -408,8 +408,8 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
.setName("Microphone device")
.setDesc(
deviceSnapshot.labelsAvailable
? "Pick the main voice input or stay on the system default input."
: "Use the system default input or grant permission once to reveal clearer device labels."
? "Choose the voice input you speak into, or leave Resonance on the system default input."
: "Use the system default input or allow microphone access once to reveal clearer device labels."
);
const micSelect = micSetting.controlEl.createEl("select");
micSelect.addClass("rxn-inline-select");
@ -436,7 +436,7 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
capture.createEl("p", {
cls: "rxn-muted",
text:
"Optional. Add loopback or monitor inputs here if you want Teams, Meet, browser tabs, or desktop audio in the same recording.",
"Optional. Add loopback or monitor inputs here if you also want call, browser, or desktop audio in the same recording.",
});
const additionalSourceCandidates = deviceSnapshot.devices.filter(
@ -455,14 +455,14 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
const stale = capture.createDiv({ cls: "rxn-inline-note is-warning" });
stale.createEl("strong", { text: "Saved sources need review" });
stale.createEl("p", {
text: `${staleAdditionalCount} saved additional source${staleAdditionalCount === 1 ? "" : "s"} are unavailable or conflict with the microphone and will be ignored until you update them.`,
text: `${staleAdditionalCount} saved additional source${staleAdditionalCount === 1 ? "" : "s"} are unavailable or duplicate the microphone and will be ignored until you update them.`,
});
}
if (additionalSourceCandidates.length === 0) {
capture.createEl("p", {
cls: "rxn-muted",
text: "No additional audio inputs are currently available.",
text: "No extra audio inputs are currently available.",
});
} else {
additionalSourceCandidates.forEach((device) => {
@ -494,7 +494,7 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
new Setting(capture)
.setName("Segment seconds")
.setDesc("How often live transcript chunks are committed.")
.setDesc("How often live transcript updates are committed.")
.addText((text) =>
text.setPlaceholder("20").setValue(String(settings.capture.segmentDurationSeconds)).onChange(async (value) => {
await this.options.saveSettings((current) => ({
@ -921,7 +921,7 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
const library = this.createGuideSection(container, {
badge: "Workspace",
title: uiCopy.library.title,
intro: "Review completed and failed sessions here.",
intro: "Review sessions, notes, storage, and cleanup actions here.",
});
const controls = library.createDiv({ cls: "rxn-toolbar" });
@ -1527,7 +1527,7 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
await this.display();
try {
new Notice("Quick test started...");
new Notice("Quick test started. Obsidian may ask for microphone access.");
const result = await this.options.controller.runSmokeTest();
this.smokeMessage = result.ok
? uiCopy.diagnostics.smokePassed
@ -1626,13 +1626,13 @@ export class ResonanceNextSettingTab extends PluginSettingTab {
private getWebPermissionLabel(permission: WebAudioPermissionState): string {
switch (permission) {
case "granted":
return "Granted";
return "Ready";
case "denied":
return "Denied";
case "prompt":
return "Not requested";
return "Needs access";
case "unsupported":
return "Unsupported";
return "Unavailable";
case "unknown":
default:
return "Unknown";

View file

@ -47,7 +47,7 @@ export const uiCopy = {
},
diagnostics: {
title: "Diagnostics",
subtitle: "Health checks for the local-first recording pipeline.",
subtitle: "Check permissions, devices, whisper.cpp, and summary provider health.",
blocking: "Blocking issues",
warnings: "Warnings",
healthy: "Healthy checks",
@ -57,7 +57,7 @@ export const uiCopy = {
},
library: {
title: "Session Library",
subtitle: "Review finished and failed sessions from manifest-backed metadata.",
subtitle: "Review saved sessions, notes, cleanup actions, and storage usage.",
all: "All",
done: "Done",
failed: "Failed",
@ -66,7 +66,7 @@ export const uiCopy = {
deleteConfirmation: "Delete this session and its vault notes?",
},
settings: {
title: "Local recording, transcription, and summaries for Obsidian.",
title: "Local recordings, transcripts, and summaries inside Obsidian.",
},
notices: {
sessionComplete: "Session completed.",

View file

@ -122,7 +122,7 @@ export class RecordingModal extends Modal {
if (!coreConfigured) {
const note = panel.createDiv({ cls: "rxn-inline-note is-warning" });
note.createEl("strong", { text: "Setup incomplete" });
note.createEl("p", { text: "Finish Capture, Transcription, and Summary before starting a recording." });
note.createEl("p", { text: "Finish Capture, Transcription, and Summary setup before starting your first recording." });
} else if (snapshot.state === "done") {
const note = panel.createDiv({ cls: "rxn-inline-note is-healthy" });
note.createEl("strong", { text: "Last session completed" });
@ -153,7 +153,7 @@ export class RecordingModal extends Modal {
this.createActionButton(actions, uiCopy.actions.openLiveTranscript, async () => {
const opened = await this.options.controller.openActiveLiveTranscript();
if (!opened) {
new Notice("Live transcript is not available yet.");
new Notice("The live transcript note appears after the first transcript chunk is committed.");
}
}, "rxn-btn-secondary", !this.options.controller.getActiveLiveTranscriptNotePath());
}
@ -173,7 +173,7 @@ export class RecordingModal extends Modal {
? "Recording is done. Resonance is writing the final note and session files."
: snapshot.state === "transcribing_live"
? "Recording is active and live transcription is catching up."
: "Recording is active. Stop when you want to finalize the summary.",
: "Recording is active. Stop when you are ready to generate the final summary.",
cls: "rxn-muted",
});
@ -210,7 +210,7 @@ export class RecordingModal extends Modal {
this.createActionButton(actions, uiCopy.actions.openLiveTranscript, async () => {
const opened = await this.options.controller.openActiveLiveTranscript();
if (!opened) {
new Notice("Live transcript is not available yet.");
new Notice("The live transcript note appears after the first transcript chunk is committed.");
}
}, "rxn-btn-secondary", !this.options.controller.getActiveLiveTranscriptNotePath());