This commit is contained in:
WiseGuru 2026-05-24 21:51:43 -07:00
parent 0c6cd44d72
commit 0587c20ff6
12 changed files with 315 additions and 49 deletions

View file

@ -93,6 +93,8 @@ The `PipelineSource` union has four variants: `audio` (recorded blob), `paste` (
API keys are stored per profile on `EnvironmentProfile.transcriptionConfig.apiKey` / `llmConfig.apiKey`. Two slots per profile, one for transcription and one for the LLM. No global by-family map; the desktop and mobile profiles each carry their own keys even when both use the same provider (deliberate: per-profile keys make per-function usage tracking easier). Persistence is in [src/secrets.ts](src/secrets.ts) using the key IDs `profile:desktop:transcription`, `profile:desktop:llm`, `profile:mobile:transcription`, `profile:mobile:llm`.
Providers may optionally implement `listModels(config, signal)` returning a string array of model IDs the configured API key can access. Implemented by: OpenAI / Groq / Mistral (via `openai.ts` shared adapter), Anthropic, Gemini, Deepgram. Not implemented for `openai-compatible` (URL-specific, list-shape varies), AssemblyAI, Rev.ai, Web Speech. The settings tab caches results to `GlobalSettings.modelCache` per side and provider ID; the Refresh button in the model field triggers `listModels` and updates the cache. The text field next to the dropdown is always the canonical source of `profile.config.model` so users can type any string the dropdown doesn't expose.
## Settings
`GlobalSettings` (defined in [src/types.ts](src/types.ts)) is the shape of `data.json`. Loading flow:

View file

@ -4,26 +4,7 @@ Forward-looking backlog. Not committed to a release. Add items as they come up;
## Open
### 1. LLM model field as a dropdown of available models
**Current state:** [src/settings/tab.ts:197-206](../src/settings/tab.ts#L197-L206) renders "LLM model" as a free-text input with a hint string from `llmModelHint()`. Same shape for "Transcription model" at [src/settings/tab.ts:136-145](../src/settings/tab.ts#L136-L145).
**Desired:** dropdown populated with models the configured provider actually exposes, instead of asking the user to type a model ID correctly.
**Open design questions (decide when picking this up):**
- **Source of the list.** Two options:
1. *Live fetch* per provider. OpenAI has `GET /v1/models`, Anthropic has `GET /v1/models` (recent), Gemini has `GET /v1beta/models`, Mistral has `GET /v1/models`, Groq mirrors OpenAI, `openai-compatible` mirrors OpenAI. Pros: always current, surfaces user's actual entitlements. Cons: requires the API key to already be set before model selection, adds a network call to the settings tab, needs caching + a refresh button, needs graceful degradation when offline.
2. *Static curated list* baked into each adapter, refreshed when we ship updates. Pros: works without a key, instant. Cons: stale list problem (this plugin's whole reason for existing is that model names move fast).
- **Recommended:** live fetch with on-disk cache (in `data.json`, per provider family, with timestamp), a "Refresh models" button next to the dropdown, and a "Custom..." escape hatch that falls back to free-text. Cache TTL ~7 days. If the fetch fails (no key, offline, 401), show the cached list if any, otherwise fall back to free-text with the current hint.
- **Same treatment for transcription model?** Yes, for the providers that expose it (OpenAI Whisper has limited models; AssemblyAI has tiers; Deepgram has many; Groq mirrors OpenAI; `openai-compatible` and `revai` should stay free-text since the namespace is user-controlled).
- **Per-profile vs. global cache?** Global per provider family — the model list doesn't depend on which profile you're configuring.
**Touch points:** [src/settings/tab.ts](../src/settings/tab.ts) (UI), per-provider adapter files under [src/llm/](../src/llm/) and [src/transcription/](../src/transcription/) (add `listModels(config)`), [src/types.ts](../src/types.ts) (cache shape on `GlobalSettings`).
---
### 2. Plugin-managed local whisper.cpp server (desktop)
### 1. Plugin-managed local whisper.cpp server (desktop)
**Goal:** let a desktop user run fully on-device transcription with whisper.cpp without ever opening a terminal during normal use. Plugin spawns the server, plugin tears it down, pipeline talks to it via the existing `openai-compatible` adapter under the hood.
@ -76,6 +57,10 @@ Once Phase A is solid:
## Done
### Model field as a dropdown of available models
Added an optional `listModels(config, signal)` method to both `TranscriptionProvider` and `LLMProvider` interfaces. Implementations: OpenAI / Groq / Mistral (shared via the OpenAI-shaped adapter), Anthropic, Gemini, Deepgram. Skipped (text-only fallback): `openai-compatible` (URL-specific, list-shape varies), AssemblyAI, Rev.ai, Web Speech. Settings tab renders a hybrid Setting per model: dropdown of cached models + Refresh button (when the provider supports listing) plus an always-canonical text input that wins for "Custom" model names not yet in the dropdown. Cache lives at `GlobalSettings.modelCache.{transcription,llm}[providerId] = { ids, fetchedAt }`. No auto-fetch on settings open; the user clicks Refresh once their key is set. Errors surface via Notice with the provider-attributed `ProviderError` message.
### Act on existing text in a note
Added a `{ kind: 'text'; text: string }` source variant to the pipeline (skips transcription, same path as `paste`). Three entry points: `rewrite-plugin:process-text` command, an editor-menu item "ReWrite with template...", and a new "From note" tab in the main modal. All three resolve text from the active editor (selection if non-empty, otherwise whole note body) and open a lightweight template quick-picker before running. Setup-card gating split into voice (full check) vs text (LLM-only check) via `isProfileConfiguredForText` and a `purpose` param on `renderSetupCard`; the modal's per-tab rendering picks the right gate. Helpers live in [src/ui/text-source.ts](../src/ui/text-source.ts) (resolution + `runTextPipeline`) and [src/ui/template-picker.ts](../src/ui/template-picker.ts) (the quick-picker modal). Shipped without the per-invocation "Replace source" checkbox; can revisit if users ask.

View file

@ -1,5 +1,5 @@
import { LLMConfig } from '../types';
import { jsonPost } from '../http';
import { jsonGet, jsonPost } from '../http';
import { LLMProvider } from './index';
interface MessagesResponse {
@ -7,6 +7,15 @@ interface MessagesResponse {
stop_reason?: string;
}
interface ModelsListResponse {
data?: Array<{ id?: unknown }>;
}
const ANTHROPIC_HEADERS = {
'anthropic-version': '2023-06-01',
'anthropic-dangerous-direct-browser-access': 'true',
};
export function createAnthropicLLM(): LLMProvider {
return {
id: 'anthropic',
@ -30,8 +39,7 @@ export function createAnthropicLLM(): LLMProvider {
body,
{
'x-api-key': config.apiKey,
'anthropic-version': '2023-06-01',
'anthropic-dangerous-direct-browser-access': 'true',
...ANTHROPIC_HEADERS,
},
signal,
);
@ -41,5 +49,23 @@ export function createAnthropicLLM(): LLMProvider {
}
return firstText.text.trim();
},
async listModels(config, signal) {
if (!config.apiKey) throw new Error('anthropic: API key is not configured');
const response = await jsonGet<ModelsListResponse>(
'anthropic',
'https://api.anthropic.com/v1/models?limit=1000',
{
'x-api-key': config.apiKey,
...ANTHROPIC_HEADERS,
},
signal,
);
const out: string[] = [];
for (const row of response.data ?? []) {
if (typeof row.id === 'string' && row.id) out.push(row.id);
}
out.sort();
return out;
},
};
}

View file

@ -1,5 +1,5 @@
import { LLMConfig } from '../types';
import { jsonPost } from '../http';
import { jsonGet, jsonPost } from '../http';
import { LLMProvider } from './index';
interface GenerateContentResponse {
@ -10,6 +10,13 @@ interface GenerateContentResponse {
promptFeedback?: { blockReason?: string };
}
interface GeminiModelsResponse {
models?: Array<{
name?: unknown;
supportedGenerationMethods?: unknown;
}>;
}
export function createGeminiLLM(): LLMProvider {
return {
id: 'gemini',
@ -49,5 +56,23 @@ export function createGeminiLLM(): LLMProvider {
}
return text.trim();
},
async listModels(config, signal) {
if (!config.apiKey) throw new Error('gemini: API key is not configured');
const url = `https://generativelanguage.googleapis.com/v1beta/models?key=${encodeURIComponent(config.apiKey)}&pageSize=1000`;
const response = await jsonGet<GeminiModelsResponse>('gemini', url, {}, signal);
const out: string[] = [];
for (const row of response.models ?? []) {
const methods = Array.isArray(row.supportedGenerationMethods)
? row.supportedGenerationMethods.filter((m): m is string => typeof m === 'string')
: [];
if (!methods.includes('generateContent')) continue;
const name = typeof row.name === 'string' ? row.name : '';
if (!name) continue;
const stripped = name.startsWith('models/') ? name.slice('models/'.length) : name;
out.push(stripped);
}
out.sort();
return out;
},
};
}

View file

@ -11,6 +11,7 @@ export interface LLMProvider {
config: LLMConfig,
signal?: AbortSignal,
): Promise<string>;
listModels?(config: LLMConfig, signal?: AbortSignal): Promise<string[]>;
}
export function createLLMProvider(id: LLMProviderID): LLMProvider {

View file

@ -1,5 +1,5 @@
import { LLMConfig, LLMProviderID } from '../types';
import { jsonPost } from '../http';
import { jsonGet, jsonPost } from '../http';
import { LLMProvider } from './index';
interface ChatCompletionResponse {
@ -8,8 +8,12 @@ interface ChatCompletionResponse {
}>;
}
interface ModelsListResponse {
data?: Array<{ id?: unknown }>;
}
export function createOpenAILLM(id: LLMProviderID): LLMProvider {
return {
const provider: LLMProvider = {
id,
async complete(
systemPrompt: string,
@ -42,6 +46,41 @@ export function createOpenAILLM(id: LLMProviderID): LLMProvider {
return content.trim();
},
};
if (id !== 'openai-compatible') {
provider.listModels = async (config, signal) => {
if (!config.apiKey) throw new Error(`${id}: API key is not configured`);
const url = id === 'mistral'
? 'https://api.mistral.ai/v1/models'
: 'https://api.openai.com/v1/models';
const response = await jsonGet<ModelsListResponse>(
id,
url,
{ Authorization: `Bearer ${config.apiKey}` },
signal,
);
return filterChatModels(response.data ?? []);
};
}
return provider;
}
function filterChatModels(rows: Array<{ id?: unknown }>): string[] {
const out: string[] = [];
for (const row of rows) {
const id = typeof row.id === 'string' ? row.id : '';
if (!id) continue;
const lower = id.toLowerCase();
if (lower.includes('whisper') || lower.includes('embedding')) continue;
if (lower.includes('transcribe') || lower.includes('tts')) continue;
if (lower.includes('dall-e') || lower.includes('image')) continue;
if (lower.includes('audio') || lower.includes('speech')) continue;
if (lower.includes('moderation') || lower.includes('search')) continue;
out.push(id);
}
out.sort();
return out;
}
function resolveEndpoint(id: LLMProviderID, config: LLMConfig): string {

View file

@ -47,6 +47,7 @@ export const DEFAULT_SETTINGS: GlobalSettings = {
lastUsedTemplateId: '',
recordingFormat: 'webm',
templates: [],
modelCache: { transcription: {}, llm: {} },
};
const PROFILE_KINDS: ActiveProfileKind[] = ['desktop', 'mobile'];
@ -129,6 +130,10 @@ function mergeSettings(
desktopProfile: mergeProfile(base.desktopProfile, partial.desktopProfile),
mobileProfile: mergeProfile(base.mobileProfile, partial.mobileProfile),
templates: partial.templates ?? base.templates,
modelCache: {
transcription: { ...base.modelCache.transcription, ...(partial.modelCache?.transcription ?? {}) },
llm: { ...base.modelCache.llm, ...(partial.modelCache?.llm ?? {}) },
},
};
}

View file

@ -5,12 +5,16 @@ import {
ActiveProfileOverride,
EnvironmentProfile,
InsertMode,
LLMConfig,
LLMProviderID,
NoteTemplate,
RecordingFormatPreference,
TranscriptionConfig,
TranscriptionProviderID,
} from '../types';
import { detectActiveProfileKind } from '../platform';
import { createTranscriptionProvider } from '../transcription';
import { createLLMProvider } from '../llm';
const TRANSCRIPTION_OPTIONS: Array<{ id: TranscriptionProviderID; label: string }> = [
{ id: 'openai', label: 'OpenAI Whisper' },
@ -121,16 +125,7 @@ export class ReWriteSettingTab extends PluginSettingTab {
});
if (profile.transcriptionProvider !== 'webspeech') {
new Setting(parent)
.setName('Transcription model')
.setDesc(transcriptionModelHint(profile.transcriptionProvider))
.addText((t) => {
t.setValue(profile.transcriptionConfig.model);
t.onChange(async (v) => {
profile.transcriptionConfig.model = v;
await this.commit();
});
});
this.renderTranscriptionModelField(parent, profile);
if (profile.transcriptionProvider === 'openai-compatible') {
new Setting(parent)
@ -170,16 +165,7 @@ export class ReWriteSettingTab extends PluginSettingTab {
});
});
new Setting(parent)
.setName('LLM model')
.setDesc(llmModelHint(profile.llmProvider))
.addText((t) => {
t.setValue(profile.llmConfig.model);
t.onChange(async (v) => {
profile.llmConfig.model = v;
await this.commit();
});
});
this.renderLLMModelField(parent, profile);
if (profile.llmProvider === 'openai-compatible') {
new Setting(parent)
@ -240,6 +226,130 @@ export class ReWriteSettingTab extends PluginSettingTab {
});
}
private renderTranscriptionModelField(parent: HTMLElement, profile: EnvironmentProfile): void {
const wrapper = parent.createDiv({ cls: 'rewrite-model-field' });
this.populateTranscriptionModelField(wrapper, profile);
}
private populateTranscriptionModelField(wrapper: HTMLElement, profile: EnvironmentProfile): void {
wrapper.empty();
const providerId = profile.transcriptionProvider;
const provider = createTranscriptionProvider(providerId);
const supportsList = typeof provider.listModels === 'function';
const cached = this.plugin.settings.modelCache.transcription[providerId]?.ids ?? [];
const current = profile.transcriptionConfig.model;
const setting = new Setting(wrapper).setName('Transcription model');
setting.setDesc(modelFieldDesc(transcriptionModelHint(providerId), supportsList, cached.length));
if (supportsList) {
setting.addDropdown((dd) => {
dd.addOption('', cached.length === 0 ? '(no cached models)' : '(pick a model)');
for (const id of cached) dd.addOption(id, id);
dd.setValue(cached.includes(current) ? current : '');
dd.onChange(async (v) => {
if (!v) return;
profile.transcriptionConfig.model = v;
await this.commit();
this.populateTranscriptionModelField(wrapper, profile);
});
});
setting.addExtraButton((b) => {
b.setIcon('refresh-cw').setTooltip('Refresh model list').onClick(async () => {
await this.refreshTranscriptionModels(providerId, profile.transcriptionConfig);
this.populateTranscriptionModelField(wrapper, profile);
});
});
}
setting.addText((t) => {
t.setValue(current);
t.setPlaceholder(supportsList ? '' : transcriptionModelHint(providerId));
t.onChange(async (v) => {
profile.transcriptionConfig.model = v;
await this.commit();
});
});
}
private renderLLMModelField(parent: HTMLElement, profile: EnvironmentProfile): void {
const wrapper = parent.createDiv({ cls: 'rewrite-model-field' });
this.populateLLMModelField(wrapper, profile);
}
private populateLLMModelField(wrapper: HTMLElement, profile: EnvironmentProfile): void {
wrapper.empty();
const providerId = profile.llmProvider;
const provider = createLLMProvider(providerId);
const supportsList = typeof provider.listModels === 'function';
const cached = this.plugin.settings.modelCache.llm[providerId]?.ids ?? [];
const current = profile.llmConfig.model;
const setting = new Setting(wrapper).setName('LLM model');
setting.setDesc(modelFieldDesc(llmModelHint(providerId), supportsList, cached.length));
if (supportsList) {
setting.addDropdown((dd) => {
dd.addOption('', cached.length === 0 ? '(no cached models)' : '(pick a model)');
for (const id of cached) dd.addOption(id, id);
dd.setValue(cached.includes(current) ? current : '');
dd.onChange(async (v) => {
if (!v) return;
profile.llmConfig.model = v;
await this.commit();
this.populateLLMModelField(wrapper, profile);
});
});
setting.addExtraButton((b) => {
b.setIcon('refresh-cw').setTooltip('Refresh model list').onClick(async () => {
await this.refreshLLMModels(providerId, profile.llmConfig);
this.populateLLMModelField(wrapper, profile);
});
});
}
setting.addText((t) => {
t.setValue(current);
t.setPlaceholder(supportsList ? '' : llmModelHint(providerId));
t.onChange(async (v) => {
profile.llmConfig.model = v;
await this.commit();
});
});
}
private async refreshTranscriptionModels(
providerId: TranscriptionProviderID,
config: TranscriptionConfig,
): Promise<void> {
const provider = createTranscriptionProvider(providerId);
if (!provider.listModels) return;
try {
const ids = await provider.listModels(config);
this.plugin.settings.modelCache.transcription[providerId] = { ids, fetchedAt: Date.now() };
await this.commit();
new Notice(`ReWrite: refreshed ${ids.length} ${providerId} models.`);
} catch (e) {
new Notice(`ReWrite: refresh failed. ${e instanceof Error ? e.message : String(e)}`);
}
}
private async refreshLLMModels(
providerId: LLMProviderID,
config: LLMConfig,
): Promise<void> {
const provider = createLLMProvider(providerId);
if (!provider.listModels) return;
try {
const ids = await provider.listModels(config);
this.plugin.settings.modelCache.llm[providerId] = { ids, fetchedAt: Date.now() };
await this.commit();
new Notice(`ReWrite: refreshed ${ids.length} ${providerId} models.`);
} catch (e) {
new Notice(`ReWrite: refresh failed. ${e instanceof Error ? e.message : String(e)}`);
}
}
private renderTemplates(parent: HTMLElement): void {
new Setting(parent).setName('Templates').setHeading();
parent.createEl('p', {
@ -478,6 +588,12 @@ function generateTemplateId(): string {
return `tpl-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 8)}`;
}
function modelFieldDesc(hint: string, supportsList: boolean, cachedCount: number): string {
if (!supportsList) return hint;
if (cachedCount === 0) return `${hint} Or click Refresh to load models from the provider.`;
return `${hint} Pick from the dropdown, or type a custom model name.`;
}
function transcriptionModelHint(id: TranscriptionProviderID): string {
switch (id) {
case 'openai':

View file

@ -1,5 +1,5 @@
import { TranscriptionConfig } from '../types';
import { providerRequest } from '../http';
import { jsonGet, providerRequest } from '../http';
import { TranscriptionProvider } from './index';
interface DeepgramResponse {
@ -10,6 +10,10 @@ interface DeepgramResponse {
};
}
interface DeepgramModelsResponse {
stt?: Array<{ canonical_name?: unknown; name?: unknown }>;
}
export function createDeepgramTranscription(): TranscriptionProvider {
return {
id: 'deepgram',
@ -46,5 +50,22 @@ export function createDeepgramTranscription(): TranscriptionProvider {
}
return transcript.trim();
},
async listModels(config, signal) {
if (!config.apiKey) throw new Error('deepgram: API key is not configured');
const response = await jsonGet<DeepgramModelsResponse>(
'deepgram',
'https://api.deepgram.com/v1/models',
{ Authorization: `Token ${config.apiKey}` },
signal,
);
const seen = new Set<string>();
for (const row of response.stt ?? []) {
const id = typeof row.canonical_name === 'string'
? row.canonical_name
: typeof row.name === 'string' ? row.name : '';
if (id) seen.add(id);
}
return [...seen].sort();
},
};
}

View file

@ -13,6 +13,7 @@ export interface TranscriptionProvider {
config: TranscriptionConfig,
signal?: AbortSignal,
): Promise<string>;
listModels?(config: TranscriptionConfig, signal?: AbortSignal): Promise<string[]>;
}
export function createTranscriptionProvider(

View file

@ -1,11 +1,15 @@
import { TranscriptionConfig, TranscriptionProviderID } from '../types';
import { MultipartPart, multipartPost } from '../http';
import { jsonGet, MultipartPart, multipartPost } from '../http';
import { audioFilename, TranscriptionProvider } from './index';
interface ModelsListResponse {
data?: Array<{ id?: unknown }>;
}
export function createOpenAITranscription(
id: TranscriptionProviderID,
): TranscriptionProvider {
return {
const provider: TranscriptionProvider = {
id,
requiresAudio: true,
async transcribe(
@ -41,6 +45,36 @@ export function createOpenAITranscription(
return res.text.trim();
},
};
if (id !== 'openai-compatible') {
provider.listModels = async (config, signal) => {
if (!config.apiKey) throw new Error(`${id}: API key is not configured`);
const url = id === 'groq'
? 'https://api.groq.com/openai/v1/models'
: 'https://api.openai.com/v1/models';
const response = await jsonGet<ModelsListResponse>(
id,
url,
{ Authorization: `Bearer ${config.apiKey}` },
signal,
);
return filterAudioTranscriptionModels(response.data ?? []);
};
}
return provider;
}
function filterAudioTranscriptionModels(rows: Array<{ id?: unknown }>): string[] {
const out: string[] = [];
for (const row of rows) {
const id = typeof row.id === 'string' ? row.id : '';
if (!id) continue;
const lower = id.toLowerCase();
if (lower.includes('whisper') || lower.includes('transcribe')) out.push(id);
}
out.sort();
return out;
}
function resolveEndpoint(id: TranscriptionProviderID, config: TranscriptionConfig): string {

View file

@ -51,6 +51,16 @@ export type ActiveProfileOverride = 'auto' | 'desktop' | 'mobile';
export type ActiveProfileKind = 'desktop' | 'mobile';
export type RecordingFormatPreference = 'webm' | 'mp4';
export interface ModelCacheEntry {
ids: string[];
fetchedAt: number;
}
export interface ModelCache {
transcription: Partial<Record<TranscriptionProviderID, ModelCacheEntry>>;
llm: Partial<Record<LLMProviderID, ModelCacheEntry>>;
}
export interface GlobalSettings {
activeProfileOverride: ActiveProfileOverride;
desktopProfile: EnvironmentProfile;
@ -59,4 +69,5 @@ export interface GlobalSettings {
lastUsedTemplateId: string;
recordingFormat: RecordingFormatPreference;
templates: NoteTemplate[];
modelCache: ModelCache;
}