mirror of
https://github.com/wiseguru/ReWrite-Voice-Notes.git
synced 2026-07-22 07:49:19 +00:00
improve whisper security
This commit is contained in:
parent
5e98637c76
commit
c9051a52ef
4 changed files with 45 additions and 5 deletions
|
|
@ -121,6 +121,8 @@ Per-adapter behavior: [src/transcription/assemblyai.ts](src/transcription/assemb
|
|||
|
||||
`start()` validates the binary and model paths exist via `fs.existsSync`, probes the port via `net.createServer().listen(port)` to detect conflicts, spawns the user's whisper-server with `child_process.spawn`, captures stdout/stderr into a 1 MB ring buffer, then polls `net.createConnection` against the port every 250 ms for up to 5 s before declaring `'running'`. Any failure transitions status to `'crashed'` with the log tail surfaced in the error message. `stop()` sends SIGTERM, waits up to 3 s, then SIGKILL.
|
||||
|
||||
**Loopback binding is enforced.** whisper-server has no auth/TLS, so `start()` pins `--host 127.0.0.1` when the user did not supply a `--host` in `extraArgs` (so we don't depend on upstream's default binding), and throws before spawning if a user-supplied `--host` is non-loopback (`getHostArg` + `isLoopbackHost` in [src/whisper-host.ts](src/whisper-host.ts); loopback = `127.0.0.1` / `localhost` / `::1`). This prevents a stray `--host 0.0.0.0` from silently exposing an unauthenticated transcription server to the LAN. The README's "Network exposure" disclosure and the Extra args field description both state this.
|
||||
|
||||
The `whisper-local` transcription provider ([src/transcription/whisper-local.ts](src/transcription/whisper-local.ts)) is a thin shim that POSTs to `http://127.0.0.1:<port>/inference` (whisper.cpp's native server route, not OpenAI's `/v1/audio/transcriptions`). Before sending, the recorded blob is transcoded to 16 kHz mono 16-bit PCM WAV via `transcodeToWavPcm` in [src/audio-transcode.ts](src/audio-transcode.ts) (Web Audio API: `AudioContext.decodeAudioData` → `OfflineAudioContext` for resample/downmix → hand-written RIFF header). whisper.cpp's server cannot decode WebM/Opus (the MediaRecorder default) without a custom ffmpeg-enabled build, so the transcode is mandatory. The same helper is reused by [src/transcription/mistral-voxtral.ts](src/transcription/mistral-voxtral.ts) (Voxtral also rejects WebM). If you add a third audio-uploading provider that needs WAV, import from `audio-transcode.ts` rather than duplicating the helpers. The shim grabs the host reference via `bindWhisperHost(host)` called from [src/main.ts](src/main.ts) on load. If the host status is anything other than `'running'`, the adapter throws a `ProviderError` with a "start it from settings" message; the pipeline surfaces that as a Notice. No API key is collected for this provider (no auth, no settings field).
|
||||
|
||||
Mobile compatibility: `WhisperHost` and the `whisper-local` option are guarded everywhere by `Platform.isDesktop`. Node modules (`child_process`, `net`, `fs`) are lazy-required inside the `Platform.isDesktop` branch, mirroring the [src/secrets.ts](src/secrets.ts) `getSafeStorage` pattern; on mobile the host is inert and the provider option is filtered out of dropdowns in [src/settings/tab.ts](src/settings/tab.ts) and [src/ui/setup-card.ts](src/ui/setup-card.ts).
|
||||
|
|
@ -258,7 +260,7 @@ Per [.editorconfig](.editorconfig): tabs (width 4), LF, UTF-8, final newline. Ma
|
|||
- **`secrets.json.nosync` uses the `.nosync` suffix on purpose**: iCloud Drive natively skips any file or folder whose name ends in `.nosync`. The README documents per-tool sync exclusion for other tools.
|
||||
- **Per-provider recording limits live in [src/transcription/limits.ts](src/transcription/limits.ts), not the recorder.** [src/recorder.ts](src/recorder.ts) does not cap recordings at any size; `validateRecording(blobSize, durationMs, providerId)` runs in [src/pipeline.ts](src/pipeline.ts) between the `persist-audio` and `transcribe` stages, throwing a friendly provider-attributed error if the recording exceeds the documented byte or duration ceiling. Both modal and Quick Record thread the recorder's `durationMs` onto the `audio` pipeline source so the duration check has data; the reprocess flow ([src/ui/audio-source.ts](src/ui/audio-source.ts)) omits `durationMs` (no cheap way to measure an arbitrary vault file), so reprocess only triggers the byte check. Limits source: `openai`/`groq` 25 MB, `assemblyai` 5 GB/10 h, `deepgram` 2 GB, `revai` 2 GB/17 h, `mistral-voxtral` 1 GB/30 min, `openai-compatible`/`whisper-local`/`webspeech` no client-side cap.
|
||||
- **WhisperHost lazy-requires Node modules** the same way [src/secrets.ts](src/secrets.ts) does for `safeStorage`. Importing `child_process` / `net` / `fs` at module top would crash on mobile load. The cached `nodeApiCache` is `null` on mobile, so any host method that needs it bails with a clear "desktop only" error.
|
||||
- **`splitArgs` is a naive whitespace split.** The local-whisper `extraArgs` value is tokenized on whitespace only, so a single argument containing spaces (e.g. a quoted path) is not supported. Not a security issue (argv array, no shell). The Extra args settings field description states the limitation; add quote-aware parsing only if users actually hit it.
|
||||
- **`splitArgs` is a naive whitespace split.** The local-whisper `extraArgs` value is tokenized on whitespace only, so a single argument containing spaces (e.g. a quoted path) is not supported. Not a security issue (argv array, no shell). The Extra args settings field description states the limitation; add quote-aware parsing only if users actually hit it. The tokenized list is also scanned by `getHostArg` so a non-loopback `--host` is rejected before spawn (see Local whisper.cpp host section).
|
||||
- **Async settings-tab buttons are concurrency-guarded.** Handlers that run an async op then `this.display()` (Populate buttons, whisper Start/Stop/probe, Lock) are wrapped in `runGuardedButton(b, fn)` in [src/settings/tab.ts](src/settings/tab.ts), which disables the button for the duration so a rapid double-click cannot launch the work (and its full-container re-render) twice. The encryption-mode dropdown uses a `modeChangeInFlight` flag for the same reason. Wrap new async-then-display buttons the same way. Each handler keeps its own try/catch; observability `console.error('ReWrite: <context>', e)` sits alongside the user-facing Notice in those catches (and in the swallowed catches in [src/main.ts](src/main.ts)).
|
||||
- **WhisperHost has three ownership states.** `'spawned'` (this session's child handle is live; log capture works), `'adopted'` (port bound, PID sidecar matches a live PID — we started it in a previous session, no log capture), `'external'` (port bound but no sidecar match — someone else started it). Status enum gains `'external'` alongside `stopped`/`starting`/`running`/`crashed`; `running` means we own it (spawned OR adopted) and Stop works, `external` means we don't and Stop is disabled. `WhisperHost.snapshot()` returns `{ status, baseUrl, ownership, pid }` for UI consumers; `formatWhisperStatus(snap)` produces labels like "Running on http://... (adopted from previous session, pid 12345)." or "External whisper-server on http://... (not started by ReWrite).".
|
||||
- **Transcription never checks `WhisperHost.status()`.** [src/transcription/whisper-local.ts](src/transcription/whisper-local.ts) just asks for `host.baseUrl()` and POSTs. The HTTP API doesn't care who owns the process — if the port is reachable, transcription works. `baseUrl()` returns the URL for both `'running'` and `'external'` states.
|
||||
|
|
|
|||
|
|
@ -54,6 +54,8 @@ If you want fully on-device transcription with no network calls, the plugin can
|
|||
|
||||
When you click Start in settings, the plugin launches whisper-server as a child process and communicates with it over loopback (`http://127.0.0.1:<port>`). The process is captured in a ring-buffered log you can view in settings. When you click Stop, or when the plugin is unloaded, the process is terminated. No code is downloaded or executed beyond the binary you provide.
|
||||
|
||||
**Network exposure**: whisper-server has no authentication and no TLS, so anyone who can reach its port can submit audio and exercise its native audio-decoding code. To keep it private, ReWrite always passes `--host 127.0.0.1` (loopback only) and **refuses to start** if you put a `--host` pointing at a non-loopback interface (such as `0.0.0.0` or a LAN IP) in Extra args. If you run whisper-server yourself from a terminal instead of letting the plugin manage it, bind it to `127.0.0.1` the same way; do not expose it to your network unless you have put your own authenticating proxy in front of it.
|
||||
|
||||
### Setup
|
||||
|
||||
1. Obtain a `whisper-server` binary:
|
||||
|
|
@ -99,7 +101,7 @@ whisper.cpp doesn't publish prebuilt Linux binaries, so you need to compile it o
|
|||
|
||||
Copy or symlink it somewhere stable (e.g. `~/.local/bin/whisper-server`) if you want to keep the absolute path short. Make sure it is executable: `chmod +x build/bin/whisper-server`.
|
||||
|
||||
4. Use that absolute path as Binary path in the plugin's "Local whisper.cpp server (desktop)" settings section. To sanity-check the binary outside the plugin first, run it once from a terminal with a model file: `./build/bin/whisper-server -m /path/to/model.bin --port 8080`. You should see `whisper server listening at http://127.0.0.1:8080`. Hit Ctrl-C to stop, then let the plugin manage it from there.
|
||||
4. Use that absolute path as Binary path in the plugin's "Local whisper.cpp server (desktop)" settings section. To sanity-check the binary outside the plugin first, run it once from a terminal with a model file: `./build/bin/whisper-server -m /path/to/model.bin --host 127.0.0.1 --port 8080`. You should see `whisper server listening at http://127.0.0.1:8080`. Pass `--host 127.0.0.1` so the unauthenticated server stays bound to loopback and is not reachable from your network. Hit Ctrl-C to stop, then let the plugin manage it from there.
|
||||
|
||||
If `cmake --build` fails with `error: 'std::filesystem' has not been declared` or similar C++17 errors, your distro's default GCC is too old. Install a newer one (`sudo apt install g++-12` on Ubuntu) and rerun the `cmake -B build ...` step with `-DCMAKE_CXX_COMPILER=g++-12` appended.
|
||||
|
||||
|
|
@ -125,12 +127,14 @@ The published checkpoints are quantized to `q8_0` and ship in the same GGML cont
|
|||
|
||||
2. In ReWrite settings under "Local whisper.cpp server (desktop)", set Model path to the absolute path of the FUTO `.bin` you just downloaded. Binary path and Port are unchanged from the standard setup above.
|
||||
|
||||
3. Expand the "Advanced" disclosure inside that section and set Extra args to:
|
||||
3. Set Extra args (in the "Local whisper.cpp server (desktop)" section) to:
|
||||
|
||||
```
|
||||
-ac 768
|
||||
```
|
||||
|
||||
Do not add a `--host` here pointing at a non-loopback interface; ReWrite binds the server to `127.0.0.1` and will refuse to start otherwise (the server is unauthenticated).
|
||||
|
||||
`-ac` (alias `--audio-context`) caps the encoder context at the given number of mel frames. Lower values run faster but only stay accurate on ACFT-finetuned models, which is the whole point of using them. A few starting points:
|
||||
|
||||
- `-ac 768` is a sensible default for short to medium clips (roughly up to ~15 s). Drop to `-ac 512` for short voice memos under ~10 s.
|
||||
|
|
|
|||
|
|
@ -699,7 +699,7 @@ export class ReWriteSettingTab extends PluginSettingTab {
|
|||
|
||||
new Setting(parent)
|
||||
.setName('Extra args')
|
||||
.setDesc('Space-separated CLI args appended after -m, --port. Split on whitespace only, so a single value containing spaces (such as a quoted path) is not supported.')
|
||||
.setDesc('Space-separated CLI args appended after -m, --port. Split on whitespace only, so a single value containing spaces (such as a quoted path) is not supported. The server has no authentication; ReWrite always binds it to 127.0.0.1 and refuses to start if a --host here points at a non-loopback interface.')
|
||||
.addText((t) => {
|
||||
t.setValue(cfg.extraArgs);
|
||||
t.onChange(async (v) => {
|
||||
|
|
|
|||
|
|
@ -208,10 +208,21 @@ export class WhisperHost {
|
|||
this.logBuffer = '';
|
||||
this.stoppingDeliberately = false;
|
||||
|
||||
const extra = splitArgs(config.extraArgs);
|
||||
// The whisper.cpp server has no authentication or TLS. Refuse to bind it
|
||||
// to anything other than loopback so a stray --host in Extra args cannot
|
||||
// silently expose an open transcription service to the LAN, and pin
|
||||
// 127.0.0.1 ourselves when the user did not specify a host (so we don't
|
||||
// rely on upstream's default binding staying loopback).
|
||||
const hostArg = getHostArg(extra);
|
||||
if (hostArg !== null && !isLoopbackHost(hostArg)) {
|
||||
throw new Error(`Refusing to start: --host ${hostArg || '(empty)'} would bind whisper-server to a non-loopback interface, exposing an unauthenticated transcription server to your network. Remove it from Extra args; ReWrite always binds 127.0.0.1.`);
|
||||
}
|
||||
const args = [
|
||||
'-m', config.modelPath,
|
||||
'--port', String(port),
|
||||
...splitArgs(config.extraArgs),
|
||||
...(hostArg === null ? ['--host', '127.0.0.1'] : []),
|
||||
...extra,
|
||||
];
|
||||
const child = api.cp.spawn(config.binaryPath, args, {
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
|
|
@ -436,6 +447,29 @@ function splitArgs(s: string): string[] {
|
|||
return trimmed.split(/\s+/);
|
||||
}
|
||||
|
||||
// Find the value of a --host argument in an already-tokenized arg list.
|
||||
// Supports both `--host 127.0.0.1` and `--host=127.0.0.1`. Returns the value
|
||||
// (possibly empty string) when present, or null when no --host is given.
|
||||
function getHostArg(args: string[]): string | null {
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (a === '--host') {
|
||||
return args[i + 1] ?? '';
|
||||
}
|
||||
if (a !== undefined && a.startsWith('--host=')) {
|
||||
return a.slice('--host='.length);
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Whether a host value binds only the loopback interface. Anything else
|
||||
// (0.0.0.0, a LAN IP, a hostname) would expose the unauthenticated server.
|
||||
function isLoopbackHost(host: string): boolean {
|
||||
const h = host.trim().toLowerCase();
|
||||
return h === '127.0.0.1' || h === 'localhost' || h === '::1' || h === '[::1]';
|
||||
}
|
||||
|
||||
function isPortInUse(net: NetAPI, port: number): Promise<boolean> {
|
||||
return new Promise((resolve) => {
|
||||
const server = net.createServer();
|
||||
|
|
|
|||
Loading…
Reference in a new issue