Expand description
Voice speech-to-text endpoint — POST /api/v1/voice/transcribe.
The renderer captures mic audio (getUserMedia, gated by the CEF permission handler #1602) and POSTs each silence-bounded utterance here; we forward it to the configured Whisper backend and return the transcript. Keys/paths stay server-side — the renderer never sees them.
Web Speech API can’t transcribe in CEF (closed-source Google service), so this capture-and-send path is the real engine. See docs/specs/SPEC_VOICE_STT_ENGINE_2026_06_20.md and #1591.
Backends (selected by the voice:engine setting):
- “groq” (default) — hosted whisper-large-v3-turbo; renderer sends webm.
- “whisper-local” — offline whisper.cpp via a local
whisper-clisubprocess; renderer sends 16 kHz mono WAV. Opt-in, configured via settings/env; needs a model file.
Structs§
Enums§
- SttError 🔒
- Transcription failure: distinguishes “not configured” (→ 501, surfaced as the frontend’s “unavailable” guidance) from a real upstream/runtime error (→ 502).
Constants§
- DEFAULT_
WHISPER_ 🔒MODEL - Transcribe via a local
whisper-cli(whisper.cpp) subprocess. The renderer sends 16 kHz mono WAV for this engine, so we write the body to a temp.wavand run the CLI. Fully offline; nothing leaves the machine. - GROQ_
MODEL 🔒 - GROQ_
TRANSCRIBE_ 🔒URL - MODEL_
DOWNLOAD_ 🔒TIMEOUT_ SECS - Cap the one-time model download so a stuck transfer can’t wedge a request.
Functions§
- ensure_
local_ 🔒model - Resolve the GGML model path, downloading an auto-managed model on first use.
- handle_
voice_ 🔒transcribe POST /api/v1/voice/transcribe?mime=audio/webm&lang=en— body is raw audio. → 200{ "text": "..." }· 400 empty body · 501 no backend configured · 502 upstream/subprocess error.- mime_
ext 🔒 - File extension matching the posted MIME, so Groq’s content sniffing accepts
the multipart
filepart. - read_
settings_ 🔒json - Read settings.json once;
Noneif absent/unparseable. - resolve_
engine 🔒 voice:engine— “groq” (default) or “whisper-local”. Env override:AGENTMUX_VOICE_ENGINE.- resolve_
groq_ 🔒key - Groq key:
AGENTMUX_GROQ_API_KEYenv, else settings.jsonvoice:groqApiKey. - resolve_
path 🔒 - Env-first, then settings.json, for a path-valued config key.
- settings_
str 🔒 - transcribe_
groq 🔒 - POST the audio to Groq’s OpenAI-compatible transcription endpoint.
- transcribe_
local_ 🔒whisper