ElevenLabs Scribe v3 Live Cuts Semantic Errors 43% — Dual-Track WASAPI Still Sets the Latency Floor

On 10 November 2026 ElevenLabs shipped Scribe v3 and Scribe v3 Live. The batch model claims 27% fewer word errors than Scribe v2 across 102 languages. The realtime variant, Scribe v3 Live, claims 43% lower semantic word-error rate versus the previous realtime model, first-time realtime speaker diarization, context-aware turn detection instead of pure silence VAD, and mid-sentence language switching. Pricing stayed flat at roughly $0.22 per audio hour for the live endpoint. Model IDs are scribe_v3 and scribe_v3_live; the WebSocket lives at the usual realtime speech-to-text endpoint. Latency numbers in the public material hover around the 150 ms class that has become the marketing floor for cloud streaming STT.

Those gains are real once clean PCM is already at the model. They do not move the capture budget on a Windows desktop when you need a private translation overlay for an interview or support call.

What a better cloud STT still cannot see

Scribe v3 Live (and every other hosted streaming model) receives the stream you send it. It does not sit on your local audio endpoints. It does not give you a second, private track of your own microphone that never enters the call. It does not let you paint a translation overlay that is invisible to screen share. And it does not run when the network is hostile or when you simply refuse to ship the audio off-box.

The practical path for a language-aid overlay is still dual-track WASAPI capture: one client on the loopback/render device that hears the remote party, one client on the capture device that hears you, both in shared mode so the rest of the machine keeps working, both with the callback kept empty so you do not add scheduling jitter.

Dual-track capture architecture

Open two IAudioClient instances:

1. Loopback on the default render endpoint (or the endpoint the call app is using). This is the remote audio.
2. Capture on the default (or preferred) input endpoint. This is your microphone.

Both in shared mode (AUDCLNT_SHAREMODE_SHARED). Periodicity around 10 ms is a reasonable target; the engine will quantize to the mix format. Use AUDCLNT_STREAMFLAGS_LOOPBACK only on the render client. Request a format you can actually deliver (usually 48 kHz float32 or the device mix format via GetMixFormat).

The capture callback (IAudioCaptureClient::GetBuffer) must return immediately. Copy the packet into a lock-free ring buffer and signal a worker. Any work inside the callback—allocation, logging, STT submission—adds jitter that shows up as gaps or increased latency.

Align the two streams by device clock or by a shared monotonic timestamp. Drift is inevitable; you correct it by dropping or duplicating a few frames on the slower side rather than resampling in the hot path unless you have a measured need.

From there the remote track goes to STT (cloud or local). The mic track can go to a separate STT stream, to a local model for your own speech, or simply be used for energy-based barge-in detection. The translation result is painted on a layered window that is excluded from screen capture (SetWindowDisplayAffinity with WDA_EXCLUDEFROMCAPTURE on Windows 10+).

Latency budget that actually matters

A 150 ms cloud STT time-to-final is interesting only after you have already spent:

• 10–40 ms in the shared-mode WASAPI engine and buffering (measured, not guessed).
• Packet alignment and any small resampling.
• Network RTT + model compute for the cloud path.
• Translation and UI paint.

Local models (faster-whisper, Whisper.cpp, Qwen3-ASR GGUF variants, LFM3-audio through Ollama, etc.) remove the network hop and keep the audio on-device. They trade accuracy on hard accents or domain terms for lower and more predictable latency. Hybrid is common: local for the hot path and partials, cloud for finalized turns when the network is clean.

Scribe v3 Live’s new context-aware turn detection and realtime diarization are useful when you are already sending a single mixed stream. They do not replace the ability to keep the two tracks separate so you can bias one stream with the remote party’s language and the other with yours, or so you can run different models on each side.

Code sketch (WASAPI loopback + capture)

// Pseudocode outline — real code needs COM init, error handling, format negotiation
IAudioClient* loopbackClient = nullptr;
IAudioClient* captureClient = nullptr;
// ... ActivateAudioInterface / GetDefaultAudioEndpoint ...

// Loopback
loopbackClient->Initialize(
    AUDCLNT_SHAREMODE_SHARED,
    AUDCLNT_STREAMFLAGS_LOOPBACK | AUDCLNT_STREAMFLAGS_EVENTCALLBACK,
    hnsPeriod, 0, &mixFormat, nullptr);

// Capture
captureClient->Initialize(
    AUDCLNT_SHAREMODE_SHARED,
    AUDCLNT_STREAMFLAGS_EVENTCALLBACK,
    hnsPeriod, 0, &mixFormat, nullptr);

// Start both, then in the event callback:
// GetBuffer -> copy to ring -> ReleaseBuffer
// Signal worker thread that owns STT submission

Measure the actual period with IAudioClient::GetCurrentPadding and ETW (Microsoft-Windows-Audio) or a simple high-resolution timer around the callback. Process Lasso’s LatencyTest or raw ETW still surface the real floor.

Trade-offs that still matter

• Cloud STT vs local. Scribe v3 Live (or Soniox, Deepgram Nova, etc.) wins on accuracy for noisy, accented, or domain-heavy speech once the audio is clean. Local checkpoints remove the network variable and the data-egress question. Most production overlays do both.
• Shared-mode vs exclusive. Shared mode is what users already have open. Exclusive mode can drop latency but breaks other apps and is brittle across driver updates. Production overlays stay in shared mode and live with the 10–40 ms floor.
• Single mixed stream vs dual track. A single stream is simpler for pure cloud agents. Dual track gives you private mic audio that never enters the call, independent biasing, and the ability to overlay only the remote-side translation.
• Overlay visibility. Paint on a layered window excluded from capture. That is the difference between a private language aid and a bot that everyone on the call can see.

Scribe v3 Live is a solid accuracy step for anyone already in the ElevenLabs realtime path. For the developer who needs the language barrier gone on their own machine, without joining a bot and without leaking the overlay, the work is still the same: open two WASAPI clients, keep the callback empty, align the tracks, and feed a tight STT/translation path. The model improvements keep arriving. The capture budget does not move until you measure and tighten it yourself.

I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.