LiquidCodec’s 361 Encode Rate on a Single CPU Thread — Dual-Track WASAPI Still Sets the Real Overlay Floor

On 12 November 2026 Liquid AI published LiquidCodec, the ~74 M-parameter streaming audio tokenizer that sits under LFM3-Audio-2B and LFM3-Omni-2B. It turns 24 kHz speech into 12 discrete tokens per 80 ms frame (12.5 Hz) using grouped finite scalar quantization, then reconstructs the waveform. On a single Apple M4 Max CPU thread at FP32 the encoder runs at roughly 361× real time (RTF 0.0028) and the decoder at 87×. GGUF builds (Q4_0, Q4_K_M, Q8_0) are already available for llama.cpp. It also posts the best ViSQOL and lowest mel distance of the codecs they compared on eight reconstruction sets.

That number is real and useful for anyone trying to keep a full speech stack on-device. It does not move the first millisecond of a live Windows overlay that has to hear both sides of a call, turn the remote party’s audio into text, and paint a translation or suggestion before the next sentence finishes.

The capture tax still dominates

A 74 M codec that encodes in a few hundred microseconds per second of audio is almost free once the samples are in memory. Getting those samples cleanly and with stable timing is not:

1. Endpoint choice — USB headset, Bluetooth HFP/A2DP, built-in, or exclusive-mode WASAPI.
2. Dual-track acquisition: a dedicated capture client for the local mic plus a render-loopback client so the remote party’s audio stays distinct.
3. Windows Audio Engine (audiodg.exe + audioeng.dll) in shared mode. Mixing, communications enhancements and vendor APOs routinely add 10–40+ ms and become non-deterministic under load.
4. Consumer thread: lock-free SPSC ring copy, format conversion, resampling, VAD.
5. Streaming STT (local GGUF or remote) plus optional local model polish for translation or short replies.
6. Overlay paint on a dedicated thread that never blocks the audio callback.

Bluetooth HFP alone can inject another 50–150 ms of buffering that no tokenizer speed recovers. Even if the codec and the language model both run locally and finish in well under a frame, the overlay still waits for the Windows path that delivered the samples.

A minimal shared-mode capture callback that stays under 1 ms and does no allocation looks like this:

// WASAPI shared-mode capture — callback must be allocation-free
static HRESULT STDMETHODCALLTYPE CaptureCallback(IAudioCaptureClient *pCapture) {
    UINT32 packetLength = 0;
    BYTE *pData = NULL;
    DWORD flags = 0;
    HRESULT hr = pCapture->GetBuffer(&pData, &packetLength, &flags, NULL, NULL);
    if (FAILED(hr) || !packetLength) return hr;

    // Writer side of a pre-allocated power-of-two SPSC ring (float32 frames)
    ring_push_float32(capture_ring, (const float *)pData, packetLength);
    pCapture->ReleaseBuffer(packetLength);
    return S_OK;
}

The consumer owns resampling (libsamplerate or a fixed polyphase), VAD, and the write to the model or WebSocket. Keep the audio thread pure; any lock or malloc here shows up as glitches or higher DPC latency.

Dual-track still matters more than tokenizer speed

A meeting bot that joins the call receives mixed audio after the conferencing platform has already done its own mixing and jitter buffering. A local overlay that needs speaker-separated, low-latency text has to start earlier. The remote-side track can be sent straight to STT (with or without the new codec as a front-end). The mic track can stay separate for local voice activity or for a second model that only listens for barge-in.

LiquidCodec’s causal encoder and symmetric decoder make it a candidate for that front-end if you want to stay entirely on-device and still feed a small language model discrete tokens instead of raw spectrograms. The 12.5 Hz frame rate already matches the kind of chunking most streaming STT pipelines use. The Windows side of the budget does not change: you still need the two WASAPI clients, the lock-free rings, and an overlay that can paint without waiting for the next audio quantum.

Practical knobs that actually move the floor

• Prefer exclusive-mode WASAPI or a dedicated USB device when the call app allows it. Shared mode plus APOs is the common source of the extra 20–40 ms.
• Keep the capture callback allocation-free and under a millisecond. Profile with ETW + WPR if the overlay starts stuttering.
• Size the SPSC rings for the worst-case STT chunk plus one quantum of jitter, not for “whatever feels safe.”
• Run VAD and format conversion on the consumer thread so the audio engine never sees a slow callback.
• If you add LiquidCodec (or any other tokenizer) as a local front-end, pin it to the same process as the overlay only after you have measured that the capture path itself is stable. A 361× encoder is useless if the samples arrive 80 ms late.

The codec result is a real improvement for on-device speech pipelines. The part that still decides whether a language overlay feels instantaneous on a Windows laptop is the same dual-track WASAPI path it has been for the last several years.

I'm building Talkaside, a Windows app for job interviews and calls in another language.
Drafted with AI assistance from this project's code and logs.