VoxLock: the best performing voice isolator

VoxLock keeps the caller’s voice and removes the other voices in the room before speech-to-text hears them. It is made for voice agents: one input, the call audio, and it picks out the caller by itself.

The same 6.5 seconds of a held-out call, with VoxLock off and then on.

It has the lowest word error rate on the Krisp Voice Isolation Benchmark of every model tested, paid or open: 4.95%, with 0.11 background words leaked per minute. Below, it runs in your browser.

Demo#

The recordings are processed ahead of time with the published model. Your own recording runs through the same model in this tab with onnxruntime-web, a 50 MB download the first time.

Results#

265 real recordings from the Krisp Voice Isolation Benchmark: office, call center and phone calls with a second voice or noise in the room. Transcribed with Parakeet TDT 0.6B v2 and scored with the benchmark’s published method.

Model	Access	Word error rate %	Caller words missed %	Background words leaked / min
VoxLock	open	4.95	4.22	0.11
Krisp VI 2.5 Default	paid	5.30	4.21	1.52
Krisp VI 2.5 Balanced	paid	5.32	4.06	2.02
Krisp VI 2.5 HD	paid	5.79	4.37	2.74
Krisp VI 2.5 Lite	paid	6.21	4.26	4.52
ai-coustics Voice Focus 2.2 L	paid	6.84	6.03	0.66
ai-coustics Voice Focus 2.2 S	paid	6.91	6.01	1.00
Sanas Voice Isolation SE 1.0	paid	9.38	4.90	13.76
REAL-TSE causal (VoxLock’s starting checkpoint)	open	9.81	5.84	13.73
Hush	open	12.09	6.27	19.58
DeepFilterNet3	open	13.30	6.46	22.77
TIGER	open	14.72	3.96	37.24
No processing		23.21	4.28	66.48
SAM-Audio small (Meta)	open	33.77	27.98	21.15
ElevenLabs Voice Isolator**	paid	43.79**	4.06**	144.6**

• Word error rate: errors in the caller’s transcript after isolation.
• Caller words missed: caller words deleted or replaced.
• Background words leaked: words transcribed while only the background was talking, per minute of that audio.

** ElevenLabs Voice Isolator, run on the 7 of these recordings with the most background speech, not all 265. On the same 7, no processing scores 45.82% and 154.3 leaked words per minute, and VoxLock 2.98% and 0.0.

On 134 windows of real AMI meetings that are not in the training data, with the headset wearer as the caller:

Model	Access	Word error rate %	Caller words missed %	Background words leaked / min
VoxLock	open	10.12	6.77	2.12
ai-coustics Voice Focus 2.2 S	paid	11.85	8.67	1.06
ai-coustics Voice Focus 2.2 L	paid	12.13	8.88	1.72
Sanas Voice Isolation SE 1.0	paid	13.30	9.09	5.30
TIGER	open	22.84	9.74	36.85
DeepFilterNet3	open	26.78	13.10	38.84
Hush	open	33.00	11.89	68.93
No processing		33.03	8.05	78.34

Per-file results, intervals and the harness are in the repository.

How it works#

VoxLock is a causal band-split RNN target-speaker extractor, 27M parameters, fine-tuned from the REAL-TSE challenge baseline. It takes the loudest 3 seconds in the first 15 seconds of the call as the caller’s voice sample. The sample becomes frame-level speaker features that each frequency band attends to, and the network predicts a mask over the call’s spectrogram that keeps the matching voice.

It streams with 24 ms of algorithmic latency. With onnxruntime on 4 CPU threads of an Apple M2 Pro, it runs in real time from 40 ms chunks.

Training#

Four stages on a single NVIDIA L4, 9,200 steps in total:

1. Simulated calls: LibriSpeech callers mixed with one or two background talkers in simulated rooms, a quarter of them telephone band, with the voice sample cut from the mix.
2. Real crosstalk: AMI headset recordings from 40 meetings, where the other people in the room bleed into the wearer’s microphone.
3. Long calls: 30-second clips where the caller talks throughout.
4. Rooms and distances: callers close to or far from the microphone, background voices dry or in the room, and real room noise.

Use it#

pip install voxlock
voxlock call.wav -o caller.wav

import voxlock

vl = voxlock.VoxLock()                     # downloads the 75 MB model on first use
caller = vl.isolate(voxlock.load_audio("call.wav"))

stream = vl.live()                         # live: finds the caller by itself
for chunk in call_audio:
    send(stream.push(chunk))               # the isolated audio, about 24 ms behind

Live, VoxLock passes the audio through until it has heard 3 seconds of the caller, then isolates. With a sample saved from an earlier call, vl.stream(sample) isolates from the first chunk. For voice agents built on Pipecat, it is an audio filter: pip install "voxlock[pipecat]".

pip runs an 8-bit version of the model by default: 5.11% word error rate on the same benchmark, still ahead of every other model above, and real time on fewer CPU threads. The model is on Hugging Face as safetensors for PyTorch and as ONNX in float32, float16 and 8-bit. The repository has the PyTorch inference, the ONNX export, the benchmark harness and the training code. Apache-2.0.

Code · Model