Lowest word error rate on the Krisp Voice Isolation Benchmark
VoxLock
A voice isolation model for voice agents. It keeps the caller's voice and removes the other voices in the room before speech-to-text hears them. It takes one input, the call audio, and picks out the caller by itself from the first seconds of the call.
- Result
- 4.95% word error rate on 265 real recordings, with 0.11 background words leaked a minute. Krisp VI 2.5 scores 5.30% and leaks 1.52.
- Size
- 27M parameters · 75 MB as the 8-bit build · real time from 40 ms chunks on 4 CPU threads
- Training
- Fine-tuned from the REAL-TSE baseline on about 100 hours of simulated calls and 87 hours of real meeting crosstalk from AMI
- Runs on
- CPU with onnxruntime, as a Pipecat audio filter, or in the browser with onnxruntime-web
- Links
- Live demo · Hugging Face · GitHub · PyPI
The demo
The video above is a recording. The live demo is on the write-up page. Pick a call with someone talking over the caller and flip the switch while it plays: off is the call as recorded, on is VoxLock, with what speech-to-text hears under each. Or record up to 15 seconds of your own voice and mix in a second talker. It runs through the same model in your tab, a 50 MB download the first time.
How it compares
On the Krisp benchmark the word error rate is level with Krisp's best model; the gap is within noise. The difference is what leaks through: Krisp passes 1.52 background words a minute to the transcript and VoxLock 0.11. On 134 windows of real AMI meetings that are not in the training data, it scores 10.12% against 11.85% for the best ai-coustics model.
The full write-up has every model's results, how the model works, the training stages and the code to use it.