Two voices, one moment

November 1, 2026

When two people talk at the same time, a dubbing system has to decide who said what. For years, speaker-detection models asked one question per person, "is A speaking? is B speaking?", and then needed a hand-tuned threshold to decide.

Plaquet and Bredin (Interspeech 2023) asked a different question: which of these situations is it? A alone, B alone, or A and B together. Making "both at once" a case of its own improved results mostly on overlapping speech, made models sturdier on unfamiliar recordings, and removed that threshold. The method is now part of the open-source pyannote.audio toolkit.

Still, no model gets every overlap right. That's why our adjustment page lets you mark "two people talking at once" yourself.

What you can correct in a dub, and how: Dubbing you can correct

Want your video dubbed into another language? Give it a try, for free: velotrasvideo.com

Source: Powerset multi-class cross entropy loss for neural speaker diarization — Alexis Plaquet, Hervé Bredin, Interspeech 2023