Xiaomi releases CocktailASR-1, a speech recognition model built for noisy, multi-speaker rooms

Xiaomi has released and open-sourced Xiaomi-CocktailASR-1, a speech recognition model designed to handle one of the harder problems in audio transcription: picking out one person’s voice when multiple people are speaking at the same time.

Xiaomi calls this the “cocktail party problem.” Most speech recognition models work well when only one person is talking, but things get much harder when several voices overlap. This can lead to garbled text, merged sentences, or parts of a conversation being assigned to the wrong speaker. 

The challenge is similar to what Xiaomi addressed with its OmniVoice text-to-speech model, although CocktailASR-1 works in the opposite direction by isolating and transcribing speech rather than generating it.

How Xiaomi-CocktailASR-1 works

To use CocktailASR-1, you first need to provide a short audio clip of the person you want to track. The model uses that clip as a voice reference and then searches through a recording with multiple speakers to identify and transcribe only that person’s speech.

Xiaomi built CocktailASR-1 around an end-to-end LLM architecture. The company says it has achieved state-of-the-art results across several multi-speaker speech recognition benchmarks, outperforming existing approaches designed for the same task.

The model is not limited to noisy group conversations, either. With just one speaker, Xiaomi says CocktailASR-1 performs at around the same level as standard automatic speech recognition (ASR) models. This means users do not have to give up single-speaker performance to get better results in crowded audio.

It can also avoid making unnecessary guesses. If the selected speaker does not appear in a recording, the model returns empty text instead of trying to transcribe another person’s voice. 

CocktailASR-1 also includes a chain-of-thought reasoning mode, allowing users to inspect the reasoning associated with a transcription rather than only seeing the final text.

Xiaomi-CocktailASR-1 is the latest in a growing list of AI models that Xiaomi has open-sourced. It follows releases such as MiMo-V2-Flash and Xiaomi Robotics-0. The model is now available on GitHub and Hugging Face for developers to try and explore.

For daily updates, please visit our News Section.

Stay ahead in tech! Join our Telegram community and sign up for our daily newsletter of top stories!