AudioShake Launches Refinery to Turn Overlapping Conversations into AI Training Data

People interrupt. They laugh at someone else’s last sentence, offer a quick agreement before the speaker is finished, and continue talking while music or traffic fills the background. For a voice AI developer, this everyday confusion creates a difficult data problem: a recording can contain exactly the conversational behavior a model needs to learn, but arrive as a single mixed audio stream.

AudioShake targets this gap The refinerya newly launched system that converts existing recordings into structured, speaker-segregated data for AI training. Instead of asking developers to stage new conversations or produce synthetic examples, the company offers a way to extract individual voices and other sound components from recordings that organizations already have the rights to use.

The announcement puts audio separation closer to the center of the voice AI development process. The interesting question is whether datasets can preserve the time and complexity of a real conversation while becoming easier to label, inspect, and use.

Transforming a completed recording into separate speaker tracks

According to AudioShake’s announcement, The Refinery can split conversations into individual speaker tracks, separating dialogue from music and background sound. Works directly from recorded audio, without requiring the original recording session or separately captured stems. When two people talk at the same time, the goal is to retrieve their voices individually while keeping the exchange overlapping.

This is different from simply cleaning a recording until no dominant voice remains. An interruption can be important training information rather than unwanted noise. A brief acknowledgment can help communicate whether someone is listening, agreeing, or preparing to take a turn. Flattening an exchange into a single flow can make these behaviors more difficult to associate with the correct speaker.

A useful way to think of the output is as a set of aligned traces. One carries the voice of a particular speaker, another carries the voice of a second speaker, and their shared timing reveals when they overlap. The recording becomes easier to review without requiring rewriting of the conversation itself.

AudioShake says the system does not generate or reconstruct speech: the separate voices and corresponding frequencies come from the original recording. This distinction is important for developers looking for examples of real conversational behavior. Processing is intended to expose what has been recorded, rather than create a new performance of it.

Because the separation of speakers goes beyond diarization

AudioShake’s Multi-Speaker Separation product page describes a combination of speaker separation and diarization. Diarization identifies when multiple speakers are active; separation produces individual audio signals. Labeling a time slot as containing two speakers does not, in itself, provide the developer with two independently usable vocal tracks.

The product also distinguishes confidence in assigning audio to the right speaker from confidence in the quality of separation. These solve several problems: a voice can be assigned correctly but still contain another person’s sound. AudioShake describes the underlying system as acoustic, rather than dependent on a language model, and supports recordings with different sample rates and capture conditions.

For a training pipeline, separating these functions can make the review more precise. A team may need to check the speaker’s identity, the clarity of a particular track, or the accuracy of a subsequently generated transcript. Treating all three as a single success or failure would obscure where an error entered the data set.

Quality scores are part of the data pipeline

The Refinery evaluates results for quality and safety, allowing organizations to sort large collections into usable, repairable, or unsuitable material. This is an important operational feature. At the scale of a substantial audio archive, manually listening to every minute becomes a bottleneck, even if the separation itself is automated.

Scores can help direct human attention towards questionable segments. For example, a dataset team might prioritize a crowded conversation where a silent speaker becomes difficult to distinguish, rather than reviewing a simple recording of a single person with the same intensity.

There is also a trade-off to manage. Selecting only the simplest and clearest clips could produce a dataset that doesn’t include the difficult situations the project was intended to capture. In our evaluation, teams using this type of pipeline should evaluate both the quality of the output and the variety of conversation that survives filtering. Preserving challenging examples with careful review can be more useful than maximizing a single aggregate confidence score.

Preparing for training therefore involves more than just generating separate files. Developers still need to determine how quality tracks, transcripts, timing, speaker labels, and metadata fit into their particular learning objective.

What the AudioShake benchmark shows and its limitations

In its Multi-Speaker 2.0 technical evaluation, AudioShake reports a minimum permutation concatenated word error rate of 9.17% on LibriCSS, compared to 37.75% for the TF-Locoformer MERL checkpoint tested. Both were assessed through the same Whisper large-v3 transcription pipeline. Lower error rates indicate fewer transcription errors in that assessment.

Qualification is important: the basic checkpoint was tested outside its training scope. AudioShake explicitly frames this as a standard comparison, rather than proof that an architecture is inherently superior under matching training conditions. His assessment also finds that accuracy becomes more difficult to maintain as prolonged overlap and the number of speakers increase.

These are company-reported results, not an independent evaluation of each Refinery implementation. They support testing the technology on representative audio, rather than assuming that the title improvement will carry over unchanged to another dataset.

Separation quality and downstream model performance were also different. A cleaner training corpus might be valuable, but the rollout doesn’t dictate how much a particular conversation model will improve after learning from it. This requires a separate experiment, with the intended application and evaluation conditions defined.

From media workflows to AI data infrastructure

AudioShake brings expertise from music and media production. His company’s website describes audio separation for tasks such as mixing, localization, audio analysis and audiovisual editing and lists clients such as ESPN, Universal Music Group and Warner Bros. Studios. In these settings, separating elements from a finished mix can make existing material useful for another production workflow.

The Refinery applies a similar idea to AI development: making an existing recording more useful by exposing its components. AudioShake’s data services page also describes preparing datasets from customer content and developing specialized separation models for particular catalogs.

This does not make every media archive an appropriate training dataset. Organizations still need to identify which recordings fit their intended use and determine what they can do with the material. The ad specifically positions The Refinery around whether audio customers already have usage rights.

A practical test for voice AI developers

In its launch blog, AudioShake reports over 100 million minutes processed and says early versions have been deployed privately with frontier AI labs over the past year. It names Luel and Rime among its clients, along with unnamed data labs and marketplaces. The scale remains a company-reported figure.

According to the announcement, the refinery can process data via AudioShake’s API or be deployed on-premise. The latter option is intended to allow organizations to manage sensitive or proprietary recordings within their environments. Customers retain ownership of their data and output.

For a developer evaluating the system, a representative proof would matter more than a perfectly clean demonstration. Useful questions include whether silent speakers survive separation, whether breaks remain aligned, how much manual correction is needed, and whether the resulting examples improve the intended speech application.

AudioShake’s launch blog provides more information on its approach. The broader meaning of The Refinery is simple: real conversation contains useful structure that a mixed recording can hide. Restoring that framework could make existing audio a more practical resource for building speech AI that can handle how people actually speak.