Back to Journal Behind the Build

How We Trained Our AI on 10,000 Hours of Podcast Audio (and Why Privacy Came First)

Marcus Delgado
Abstract visualization of audio training data

When we started building Reverbwell, one of the first questions we had to answer was: where does the training data come from? For a lot of AI audio tools, that question gets a vague non-answer. "Publicly available data." "Licensed audio." We wanted to be specific about what we did and why, because the choices we made early define what the model can and can't do.

The short version: we trained on 10,000+ hours of podcast audio. None of it was customer recordings. Here is what that actually meant in practice.

The temptation of the easy path

There is a lot of podcast audio sitting on the internet. RSS feeds are public. Show notes are public. Audio files are technically downloadable. The scraping route would have been fast and cheap, and plenty of models in adjacent categories were built exactly that way.

We decided early that using public podcast audio without explicit licensing would be the wrong call, and not just for legal reasons. If we scraped podcasts to train the model, we would be using creators' work to build a tool that competes for the same editing time those creators spend. That felt like a bad deal for the people we were trying to help.

Beyond the ethics, there is a practical problem: publicly scraped audio is noisy in a way that hurts model quality. You get inconsistent microphone quality, unknown recording environments, variable loudness, and audio that has already been processed by other tools. Training on that teaches the model to generalize across chaos rather than to understand what good podcast audio sounds like at the source.

What we actually used

The 10,000 hours came from three main categories.

Licensed production audio. We worked with production houses that had extensive archives of raw and processed recordings, mostly from corporate podcast production work. These came with explicit licensing agreements covering AI training use. The audio covered interview formats, solo narration, roundtable discussions, and remote multitrack sessions. Importantly, we had access to both the raw files and the edited finals, which gave the model ground truth to train against.

Our own recording sessions. Jordan spent a lot of time in the Nashville area booking time at studios and with independent recording engineers to capture specific noise environments, microphone configurations, and recording conditions we knew we needed the model to handle. HVAC profiles, street noise bleed, laptop fan interference, room tone from untreated home studios. These sessions were targeted specifically at the edges of what the model would encounter in real production use.

Contributed internal audio. The three of us have recorded a combined amount of podcast content as hobbyists over the years. That went into the pool. It is not a large contribution to the total hours, but it added some natural human recording patterns that are hard to get from professional production audio.

Strict data isolation from customer recordings

We made a specific decision that customer recordings submitted through Reverbwell would never feed back into model training without explicit written consent. This was a policy decision before we had any customers, and it shaped how we architected the processing pipeline.

When a host uploads a recording to Reverbwell, it is processed in an isolated compute context. The edited output is returned, the intermediate processing artifacts are cleared, and the audio does not touch the training pipeline. The trained model is updated on a separate cycle using only the licensed and internal data described above.

This matters practically because a lot of podcast content is sensitive. Interview subjects say things off the record. Pre-interviews have personal context. Early takes include conversations hosts would not want made public. When you hand a recording to an editing service, you should not have to wonder whether that audio is being stored somewhere or used to train a model you did not agree to contribute to.

What training on good data looks like in practice

The difference between training on licensed production audio and training on scraped public audio shows up in specific ways. The most obvious is filler word detection. "Um" and "uh" are easy. The harder cases are the false positives: the thoughtful pause before a key point, the half-started sentence that a host corrects naturally, the verbal placeholder that is part of how a particular speaker communicates. Training on audio where we had human-edited ground truth meant the model learned to read context, not just to pattern-match on a sound.

Noise reduction is another area where data quality compounds quickly. Generic noise models trained on speech datasets tend to over-process: they hear room tone as noise and try to remove it, creating that characteristic "processed in a phone booth" artifact that makes heavy AI-cleaned audio so recognizable. Our noise profiles were built around the specific acoustic environments podcasters record in, not the clean-room conditions of a speech recognition dataset.

What this means for you

If you use Reverbwell, your recordings are not training data. The model does not improve by processing your content. The processing is a one-way operation: audio in, edited audio out, no retention.

We publish this because we think it matters, and because "AI-trained responsibly" is an easy claim to make without substance. The substance is the licensing agreements, the isolated compute architecture, and the policy choice we made before we had a single customer to pressure us in either direction.

The model will keep getting better as we continue the internal recording program and expand the licensed archive. That work is ongoing. What will not change is the boundary around customer audio.

Spend less time editing, more time recording.

Reverbwell handles the repetitive part. Free to start.

Try it free