Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×

Diffio AI Is a Speech Restoration Model

It’s intended for “the recording you can’t go back and redo”

A new tool called Diffio AI restores speech in audio files, “for interviews, podcasts and lectures recorded in rooms that weren’t quiet … Your guest recorded from a kitchen. The interview happened in a lobby. Diffio restores the speech in the file you’ve got, because you can’t go back and record it again.”

We asked founder Nathan Harmon about it by email. He’s based in Colorado Springs, Colo.

Nathan Harmon (courtesy Diffio).
Nathan Harmon (courtesy Diffio).

Radio World: What is Diffio AI?

Nathan Harmon: Diffio AI is a speech restoration tool. You give it a recording of someone talking, audio or video, and it takes out the background noise, hiss and hum, reduces room echo, and repairs supported clipping and distortion. It runs in a web browser and through a developer API. 

In the web app you get the restored speech and the background as two separate tracks, so you decide how much of the room comes back. With video, the picture stays as it was and only the soundtrack is restored.

RW: Who are the target users?

Harmon: People who already have a recording and can’t make it again: podcasters, video producers, newsrooms and field reporters, educators with recorded lectures, archivists and developers who want speech cleanup inside their own products. 

One of our early users is a retired lawyer and part-time college professor who recorded a lecture series on cassette in the 1980s and is now putting it back online.

RW: You had an interesting application involving a 1935 broadcast. What was that?

Harmon: In 1935, two years before she disappeared over the Pacific, Amelia Earhart gave a radio talk known as “A Woman’s Place in Science.” 

The recording survives, but like most radio audio of that era it is hard work to listen to: a steady bed of hiss, crackle on top, and a voice that sounds as if it’s coming through a wall. 

I wanted the demo on our homepage to be the kind of recording nobody can make again, so we restored that broadcast with Diffio and put the original and the restored version side by side. 

Here is the original:

Here is the restored version:

The Daily Mail wrote about it on Oct. 24. Since then we’ve done the same with other public-domain recordings from the Library of Congress National Jukebox and the Nation’s Forum series: Theodore Roosevelt, Woodrow Wilson and William Howard Taft in the 1912 election; General Pershing from France in 1918; and President Harding’s 1921 address at Hoboken for the return of 5,212 American war dead.

RW: What is your background and what brought you to design Diffio AI?

Harmon: I spent 10 years building signal-processing systems for defense and commercial teams, most recently RF geolocation at Northrop Grumman. 

Finding a weak signal under noise is the same job whether the signal is a transmitter or a voice. I founded Diffio in 2025 in Colorado Springs. It’s bootstrapped, and I still do the work myself, from training the models to running the machines that serve them.

The Diffio web app, with the original/restored toggle and the background slider (courtesy Diffio).
The Diffio web app, with the original/restored toggle and the background slider (courtesy Diffio).

RW: How does your technology work?

Harmon: Diffio uses machine learning models trained to tell a speaking voice apart from everything around it. The model separates the voice from the noise, echo and distortion, and returns the speech clean, with the rest on its own track. 

There are two models: Flash makes one restoration pass and runs about 30 times faster than real time; Pro restores every section three times, keeps the cleanest result, and runs about 10 times faster than real time. 

Before and after spectrogram of a 2011 NHL Draft media scrum recorded on camera audio, restored with Diffio 4.6 Pro. (Image by Diffio. Audio: TermiteMound, CC BY 3.0)

It’s built to stay faithful to the recording: the same person, the same voice and what they said. The original always stays in the project, and you can flip between original and restored while it plays.

RW: You told me that the model works better for some applications than others. 

Harmon: It does best on a single English-speaking voice buried under steady noise: hiss, hum, HVAC, traffic, a crowd, room echo. That covers a lot of archive tape and field audio. 

It’s weaker in a few places, and I’d rather your readers hear that from me:

  • Phone and remote-call audio. In our test of six speech-cleanup tools on 59 real recordings (Diffio 4.7 ran Nov. 3, the other tools Oct. 22), Diffio had the largest quality gain on calls, but a speech recognizer made more errors on our restored calls (4.3% of words on Flash, 5.0% on Pro) than on the untouched originals (3.8%). On narrow-band phone audio, the original is often already as intelligible as it gets.
  • Speech over music. Other tools beat us on narration over music in that test.
  • Two people talking at once, a voice far from the mic, heavy clipping, and noise louder than the voice are the hardest cases.
  • It works best in English. Other languages run, but quality varies.

We check our historical restorations word by word against the original. 

On Harding, that check found one garbled phrase after we published it, and the page now carries a dated correction. Across the 4 minute 25 second Harding record, a speech recognizer disagreed with the published text of the speech on 9.4% of words for the original disc and 3.9% for our restoration. That’s why I tell anyone using it for archives to keep the original as the reference copy and check quotes against it.

RW: What is your business model, how do you make money from the product?

Harmon: Subscriptions to the web app, and pay-as-you-go use of the API, billed per second of audio. There are no outside investors and no advertising.

RW: What does it cost to use?

Harmon: The Free tier provides three uploads with a two-minute preview of each. Basic is $8 a month for 6 hours of audio or video, and it includes the Pro model. Pro costs $29 a month for 30 hours. The API costs $5 per hour of audio on Flash and $10 per hour on Pro, billed per second, with $5 of free credit to start. Files can be up to 90 minutes on every plan.

RW: What applications do you see for Radio World readers of your technology?

Harmon: Station archives: Reel, cassette and cart transfers of old interviews and local programs, where the voice is there but the hiss and hum make it hard to air.

Remotes and field interviews recorded in loud places: Ballgames, fairs, a reporter’s phone at a city council meeting.

Podcast and on-demand versions of shows, where listeners on earbuds hear every bit of room noise.

Transcripts and captions: In our test most cleanup tools made speech harder for a recognizer to transcribe than the untouched recording (13.2% word error rate); Diffio’s runs were the only ones that lowered it, to 3.9% on Flash and 4.1% on Pro. Diffio is guided by the same reference transcript the test scores against, which favors us on this measure, and the page says so. We ran the test and make one of the tools, so every clip and score is public for anyone to check.

Engineers who want to automate cleanup can call the API from their own scripts and get a webhook when each file is done.

RW: Many people who love radio history say that much of it is lost due to neglect or lack of interest. What’s your experience with how well radio is protecting its legacy?

Harmon: I’m not a radio historian, so I can only speak to what I’ve seen working with old recordings. The material that survives is often already digitized and public, like the Library of Congress National Jukebox, but most people never get past the first 10 seconds because it’s so hard to listen to. 

Digitizing saves the tape; it doesn’t get anyone to hear it. Restoration can help with that second part, as long as the original stays the archival copy. 

What worries me more are the recordings still sitting on shelves at stations and in private collections, where nobody knows what’s on them until someone transfers them, and tape doesn’t wait.

RW: What else should we know?

Harmon: Anyone can test it on their own worst recording for free, and I’m happy to restore a clip for any Radio World reader who sends one.

[Read More Radio World Stories About Artificial Intelligence]

Close