We released 22.7 hours of Amharic speech, and Addis AI benchmarked it in days

Amharic has around sixty million speakers, more than Italian or Korean. Until this week there was almost no open speech data for it. Today there is a little more.

We have published Dataset.ET Amharic Speech v0.1.0: 22.7 hours of read speech, 7,405 recordings, from 320 different people. It is free, it is CC BY 4.0, and anyone can download it right now.

This is a first batch out of the roughly five hundred hours we have collected so far. We are releasing it early rather than waiting, because we would rather people use it and tell us what is wrong with it than have us guess.

How the data gets made

Almost everyone in Ethiopia uses Telegram, so we built the whole thing inside it. There is no app to download and no laptop needed.

You open the bot, it shows you a sentence, and you read it aloud as a voice message. Other contributors then listen to what you recorded and vote on whether it matches the text. When your recording is accepted you earn mobile airtime, delivered to the phone you are already holding.

More than a thousand people have recorded for us this way. That is the part we are proudest of, and it is also the part that is hard to copy.

What we found when we looked closely

Here is the uncomfortable part, and the reason we are writing this rather than just posting a link.

Before releasing anything we built a screening pipeline. It converts every recording to a standard format and measures it: how loud it is, how much of it is actual speech rather than silence, whether it clips, and whether the length makes sense for the sentence the person was asked to read.

We then ran it over our best material. Not all of it, just the recordings that three or more people had unanimously approved. The cleanest tier we have.

The largest single problem was people not reading the whole sentence. Someone starts, tails off, and sends it anyway. To a human validator listening quickly it sounds like a normal recording. It is not one.

This is not a criticism of our contributors or our validators. It is a criticism of how we had set the job up. People cannot hear whether a recording is loud enough to train a model on, or whether it is forty percent silence. Those are measurements, not judgements, and we were asking humans to make them.

Everything that failed was left out of the release. The 22.7 hours we published is what survived.

Then Addis AI benchmarked it

Within days of publishing, Addis AI took our held-out test set and ran seven speech recognition systems against it. We did not ask them to. Biniyam Daniel, who founded the company, wrote the whole thing up: seven ASR systems on the Amharic speech dataset.

Two things stand out. The first is that Addis AI, a team in Addis Ababa, beat Google and Meta on Amharic. Those two companies are worth more than five trillion dollars between them. The gap was never compute or talent. It was data.

The second is that our test set is hard. Meta's MMS-1B scores 26.1 on another Amharic benchmark and 52.0 on ours. We think that is because our recordings come from ordinary phones in ordinary rooms, which is what real speech actually sounds like.

What comes next

We want every clip checked by a machine before a human ever sees it. Speech recognition can compare what was said against what was meant to be said, and catch the incomplete readings automatically. That needs GPUs, and it is what we are working on now.

After that, we train our own models on this data and give them away. The best measured result on our test set today is 25.0. We would like to beat it.

We are also going to keep publishing what we find, including the parts that do not flatter us. A dataset nobody can check is not worth much.

Try it

The data is on Hugging Face. If you are working on Ethiopian languages, low-resource speech, or you just want to argue with our numbers, we would like to hear from you.

And if you want to help build it, the bot is at t.me/dataset_et_bot. It takes a few minutes, and you get paid.