Arabic YODAS v3, cut into utterance-length clips

The data/ar/ partition of espnet/yodas3, cut from long recordings (median ~5 minutes) into clips of 5-45 seconds, each with its own transcript. Ready to batch for ASR training without further segmentation.

from datasets import load_dataset
ds = load_dataset("oddadmix/yodas-v3-ar-labeled", split="train", streaming=True)

Audio is Opus 48 kbps mono at 16 kHz in Ogg.

Where the cuts come from

Upstream timestamps are YouTube caption cues, not utterance segments. Measured on a shard of data/ar:

71.8% of consecutive cues strictly overlap (median gap -2.36 s): start + duration is how long a cue stays on screen, not when its speech ends.
Only 9.7% of cue boundaries have a real silence gap >= 0.3 s, so cutting on caption boundaries puts most cuts inside continuous speech.

Snapping caption boundaries to the quietest nearby window does not fix this: scored by an independent VAD, 54.8% of such cuts still landed in speech. So the cuts here are chosen the other way round -- Silero VAD finds the pauses, pauses at least 150 ms long are the only legal cut points, and the transcript is then assigned to whatever span results. Measured the same way: 0.0% of cut points land in speech.

Because text follows the cut, word timings do the assignment where the cue has them (68% of cues), and majority overlap otherwise. cues keeps the contributing cue fragments with times rebased to the clip and a partial flag, so word-level alignment stays derivable.

clean_cut is false when a stretch of speech longer than 45 s contained no pause and the cut had to be forced past the limit; those are dropped here. speech_ratio is the fraction of the clip the VAD calls speech.

start/end locate the clip in the source recording and source_id groups clips cut from the same one -- split by source_id, not by row, or the same speaker and topic land on both sides of your split.

Filtering applied

Cues with no text, and consecutive duplicate cue text (some recordings repeat one channel-outro caption for minutes), are dropped.
Non-speech markers ([موسيقى], [ضحك], ...) are stripped; clips left with no text go.
Clips below 35% speech by VAD, and outside 1-30 characters per second, are dropped as silence or misalignment.

Dialect labels

dialect is one of 13 Arabic dialects or msa from Nawah-Dialect-BERT-6M, predicted on the whole source recording and copied to each of its clips -- a single clip carries too little text to classify reliably. dialect_probs has all 14 class probabilities.

These are predictions, not ground truth: they read the transcript's language register rather than the speaker's accent, and the transcripts are themselves machine-generated.

Dataset statistics

Measured over all 2,989,202 clips in the repository (dataset_stats.py, 07 Oct 2026).


Clips	2,989,202
Total audio	20,718 hours
Source recordings	103,022
Clips per recording	29.0 on average
Clip length	25.0 s mean, 24.9 s median, 7.6-42.5 s at the 5th-95th percentile
Speech by VAD	77.3% of clip duration on average
Transcript	37.7 words per clip, 7.9 characters/second median
Total words	112,577,266
Shards covered	208 of 208
Download size	474 GB

Hours per dialect

dialect is the argmax label; the probability-mass column weights every clip by its full dialect_probs distribution, which is the more honest read because the tagger is scoring a transcript's register rather than a speaker's accent.

Dialect	Clips	Hours	Share	Hours by probability mass
`msa`	1,706,463	11,861	57.2%	10,070
`eg`	531,912	3,681	17.8%	2,893
`ma`	145,591	1,014	4.9%	994
`tn`	130,160	903	4.4%	1,536
`dz`	121,202	840	4.1%	1,036
`ye`	76,807	525	2.5%	954
`bh`	68,372	467	2.3%	773
`sd`	51,931	354	1.7%	682
`iq`	49,194	339	1.6%	396
`sy`	33,595	231	1.1%	297
`sa`	30,704	207	1.0%	270
`ps`	18,533	127	0.6%	303
`ly`	12,591	86	0.4%	401
`lb`	12,147	83	0.4%	112

Hours per region

Region	Clips	Hours	Share
MSA	1,706,463	11,861	57.2%
Egyptian	531,912	3,681	17.8%
Maghrebi	409,544	2,843	13.7%
Gulf	175,883	1,200	5.8%
Levantine	64,275	441	2.1%
Sudanese	51,931	354	1.7%
Iraqi	49,194	339	1.6%

Clip length distribution

Length	Clips	Hours	Share of hours
5-10 s	335,983	721	3.5%
10-15 s	391,405	1,359	6.6%
15-20 s	386,779	1,879	9.1%
20-25 s	387,653	2,422	11.7%
25-30 s	383,241	2,926	14.1%
30-35 s	384,434	3,470	16.7%
35-40 s	384,774	4,008	19.3%
40-45 s	334,933	3,934	19.0%

transcript_lang: ar 2,989,202. clean_cut is true on 100.0% of clips.

Provenance

CC-BY-3.0, inherited from YODAS v3. Audio was decoded from upstream Opus, resampled to 16 kHz mono and re-encoded, so it is a second lossy generation.