Published 17 hours ago • loading... • Updated 17 hours ago

Humyn Labs launches report for AI voice benchmarking across Global South languages

Humyn Labs released the BRIDGE benchmark to evaluate 15 commercial AI speech-recognition models across 22 non-English languages spoken by over 5.5 billion people in the Global South using real conversational data.
ASR providers previously published accuracy scores using English-first, internet-trained datasets with little independent validation, misleading enterprises making million-dollar deployment decisions, said Manish Agarwal, co-founder, Humyn Labs.
Built on real two-person conversations rather than scripted audio, the benchmark applies a seven-metric scoring stack including Semantic Similarity and Phoneme-Informed Error Rate beyond standard word error measures.
ElevenLabs Scribe v2 leads globally at 10.6% word error rate, yet rankings obscure regional performance; AssemblyAI Universal ranks third globally but outperforms all competitors in Vietnamese.
Single aggregate benchmark scores cannot reliably guide cross-regional deployment decisions, Ishank Gupta, co-founder, noted; the full dataset is available on Hugging Face for independent analysis.

Insights by Ground AI