Open TTS Leaderboard Automates Scalable Evaluation for Multilingual TTS

The rapid expansion of open-source text-to-speech models has outpaced the development of standardized evaluation frameworks. To address this disparity, developers have introduced the Open TTS Leaderboard, a scalable platform designed to systematically assess multilingual text-to-speech and voice cloning capabilities. While traditional arena-style rankings rely on human preference scores and Elo ratings, these methods struggle with scalability, often favoring commercial APIs due to lower hosting overhead and introducing voter inconsistency over time. The new leaderboard counters these limitations by implementing objective, reproducible metrics that reduce evaluation cycles from weeks to hours. Performance is measured across three primary dimensions. Intelligibility is quantified using word and character error rates, processed through Qwen3 automatic speech recognition. Computational efficiency is tracked via inverse real-time factors for batched offline inference and time-to-first-audio for streaming latency, both benchmarked on H200 GPUs and select CPU environments. Speaker similarity is calculated through cosine similarity metrics between reference and generated audio embeddings. Initial rankings highlight strong English performance from models such as Kokoro-82M and Supertonic-3, while multilingual capabilities are led by OmniVoice and Fun-CosyVoice3. Character-based languages including Chinese, Japanese, and Korean are evaluated using character error rates to ensure accurate cross-linguistic comparison. The platform also features dedicated streaming and voice cloning assessments. Time-to-first-audio measurements distinguish between native streaming models and non-streaming alternatives, identifying lightweight solutions optimized for interactive voice agents. The voice cloning module tracks speaker similarity scores alongside inference performance, revealing which models maintain vocal fidelity while reducing latency. A community voting interface allows users to compare raw outputs, submit preference data, and influence future metric weighting, ensuring the framework evolves alongside developer needs. By prioritizing open-weights models, multilingual coverage, and reproducible benchmarks, the Open TTS Leaderboard establishes a transparent alternative to proprietary evaluation ecosystems. Evaluation scripts will soon be released openly, inviting developer contributions to expand dataset coverage and refine scoring methodologies. This initiative aims to standardize open-source text-to-speech assessment, accelerate model iteration, and provide developers with reliable, scalable performance data.