In late 2023, we found the same gap across several TTS projects: no quick, repeatable way to tell whether a new model checkpoint had improved. Human listening panels remain the gold standard, but a study can take weeks and cost hundreds of dollars. Most teams instead rely on casual listening, a method that eventually breaks down.
Today, we are launching the first Podonos API version for TTS teams. Here is what it does, what it currently supports, and who should use it.
The recurring problem
TTS training generates many checkpoints. Fine-tuning a new speaker can yield 50 to 100 checkpoints over several days. At each one, the decision is unchanged: did quality improve over the prior version, and by how much?
Teams with dedicated QA can send samples to a listening panel. Most early-stage TTS teams cannot. Their engineers listen too, often catching familiar artifacts well but missing small prosody regressions between neighboring checkpoints. Results vary with the listener, their fatigue, and how many samples they chose from the batch.
We wanted an automatic signal for every checkpoint, consistent across experiments and users, that showed the direction and type of quality change, rather than merely flagging that a change occurred.
Podonos defined
Podonos is an automated voice quality scoring API. Send a WAV or FLAC file to its REST endpoint and receive perceptual scores within a few seconds. The main result is a MOS (Mean Opinion Score) estimate on the standard 1-5 scale, accompanied by three sub-scores: naturalness, prosody, and pronunciation.
We calibrated the scoring model with 2,400 ratings from a controlled listening study using the ITU-T P.800 ACR methodology. The ratings covered samples from multiple TTS systems at varied quality levels, from early concatenative synthesis through current neural architectures. On held-out samples, the model predicted human MOS values. We publish the held-out R-squared and mean absolute error figures so calibration quality can be reviewed directly.
Automated MOS is not a replacement for human evaluation. For regression detection and checkpoint comparison during iterative development, however, it is fast and accurate enough to help. Without it, teams that cannot run a panel for every training run pay a real cost.
First version scope
The first version supports English TTS outputs. Because calibration used English speech, performance is most reliable in that language. Calibration work for other languages is underway, and the benchmark page will state clearly what is covered and what is not.
The API provides:
- Evaluate one file with POST /v1/evaluate
- Evaluate up to 500 files per job with POST /v1/batch
- Benchmark endpoint with current scores for publicly available TTS systems scored on a standard evaluation set
The response format remains stable. Each response includes the scoring model version, allowing you to identify which version produced each historical result when the underlying model changes.
Intended users
Podonos serves engineers and researchers who train or fine-tune TTS models and need a consistent signal across checkpoints. A typical user has a training loop running nightly or for each experiment and prefers an endpoint call to maintaining a local model installation.
It also suits teams assessing open-source TTS systems for deployment. To compare StyleTTS2 with VITS on a target domain, score both on the same evaluation set and obtain a consistent MOS comparison without conducting a listener study. The benchmark page lists scores for several publicly available systems, showing how a candidate compares with a shared baseline.
Podonos does not currently score speaker similarity, meaning how closely a cloned voice matches its target. It also does not classify emotion in speech or support non-English languages yet. Those capabilities are on the roadmap. This release stays within what the calibration data supports, rather than claiming broader coverage.
Free tier details
The free tier provides 500 evaluations each month. That covers a daily loop on a 20-sample evaluation set for two and a half weeks before the limit, or one substantial benchmark comparison. We chose this level so teams can use the tool for real evaluation work before deciding whether it fits their workflow.
Paid plans increase with volume, and the pricing page lists current tiers. Feature fees do not vary by plan: all tiers include the three sub-scores and batch endpoint.
Getting started
Sign-up provides API keys immediately. At podonosvoice.com/docs, the documentation covers endpoint details, Python and curl examples, score meanings, and how to read differences between sub-scores.
For limitations or use cases the current API does not cover, contact [email protected]. We are a small team and read every message. Direct feedback from people running TTS evaluation workflows is the clearest guide to building the right tool.