A naturalness rating compresses several questions into one judgment: how smoothly does speech flow, how human does the voice sound, how well do rhythm and intonation fit, and are artifacts absent? A listening study returns one value for these interacting dimensions. Knowing what the score combines, and what automated metrics capture or miss, is essential for interpreting it correctly.
What listeners judge
Speech perception research indicates that naive listeners weigh at least four properties at once, although their relative importance varies and the process is not conscious:
Fluency and continuity. Does speech proceed smoothly, without audible breaks, restarts, or odd pauses inside words or phrases? Concatenative TTS historically struggled because joins between acoustic units produced discontinuities that even untrained listeners heard at once. Neural TTS has largely addressed continuity, though some models still show weaknesses in long-range prosodic coherence.
Human-like voice quality. Does the sound suggest a human vocal tract, or does it reveal the tone of a synthesis artifact? This includes breathiness, creakiness, and the fine acoustic variation that gives a human voice texture instead of a flat character. Older WaveNet implementations could expose vocoder artifacts here despite accurate prosody.
Rhythm and intonation. Does stress land where the phrase requires it? Do questions sound like questions? Does pacing reflect the sentence's meaning? Even listeners without linguistic training notice prosodic patterns. If TTS parses a sentence incorrectly and places stress in the wrong position, naturalness falls despite accurate production of every phoneme.
Unexpected artifacts. Clicks, pops, background noise, abrupt volume shifts, and pitch discontinuities can all sharply reduce naturalness. They are often easiest to hear, but their presence varies most between systems. One artifact in an otherwise clean utterance can reduce a sample's MOS by 0.5 to 1.0 scale points.
How the dimensions combine
Because naturalness is composite, the same score can reflect different weaknesses. Take two samples with a MOS of 3.4: one may have strong prosody and voice quality but one distracting artifact; the other may be clean yet flat and monotone. Their aggregate results match, while the diagnosis and appropriate fix do not.
That is one reason we publish sub-scores independently. In the Podonos API response, naturalness is a composite weighted strongly toward fluency and the absence of artifacts. Prosody reflects rhythmic and intonation accuracy. They may differ sharply for one sample, and that gap can reveal more than either score by itself.
What the Podonos naturalness sub-score captures
The naturalness model was trained mainly on variation in fluency, voice quality, and artifact presence. Across the range of TTS systems used for calibration, from several technology generations and quality levels, these factors account for the greatest share of variation in human naturalness ratings.
The model is strongest at finding waveform artifacts such as clicks, pitch jumps, and clipping, assessing voice-quality smoothness, and spotting utterance-level intonation anomalies. With modern neural TTS backbones, these qualities are usually clean, so predictions can cluster and discrimination can weaken near the top. This is expected in MOS estimation: 4.0 to 5.0 is harder to separate because the acoustic cues distinguishing 4.2 from 4.5 are subtle and depend on the rater.
The model reflects some prosodic naturalness, but it does not fully separate every part of it. Sentence-level MOS models do not capture long-range discourse prosody well, including handling of paragraph-length text, chapter breaks, or conversational back-and-forth. Because training uses sentence-level samples rather than document-level passages, effects that appear only across multiple sentences fall outside its observational window.
What the naturalness sub-score excludes
Comprehensibility and naturalness are different properties. A voice may sound highly natural while mispronouncing particular words, or sound less natural while remaining easy to understand. We keep these measures separate. Deployments involving technical vocabulary, product names, or domain-specific terminology should assess pronunciation and intelligibility independently. The Podonos API therefore returns naturalness and pronunciation as separate sub-scores.
Speaker similarity is separate as well. In voice cloning or multi-speaker models, a naturalness result does not show whether a specific target identity was reproduced. A voice can score 4.2 on naturalness and still differ greatly from the reference speaker. Assess similarity with a separate embedding-based metric or by directly comparing the samples with the reference.
The ceiling at the top of the scale
Teams using current neural TTS architectures with high-quality training data will often see naturalness scores from 4.0 to 4.5 on well-formed sentences. At that level, checkpoint differences may sit inside automated MOS estimation's uncertainty band. The tool is most useful for finding regressions and making within-system comparisons, rather than ranking slight differences among the highest-quality outputs.
To separate 4.3 from 4.5 in absolute terms, use a human listening panel. Automated MOS suits the question "did this checkpoint get better or worse than last week's?" but not "is this model slightly better than that other top-tier model?" Upper-range perceptual differences are real, yet a controlled human study is needed to detect them reliably.
This matters when reading scores near the ceiling. A 4.3 does not indicate a voice that is only slightly imperfect; it indicates speech that sounds natural to most listeners across most utterances. Pursuing a higher automated result there is unlikely to track a meaningful gain in perceived quality.
Naturalness keeps moving
Listener expectations of natural speech change over time. As daily exposure to voice assistants and audiobook narrators grows, the reference point for "natural" shifts. A voice that impressed listeners in 2020 may seem slightly robotic to that same panel in 2025 because expectations for synthetic speech have advanced.
This reflects calibration, not a defect in MOS methodology. Absolute MOS values from different years cannot be compared directly without considering the calibration period. When we updated the scoring model in mid-2025, we used a new listening panel for recalibration and regenerated benchmark entry scores, preserving comparability within that new calibration. The benchmark page records the calibration date for this reason.