Text-to-Speech › Key Parameters and Design Decisions

The VITS TTS system demonstrated above exposes several important configuration choices, each with real trade-offs between output quality and speed under a given resource budget. Understanding these parameters helps you adapt the pipeline to specific applications.

The sampling rate (16,000 Hz in the MMS model) determines the temporal density of the audio signal and thus the maximum representable frequency, limited by the Nyquist theorem to half the sampling rate (8,000 Hz at 16kHz). Telephone-quality audio typically operates at 8,000 Hz, which captures sufficient bandwidth for intelligible speech but misses the presence frequencies (around 4,000 to 8,000 Hz) that contribute to the clarity of fricatives like "s" and "f." High-fidelity applications such as audiobook narration or professional voice assistants use 22,050 Hz or 44,100 Hz to preserve the full audible range. Higher sampling rates increase both the length of the waveform sequence and the computational load of the vocoder, since the decoder must produce more samples per second. For the VITS architecture, this trade-off is particularly acute because the HiFi-GAN decoder processes all samples in parallel, so doubling the sampling rate roughly doubles the memory footprint.

The number of mel bins (80 in this implementation, with 128 becoming common in newer systems) controls the spectral resolution of the intermediate representation. Each bin corresponds to a frequency band on the mel scale, and 80 bins give sufficient resolution to distinguish different vowel qualities and consonant types in English. Increasing to 128 bins captures finer spectral detail, which can improve the quality of sibilants and fricatives, but widens the target for the acoustic model and requires proportionally more model capacity to generate accurately. Models trained for tonal languages like Mandarin sometimes use more bins to resolve the subtle spectral differences in tone contrasts.

The hop length and FFT window size together define the time-frequency trade-off in the spectrogram. A hop length of 256 samples at 16,000 Hz yields approximately 80 frames per second (frame shift of 12.5ms), which is short enough to track rapidly changing sounds like stop consonants but long enough to keep the sequence manageable for the acoustic model. The FFT window of 1,024 samples (64ms) gives good frequency resolution at the cost of temporal smearing: brief events shorter than the window duration blend across adjacent frames. In practice, these values are chosen to match the acoustic properties of speech, where phonemic transitions occur on the order of 20 to 50ms and spectral resolution below 100 Hz matters for distinguishing formant patterns.

The vocabulary size depends entirely on the tokenization scheme. Character-based models have small vocabularies (perhaps 100 to 200 entries for the characters and special tokens of a given language) and require no external G2P system, but must learn pronunciation from distributional context. Phoneme-based models typically use 40 to 80 phoneme symbols per language and produce more reliable pronunciation of rare words, at the cost of requiring a G2P preprocessing step. The MMS model uses a phoneme vocabulary derived from the IPA, which generalizes across the 1,000+ supported languages because IPA was designed to stand for all human speech sounds in a unified symbol set.

These parameters interact in ways that are not always obvious. Increasing the sampling rate while keeping the hop length fixed in samples shortens the frame duration and increases the number of frames per utterance, making the acoustic modeling sequence longer and potentially harder to learn. A common solution is to scale the hop length proportionally with the sampling rate, maintaining constant frame duration regardless of audio quality level. Similarly, widening the mel filter bank beyond 80 bins often requires retraining the vocoder, since the GAN discriminators learn to detect artifacts at a resolution specific to the training configuration. When adapting a pre-trained system to a new application, understanding which parameters are tightly coupled helps identify the minimal changes needed to reach the desired quality and efficiency profile.