– A Developer’s Perspective
Published based on research by IoT++ AI COE ( Center of Excellence)
Text-to-speech (TTS) technology has come a long way, finding its way into virtual assistants, automated announcements, audiobooks, and even accessibility tools. But how does TTS actually work, and why does it still struggle with certain tasks—like pronouncing numbers correctly? As a developer, I find this tech fascinating and challenging. Here’s a breakdown of how TTS works, what common issues it faces, and some insights into its quirks.
What is Text-to-Speech?
TTS is part of the broader field of speech recognition technology, responsible for transforming written text into spoken words. At its core, TTS involves complex natural language modeling techniques that convert text into audio. TTS systems use programming languages like Python extensively, leveraging libraries such as gTTS, pyttsx3, and OpenAI’s TTS models to produce audio from text. However, not all TTS libraries perform equally—each has unique strengths and challenges.
How Does TTS Work?
The TTS process involves several key stages:
- Preprocessing: The text is first tokenized into words, then further broken down into phonemes (e.g., “Hello, have a good day” becomes phonetic units like HH AH0 L OW1). Each phoneme’s duration is calculated to control how long it is played in the audio, while features like pitch and energy are also factored in. These elements are crucial for making speech sound natural, as they influence prosody—the rhythm and melody of spoken language.
- Encoding: Here, phonemes are converted into a compressed, n-dimensional “latent feature” embedding, representing the essential components of speech. This latent feature carries the information needed for pitch, energy, and other properties that will shape the sound.
- Decoding: The decoder transforms the encoded features into a mel-spectrogram, a visual representation of sound that essentially maps audio frequency over time.
- Vocoder: Finally, the vocoder takes the mel-spectrogram and translates it into a waveform—actual sound. This can be achieved with mathematical models like Griffin-Lim or, more commonly today, using neural networks that are trained specifically for this purpose. Splitting the TTS process this way (mel-spectrogram generation and then waveform creation) has proven to deliver higher-quality results.
Challenges with Numbers in TTS
Despite the advancements, TTS systems often encounter challenges with numbers. Here are some key issues:
- Pronunciation Errors: Numbers within text, especially in longer strings, are frequently mispronounced due to phonetic complexities and the various ways numbers can be read.
- Dropping Numbers: Some TTS models have been known to omit certain numbers, e.g., reading “123456789” as “12378” or “1356789.”
- Pause Handling: TTS often struggles to apply pauses naturally around numbers, which can make them sound awkward or merge with surrounding words.
- Technical Constraints: The algorithms within TTS may lack optimization for long number sequences, making them prone to errors.
- Context Confusion: Numbers used in various contexts—dates, quantities, or addresses—may not be interpreted accurately without the model discerning the specific scenario.
Why Do These Problems Happen?
The way TTS models are trained often explains why numbers can be a stumbling block:
- Contextual Misinterpretation: Numbers in different contexts (dates vs. amounts) may require different pronunciations, and TTS can miss these cues.
- Limited Training Data: TTS systems are trained on large datasets, but these datasets don’t always have enough examples of numbers in diverse formats, leading to inconsistent results.
- Phonetic Ambiguity: Numbers have multiple possible readings—digit by digit, in groups, or as words—and deciding on the correct format can be challenging.
- Regional Variability: Pronunciation varies greatly across accents and dialects, and TTS models may not always capture these subtleties.
- Prosody Needs: Correct prosody, or the rhythm and intonation of speech, is essential for longer numbers but difficult for TTS models to apply consistently.
Testing TTS: A Look at Different Models
To better understand how TTS handles numbers, a few tests were run across different Python-based TTS libraries using phrases with numbers in various formats. Here are the phrases that were used:
Samples of TTS for the same Text:
Text-1: The data shows reading of 26795 kg and 1008.96 psi
1)gTTS : Google Text to Speech Python library.
2)pyttsx3: Python TTS library
3)open ai : Model:tts-1
4)deepgram: model: aura-asteria-en
5)coquii: model: xtts_v2
Text-2: The data shows reading of 26,795 kg and 1,008.96 psi
1)gTTS : Google Text to Speech Python library.
2)pyttsx3: Python TTS library
3)open ai : Model:tts-1
4)deepgram: model: aura-asteria-en
5)coquii: model: xtts_v2
Text-3: The data shows the reading of twenty-six hundred thousand seven hundred and ninety-five kg and one thousand and eight-point nine six psi
1)gTTS : Google Text to Speech Python library.
2)pyttsx3: Python TTS library
3)open ai : Model:tts-1
4)deepgram: model: aura-asteria-en
5)coquii: model: xtts_v2
The results showed that some TTS systems, like OpenAI’s TTS model, had difficulty with the numbers, while others performed better when numbers were spelled out. This suggests that using written-out numbers (e.g., “ten” instead of “10”) could improve the accuracy and clarity of TTS outputs in number-heavy text.
Why Accurate TTS for Numbers Matters
In industries where accuracy is paramount, like manufacturing, finance, or logistics, TTS misinterpretation of numbers could have serious repercussions. A misread measurement or financial figure can lead to costly mistakes, from flawed data reporting to incorrect instructions in critical processes. For such cases, carefully structured TTS data—or manually writing out numbers—can be crucial.
Conclusion
While TTS has made impressive strides, handling numbers in diverse contexts remains a challenge. As developers, we can mitigate some issues by structuring input text thoughtfully, especially when it contains critical numerical data. Exploring improvements in TTS models, particularly in prosody and context recognition, will help further unlock the potential of TTS technology for professional and personal use.
The journey of TTS technology is far from over, and with ongoing innovation, we’ll continue to see improvements that make the spoken language in AI even more natural, flexible, and accurate.
For more information about our Text-To-Speech Solution, please contact: hello@iotplusplus.com
Let us help you modernize your operations today. Contact us to learn more about our tailored IoT solutions.
