Tacotron 2 is a LSTM-based Encoder-Attention-Decoder model that converts text to mel spectrograms. The encoder network The encoder network first embeds either characters or phonemes. The embedding is sent through a convolution stack, and then sent through a bidirectional LSTM. The decoder is an autoregressive LSTM: it generates one time slice of the mel spectrogram on each call. The decoder is connected the encoder via the attention module which tells the decoder which part of the encoded text to use to generate each slice of the spectrogram.
Tacotron 2 is intended to be used as the first part of a two stage speech synthesis pipeline. Tacotron 2 takes text and produces a mel spectrogram. The second stage takes the generated mel spectrogram and returns audio.
Input English text strings
Output Mel spectrogram of shape (batch x mel_channels x time)
The provided .nemo checkpoint can be used, in junction with a WaveGlow checkpoint, to generate speech via Jarvis. To deploy a TTS service via Jarvis, please refer to the Jarvis documentation
This model is trained on LJSpeech sampled at 22050Hz, and can be used to generate female English voices with an American accent.
By downloading and using the models and resources packaged with TLT Conversational AI, you would be accepting the terms of the Jarvis license
NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.
Tacotron 2 paper: https://arxiv.org/abs/1712.05884