NGC | Catalog
CatalogModelsSpeech Synthesis English Tacotron2

Speech Synthesis English Tacotron2

Logo for Speech Synthesis English Tacotron2
Description
Mel-Spectrogram prediction conditioned on input text with LJSpeech voice.
Publisher
NVIDIA
Latest Version
deployable_v1.0
Modified
August 26, 2021
Size
107.6 MB

Speech Synthesis: Tacotron 2 Model Card =======================================

Model Overview --------------

Tacotron 2 is a LSTM-based Encoder-Attention-Decoder model that converts text to mel spectrograms. The encoder network The encoder network first embeds either characters or phonemes. The embedding is sent through a convolution stack, and then sent through a bidirectional LSTM. The decoder is an autoregressive LSTM: it generates one time slice of the mel spectrogram on each call. The decoder is connected the encoder via the attention module which tells the decoder which part of the encoded text to use to generate each slice of the spectrogram.

Intended Use ------------

Tacotron 2 is intended to be used as the first part of a two stage speech synthesis pipeline. Tacotron 2 takes text and produces a mel spectrogram. The second stage takes the generated mel spectrogram and returns audio.

Input English text strings

Output Mel spectrogram of shape (batch x mel_channels x time)

How to Use This Model ---------------------

The provided .nemo checkpoint can be used, in junction with a WaveGlow checkpoint, to generate speech via Jarvis. To deploy a TTS service via Jarvis, please refer to the Jarvis documentation

Training Information --------------------

This model is trained on LJSpeech sampled at 22050Hz, and can be used to generate female English voices with an American accent.

License -------

By downloading and using the models and resources packaged with TLT Conversational AI, you would be accepting the terms of the Jarvis license

Ethical AI

NVIDIA’s platforms and application frameworks enable developers to build a wide array of AI applications. Consider potential algorithmic bias when choosing or creating the models being deployed. Work with the model’s developer to ensure that it meets the requirements for the relevant industry and use case; that the necessary instruction and documentation are provided to understand error rates, confidence intervals, and results; and that the model is being used under the conditions and in the manner intended.

Citations and Further Reading =============================

Tacotron 2 paper: https://arxiv.org/abs/1712.05884