Advanced Speech Recognition and Synthesis
Chapter 1: Foundations of Modern Speech Processing Pipelines
Advanced Audio Feature Extraction
Statistical Modeling Review for Speech
Deep Learning Architectures for Sequences
Components of ASR Systems
Components of TTS Systems
Evaluation Metrics Revisited
Chapter 2: Advanced Acoustic Modeling for ASR
Hybrid HMM-DNN Systems
Connectionist Temporal Classification (CTC)
Attention-Based Encoder-Decoder Models
RNN Transducer (RNN-T)
Transformer Architectures for ASR
Advanced Training Techniques
Decoding Algorithms Comparison
Hands-on Practical: Building an End-to-End ASR Model
Chapter 3: Language Modeling and Adaptation in ASR
Neural Language Models for ASR
Shallow Fusion and Deep Fusion
Contextual ASR
Speaker Adaptation Techniques
Environment and Channel Adaptation
Unsupervised and Semi-Supervised Learning for ASR
Multi-Lingual and Cross-Lingual ASR
Practice: Fine-tuning ASR with Adaptation Data
Chapter 4: Advanced Text-to-Speech Synthesis
Autoregressive Acoustic Models (Tacotron, Transformer TTS)
Non-Autoregressive Acoustic Models (FastSpeech, ParaNet)
Flow-Based Models for TTS
Generative Adversarial Networks (GANs) in TTS
Prosody Modeling and Control
Expressive Speech Synthesis
Voice Cloning and Conversion
Hands-on Practical: Training an Advanced TTS Model
Chapter 5: Neural Vocoders and Waveform Generation
Limitations of Traditional Vocoders
Autoregressive Waveform Models (WaveNet, WaveRNN)
Flow-Based Vocoders (WaveGlow, FloWaveNet)
GAN-Based Vocoders (MelGAN, HiFi-GAN)
Diffusion Models for Vocoding
Conditioning Neural Vocoders
Evaluation of Synthesized Audio Quality
Hands-on Practical: Using a Neural Vocoder
Chapter 6: Optimization, Deployment, and Toolkits
Model Quantization for Speech Models
Model Pruning and Sparsification
Knowledge Distillation for ASR/TTS
Optimized Inference Engines (ONNX Runtime, TensorRT)
Deployment Considerations for Streaming ASR
Deployment Considerations for Real-Time TTS
Overview of Speech Processing Toolkits (ESPnet, NeMo, Coqui)
Practice: Optimizing a Speech Model
Flow-Based Vocoders (WaveGlow, FloWaveNet)
Was this section helpful?
- WaveGlow: A Flow-based Generative Network for Speech Synthesis, Ryan Prenger, Rafael Valle, Bryan Catanzaro, 2019 ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) DOI: 10.48550/arXiv.1811.00002 - Presents the WaveGlow architecture and its application to speech synthesis, offering parallel generation.
- Glow: Generative Flow with Invertible 1x1 Convolutions, Diederik P. Kingma, Prafulla Dhariwal, 2018 Advances in Neural Information Processing Systems (NeurIPS 2018) (Neural Information Processing Systems Foundation) DOI: 10.48550/arXiv.1807.03039 - Introduces the Glow model, which WaveGlow adapts, featuring affine coupling layers and invertible 1x1 convolutions for generative flows.
- FloWaveNet: A Generative Flow for Raw Audio, Taejun Kim, Hyoung-Seok Kwon, Jae-Sung Bae, Kyeong-Jin Mun, Kook-Cho Roh, 2019 Proc. Interspeech 2019 (International Speech Communication Association (ISCA)) DOI: 10.21437/Interspeech.2019-2169 - Describes FloWaveNet, an alternative flow-based vocoder architecture mentioned in the section, designed for raw audio generation.
- Density Estimation using Real NVP, Laurent Dinh, Jascha Sohl-Dickstein, Samy Bengio, 2017 International Conference on Learning Representations (ICLR 2017) DOI: 10.48550/arXiv.1605.08803 - Introduces Real NVP, a normalizing flow model that uses affine coupling layers, a core component discussed in the section for flow-based models.