Advanced Bayesian Machine Learning
Chapter 1: Revisiting Bayesian Foundations
Probabilistic Modeling Principles
Bayes' Theorem in High Dimensions
Subjectivity and Objectivity in Prior Selection
Information Theory Connection: Entropy and KL Divergence
Computational Challenges in Bayesian Inference
Advanced Model Checking and Criticism
Chapter 2: Markov Chain Monte Carlo Methods
Principles of Monte Carlo Integration
Markov Chain Theory for MCMC
Metropolis-Hastings Algorithm Variants
Gibbs Sampling for Conditional Structures
Hamiltonian Monte Carlo (HMC) Mechanics
The No-U-Turn Sampler (NUTS)
Diagnosing MCMC Convergence and Performance
Hands-on Practical: Implementing Advanced MCMC
Chapter 3: Variational Inference Techniques
Optimization as Inference: The VI Perspective
Deriving the Evidence Lower Bound (ELBO)
Mean-Field Approximation Details
Coordinate Ascent Variational Inference (CAVI)
Stochastic Variational Inference (SVI) for Large Data
Black Box Variational Inference (BBVI)
Advanced Variational Families
Comparing MCMC and VI Strengths
Hands-on Practical: Scalable Variational Inference
Chapter 4: Gaussian Processes
Bayesian Non-parametric Modeling Introduction
Defining Gaussian Processes: Priors Over Functions
Covariance Functions (Kernels): Properties and Selection
Gaussian Process Regression Formulation
Hyperparameter Marginal Likelihood Optimization
Gaussian Process Classification Techniques
Approximations for Scalable Gaussian Processes
Hands-on Practical: Gaussian Process Modeling
Chapter 5: Advanced Topics in Probabilistic Graphical Models
Bayesian Networks: Structure Learning
Parameter Learning in Bayesian Networks
Advanced Inference in PGMs: Junction Tree Algorithm
Approximate Inference in Large PGMs
Latent Dirichlet Allocation (LDA): Bayesian Formulation
Inference for LDA: Collapsed Gibbs Sampling
Inference for LDA: Variational Bayes
Hands-on Practical: Topic Modeling with LDA
Chapter 6: Bayesian Deep Learning
Motivation for Bayesian Deep Learning
Bayesian Neural Networks (BNNs): Priors over Weights
Inference Challenges in BNNs
MCMC Methods for BNNs (e.g., Stochastic Gradient HMC)
Variational Inference for BNNs (e.g., Bayes by Backprop)
Uncertainty Estimation in BNNs
Variational Autoencoders (VAEs) as Probabilistic Models
Dropout as Approximate Bayesian Inference
Practical Training and Evaluation of BNNs
Hands-on Practical: Building a Bayesian Neural Network
Approximate Inference in Large PGMs
Was this section helpful?
- Probabilistic Graphical Models: Principles and Techniques, Daphne Koller and Nir Friedman, 2009 (MIT Press) - This comprehensive textbook is the definitive reference for probabilistic graphical models, detailing exact inference, the necessity of approximate methods, and in-depth coverage of MCMC (including Gibbs sampling in PGMs) and Variational Inference.
- Pattern Recognition and Machine Learning, Christopher M. Bishop, 2006 (Springer) DOI: 10.1007/978-0-387-45528-0 - A widely-used textbook that provides a clear and comprehensive introduction to the theoretical foundations of both Markov Chain Monte Carlo (MCMC) and Variational Inference, including the mean-field approximation and ELBO.
- Variational Inference: A Review for Statisticians, David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe, 2017 Journal of the American Statistical Association, Vol. 112 (American Statistical Association) DOI: 10.1080/01621459.2017.1285773 - This highly influential review article provides an accessible and comprehensive overview of Variational Inference, covering its theoretical underpinnings, the mean-field approximation, and modern advancements like Stochastic Variational Inference.
- Bayesian Data Analysis, Andrew Gelman, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin, 2013 (CRC Press) DOI: 10.1201/b16018 - This foundational book on Bayesian statistics offers detailed coverage of Markov Chain Monte Carlo methods, including Gibbs sampling, practical implementation considerations, and convergence diagnostics, essential for robust approximate inference.