Variational Inference: A Review for Statisticians, David M. Blei, Alp Kucukelbir, Jon D. McAuliffe, 2017Journal of the American Statistical Association, Vol. 112 (Taylor & Francis)DOI: 10.1080/01621459.2017.1285773 - Provides a comprehensive overview of variational inference, including its theoretical foundations, various algorithms (mean-field, SVI), and limitations, aiding in understanding VI's merits and downsides.
The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo, Matthew D. Hoffman and Andrew Gelman, 2014Journal of Machine Learning Research, Vol. 15 - Introduces the No-U-Turn Sampler (NUTS), a highly efficient and widely used MCMC algorithm that adaptively tunes parameters for Hamiltonian Monte Carlo, addressing aspects of MCMC efficiency.
Auto-Encoding Variational Bayes, Diederik P Kingma, Max Welling, 2013International Conference on Learning Representations (ICLR)DOI: 10.48550/arXiv.1312.6114 - Introduces the reparameterization trick, a technique central to modern variational inference methods like stochastic variational inference (SVI), enabling efficient gradient-based optimization via automatic differentiation.
Pattern Recognition and Machine Learning, Christopher M. Bishop, 2006 (Springer)DOI: 10.1007/978-0-387-44933-2 - A classic textbook offering comprehensive coverage of both Markov Chain Monte Carlo methods (Chapter 11) and Variational Inference (Chapter 10), providing fundamental knowledge for comparing their approaches.