Chapter 1: Mathematics of Invertible Transformations

Normalizing flows operate on a specific mathematical premise. They map a simple, known probability distribution to a highly complex target distribution using a sequence of invertible functions. To build and train these models in PyTorch, we must first establish the rules governing how probability densities update when transformed.

Let's take for example a random variable $z$ from a simple base distribution. If we apply an invertible function $f$ to it, we get a new random variable $x = f(z)$. The relationship between their probability densities is defined by the change of variables theorem:

$p(x) = p(z) \left| \det \frac{\partial z}{\partial x} \right|$

This chapter focuses on the mathematical mechanics that make this equation work. You will learn how to apply the change of variables theorem to track density changes across different functions. We will study the Jacobian matrix and its determinant, which measures how the volume of space expands or contracts during a mathematical transformation.

You will also learn to differentiate between the forward and inverse operations of a flow model. The forward pass evaluates the exact probability density of data, while the inverse pass allows us to sample new data points from the learned distribution. The chapter concludes with a programming exercise where you will write Python code to calculate Jacobian determinants for simple mathematical functions, preparing you to implement full flow layers in the subsequent chapters.