Denoising Diffusion Probabilistic Models (DDPM)

Denoising Diffusion Probabilistic Models (DDPMs) represent a prominent and highly effective framework within the broader family of diffusion models. Introduced by Ho et al. (2020), DDPMs utilize a fixed, multi-step forward process that gradually adds Gaussian noise to data, paired with a learned reverse process that iteratively removes the noise to generate new data samples.

The Forward Process: Adding Noise

The forward process, also known as the diffusion process, is defined as a Markov chain that progressively corrupts an initial data point $\mathbf{x}_0 \sim q(\mathbf{x}_0)$ by adding small amounts of Gaussian noise over $T$ discrete timesteps. The noise level at each step $t$ is controlled by a predefined variance schedule $\{\beta_t\}_{t=1}^T$, where $0 < \beta_1 < \beta_2 < ... < \beta_T < 1$. Typically, $T$ is large (e.g., $T=1000$), and the $\beta_t$ values are small, ensuring that the change at each step is subtle.

The distribution of the noisy sample $\mathbf{x}_t$ given the previous sample $\mathbf{x}_{t-1}$ is a Gaussian:

$$q(\mathbf{x}_t | \mathbf{x}_{t-1}) = \mathcal{N}(\mathbf{x}_t; \sqrt{1-\beta_t} \mathbf{x}_{t-1}, \beta_t \mathbf{I})$$

Here, $\mathcal{N}(\mathbf{x}; \boldsymbol{\mu}, \sigma^2\mathbf{I})$ denotes a Gaussian distribution over variable $\mathbf{x}$ with mean $\boldsymbol{\mu}$ and diagonal covariance $\sigma^2\mathbf{I}$.

A significant property of this process is that we can sample $\mathbf{x}_t$ directly from the original data $\mathbf{x}_0$ in a closed form, avoiding the need to iterate through all intermediate steps. Let $\alpha_t = 1 - \beta_t$ and $\bar{\alpha}_t = \prod_{i=1}^t \alpha_i$. Then, the distribution of $\mathbf{x}_t$ conditioned on $\mathbf{x}_0$ is also Gaussian:

$$q(\mathbf{x}_t | \mathbf{x}_0) = \mathcal{N}(\mathbf{x}_t; \sqrt{\bar{\alpha}_t} \mathbf{x}_0, (1-\bar{\alpha}_t) \mathbf{I})$$

This equation can be interpreted as scaling the original image $\mathbf{x}_0$ by $\sqrt{\bar{\alpha}_t}$ and adding Gaussian noise $\boldsymbol{\epsilon} \sim \mathcal{N}(0, \mathbf{I})$ scaled by $\sqrt{1-\bar{\alpha}_t}$:

$$\mathbf{x}_t = \sqrt{\bar{\alpha}_t}\mathbf{x}_0 + \sqrt{1-\bar{\alpha}_t}\boldsymbol{\epsilon}$$

As $t$ approaches $T$, $\bar{\alpha}_t$ approaches 0 (since $0 < \alpha_t < 1$). Consequently, $\mathbf{x}_T$ becomes almost entirely independent of $\mathbf{x}_0$ and resembles an isotropic Gaussian distribution $\mathcal{N}(0, \mathbf{I})$. The variance schedule $\beta_t$ is often chosen to be linearly increasing, quadratically increasing, or based on a cosine schedule.

The Reverse Process: Learning to Denoise

The generative power of DDPMs comes from learning the reverse process: starting from pure noise $\mathbf{x}_T \sim \mathcal{N}(0, \mathbf{I})$ and gradually removing noise to obtain a realistic sample $\mathbf{x}_0$. This involves learning the transition probabilities $p_\theta(\mathbf{x}_{t-1} | \mathbf{x}_t)$ for $t = T, T-1, ..., 1$.

If $\beta_t$ is sufficiently small, the true reverse transition $q(\mathbf{x}_{t-1} | \mathbf{x}_t)$ would also be approximately Gaussian. The challenge is that computing the true reverse transition $q(\mathbf{x}_{t-1} | \mathbf{x}_t)$ requires marginalizing over all possible data points, which is intractable. However, the posterior $q(\mathbf{x}_{t-1} | \mathbf{x}_t, \mathbf{x}_0)$ is tractable and can be shown to be Gaussian:

$$q(\mathbf{x}_{t-1} | \mathbf{x}_t, \mathbf{x}_0) = \mathcal{N}(\mathbf{x}_{t-1}; \tilde{\boldsymbol{\mu}}_t(\mathbf{x}_t, \mathbf{x}_0), \tilde{\beta}_t \mathbf{I})$$

where

$$\tilde{\boldsymbol{\mu}}_t(\mathbf{x}_t, \mathbf{x}_0) = \frac{\sqrt{\bar{\alpha}_{t-1}}\beta_t}{1-\bar{\alpha}_t} \mathbf{x}_0 + \frac{\sqrt{\alpha_t}(1-\bar{\alpha}_{t-1})}{1-\bar{\alpha}_t} \mathbf{x}_t$$

and $\tilde{\beta}_t = \frac{1-\bar{\alpha}_{t-1}}{1-\bar{\alpha}_t} \beta_t$.

Since we don't have $\mathbf{x}_0$ during the reverse sampling process (that's what we want to generate!), we approximate this distribution using a neural network. DDPM parameterizes the reverse transitions $p_\theta(\mathbf{x}_{t-1} | \mathbf{x}_t)$ as Gaussians:

$$p_\theta(\mathbf{x}_{t-1} | \mathbf{x}_t) = \mathcal{N}(\mathbf{x}_{t-1}; \boldsymbol{\mu}_\theta(\mathbf{x}_t, t), \boldsymbol{\Sigma}_\theta(\mathbf{x}_t, t))$$

The goal is to train the parameters $\theta$ such that this distribution closely matches the true (but unknown) $q(\mathbf{x}_{t-1} | \mathbf{x}_t)$.

Network Parameterization and the Simplified Objective

In the original DDPM paper, the variance of the reverse process is fixed to untrained constants $\boldsymbol{\Sigma}_\theta(\mathbf{x}_t, t) = \sigma_t^2 \mathbf{I}$. Two common choices are $\sigma_t^2 = \beta_t$ or $\sigma_t^2 = \tilde{\beta}_t$. The neural network, therefore, only needs to predict the mean $\boldsymbol{\mu}_\theta(\mathbf{x}_t, t)$.

Instead of directly predicting the mean $\boldsymbol{\mu}_\theta$, DDPMs reparameterize the network to predict the noise $\boldsymbol{\epsilon}$ that was added at step $t$. Recall the forward process equation: $\mathbf{x}_t = \sqrt{\bar{\alpha}_t}\mathbf{x}_0 + \sqrt{1-\bar{\alpha}_t}\boldsymbol{\epsilon}$. We can rearrange this to express $\mathbf{x}_0$ in terms of $\mathbf{x}_t$ and $\boldsymbol{\epsilon}$:

$$\mathbf{x}_0 = \frac{1}{\sqrt{\bar{\alpha}_t}}(\mathbf{x}_t - \sqrt{1-\bar{\alpha}_t}\boldsymbol{\epsilon})$$

Substituting this expression for $\mathbf{x}_0$ into the formula for the ideal mean $\tilde{\boldsymbol{\mu}}_t(\mathbf{x}_t, \mathbf{x}_0)$ allows us to express the mean $\boldsymbol{\mu}_\theta$ in terms of the predicted noise $\boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t)$:

$$\boldsymbol{\mu}_\theta(\mathbf{x}_t, t) = \frac{1}{\sqrt{\alpha_t}} \left( \mathbf{x}_t - \frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}} \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t) \right)$$

This reparameterization leads to a connection between minimizing the difference between the predicted mean $\boldsymbol{\mu}_\theta$ and the true mean $\tilde{\boldsymbol{\mu}}_t$ and minimizing the difference between the predicted noise $\boldsymbol{\epsilon}_\theta$ and the actual noise $\boldsymbol{\epsilon}$.

The authors found that training the model using a simplified objective function, derived from the variational lower bound (VLB) on the data log-likelihood, works very well in practice and is easier to implement. This simplified objective aims to minimize the mean squared error between the true noise $\boldsymbol{\epsilon}$ added during the forward process and the noise predicted by the network $\boldsymbol{\epsilon}_\theta$:

$$L_{simple}(\theta) = \mathbb{E}_{t \sim \mathcal{U}(1, T), \mathbf{x}_0 \sim q(\mathbf{x}_0), \boldsymbol{\epsilon} \sim \mathcal{N}(0, \mathbf{I})} \left[ \| \boldsymbol{\epsilon} - \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t) \|^2 \right]$$

where $\mathbf{x}_t = \sqrt{\bar{\alpha}_t}\mathbf{x}_0 + \sqrt{1-\bar{\alpha}_t}\boldsymbol{\epsilon}$ is the noisy sample generated using the original data $\mathbf{x}_0$ and the sampled noise $\boldsymbol{\epsilon}$ at a randomly chosen timestep $t$.

Essentially, the network $\boldsymbol{\epsilon}_\theta$ takes a noisy sample $\mathbf{x}_t$ and the timestep $t$ as input and tries to predict the noise component $\boldsymbol{\epsilon}$ that was used to create $\mathbf{x}_t$ from $\mathbf{x}_0$. Training with this objective effectively teaches the network to denoise samples at various noise levels.

Diagram illustrating the fixed forward (noising) process from data $\mathbf{x}_0$ to noise $\mathbf{x}_T$, and the learned reverse (denoising) process generating data $\mathbf{x}_0$ from noise $\mathbf{x}_T$ by predicting the noise $\boldsymbol{\epsilon}_\theta$ at each step.

DDPM Training and Sampling Algorithms

Training:

1. Sample a real data point $\mathbf{x}_0 \sim q(\mathbf{x}_0)$.
2. Sample a timestep $t$ uniformly from $\{1, ..., T\}$.
3. Sample noise $\boldsymbol{\epsilon} \sim \mathcal{N}(0, \mathbf{I})$.
4. Compute the noisy sample $\mathbf{x}_t = \sqrt{\bar{\alpha}_t}\mathbf{x}_0 + \sqrt{1-\bar{\alpha}_t}\boldsymbol{\epsilon}$.
5. Train the neural network $\boldsymbol{\epsilon}_\theta$ by minimizing the loss $\| \boldsymbol{\epsilon} - \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t) \|^2$ using gradient descent. Repeat this process over many iterations and data points.

Sampling (Generation):

1. Start with pure noise $\mathbf{x}_T \sim \mathcal{N}(0, \mathbf{I})$.
2. Iterate backwards from $t=T$ down to $1$: a. Sample $\mathbf{z} \sim \mathcal{N}(0, \mathbf{I})$ if $t > 1$, otherwise $\mathbf{z} = 0$. b. Predict the noise $\boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t)$ using the trained network. c. Compute the denoised sample $\mathbf{x}_{t-1}$ using the reverse transition formula which incorporates the predicted noise: $\mathbf{x}_{t-1} = \frac{1}{\sqrt{\alpha_t}} \left( \mathbf{x}_t - \frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}} \boldsymbol{\epsilon}_\theta(\mathbf{x}_t, t) \right) + \sigma_t \mathbf{z}$ (where $\sigma_t$ is the fixed standard deviation, e.g., $\sqrt{\beta_t}$ or $\sqrt{\tilde{\beta}_t}$).
3. The final result $\mathbf{x}_0$ is the generated sample.

DDPMs provide a solid foundation for understanding modern diffusion-based generative modeling. Their ability to generate high-fidelity samples, particularly in image synthesis, stems from this carefully defined noising process and the learned denoising network trained via the intuitive noise-prediction objective. Subsequent sections will explore variations like score-based interpretations and techniques to improve sampling efficiency and control.