Motivation: The Need for Faster Sampling

Diffusion models, such as the DDPM and DDIM variants, stand out for their ability to generate high-fidelity images, audio, and other data types. They often produce results with better sample quality and distribution coverage compared to other generative approaches. However, this high quality comes at a significant computational price, primarily during the generation (sampling) phase.

The core mechanism of diffusion models involves a reverse process that iteratively refines a noise sample $x_T \sim \mathcal{N}(0, I)$ back towards a sample from the data distribution, $x_0$. This process typically requires evaluating the model's neural network (often a U-Net or Transformer) numerous times, once for each timestep $t$ in the reverse sequence $T, T-1, \dots, 1$. Standard implementations might use $T=1000$ steps for DDPM. While faster samplers like DDIM can reduce this to perhaps $50-200$ steps by taking larger jumps along the trajectory, this still represents a substantial number of sequential network evaluations.

Consider generating a single image. If each network evaluation takes milliseconds, performing 1000 evaluations translates to seconds per image. Even 50 steps, while a significant improvement, can still represent a barrier compared to models capable of generating samples in a single forward pass. This iterative, sequential dependency makes parallelization across timesteps impossible, fundamentally limiting inference speed.

This multi-step sampling procedure creates several practical limitations:

• Latency: Generating samples takes time, often seconds or even minutes for high-resolution outputs or long sequences. This latency prohibits use in real-time or interactive applications where users expect immediate feedback. Imagine trying to build an interactive image editor or a real-time audio synthesis tool based on a standard diffusion model; the delay would make it unusable.
• Computational Cost: Each sampling step requires a forward pass through a typically large neural network. Accumulating these costs over many steps leads to high overall compute requirements (GPU hours, energy consumption). This impacts the feasibility of deploying these models at scale or on resource-constrained hardware like mobile devices or embedded systems. Training is already resource-intensive; high inference costs add another layer of expense and deployment complexity.
• Throughput: The sequential nature limits the rate at which samples can be generated. This affects applications requiring high throughput, such as generating large synthetic datasets for training other models or rendering frames for video generation.

Approximate comparison of network evaluations needed per sample generation for different approaches. Note the logarithmic scale on the Y-axis, highlighting the orders-of-magnitude difference targeted by faster methods.

The substantial gap between the iterative nature of standard diffusion sampling and the single-step inference common in other generative frameworks has driven research into accelerating the sampling process. While techniques like DDIM and more advanced ODE solvers (which we'll discuss later) offer significant speedups over the original DDPM, the demand for even faster generation, ideally approaching single-step inference without a major drop in the remarkable quality of diffusion models, remains high.

This is the primary motivation behind Consistency Models. The objective is to develop a method that can potentially map noise directly to a high-quality sample in one or very few steps, effectively bypassing the slow, iterative refinement process inherent in traditional diffusion sampling. Subsequent sections will detail how the "consistency property" is defined and leveraged to enable this significant acceleration in generation speed.