ApX logoApX logo
ModelsCalculatorHardwareCoursesPricing
ApX logoApX logo
ModelsCalculatorHardwareCoursesPricing

All Courses

Introduction to Normalizing Flows
Chapter 1: Mathematics of Invertible Transformations
Random Variables and Distributions
The Change of Variables Theorem
Computing Jacobian Determinants
Forward and Inverse Passes
Calculating Jacobians Practice
Chapter 2: Building Normalizing Flows
Defining a Normalizing Flow
Stacking Transformations
Planar and Radial Flows
Choosing Base Distributions
Implementing Planar Flows Practice
Chapter 3: Autoregressive Flows
Autoregressive Generative Models
Masked Autoencoders for Distributions
Masked Autoregressive Flow
Inverse Autoregressive Flow
Building Autoregressive Models Practice
Chapter 4: Coupling Architectures
Affine Coupling Layers
The RealNVP Architecture
Multi-Scale Architectures
ActNorm and Invertible Convolutions
Implementing a Coupling Layer Practice
Chapter 5: Training Flow Models
Maximum Likelihood Estimation
Loss Functions for Density Estimation
Data Dequantization Methods
Sampling Procedures
Training a Normalizing Flow Practice

The RealNVP Architecture

The RealNVP architecture, which stands for Real-valued Non-Volume Preserving transformations, scales affine coupling layers to high-dimensional datasets like images. Instead of simply slicing a one-dimensional array in half, RealNVP introduces structured ways to partition multi-dimensional data using binary masks.

When dealing with spatial data such as images, explicitly splitting a tensor into two separate tensors becomes complex and computationally inefficient. RealNVP simplifies this by applying a binary mask. A binary mask is a tensor of the exact same shape as the input, filled entirely with zeros and ones. We use this mask to explicitly define which features pass through the layer unchanged and which features undergo the affine transformation.

Let bbb represent this binary mask. We can rewrite the standard affine coupling operation using element-wise multiplication.

y=b⊙x+(1−b)⊙(x⊙exp⁡(s(b⊙x))+t(b⊙x))y = b \odot x + (1 - b) \odot (x \odot \exp(s(b \odot x)) + t(b \odot x))y=b⊙x+(1−b)⊙(x⊙exp(s(b⊙x))+t(b⊙x))

In this equation, the elements where b=1b=1b=1 remain exactly as they were in the input xxx. The elements where b=0b=0b=0 are modified by the scaling function sss and the translation function ttt. Because sss and ttt only receive the unmodified parts of the input, the Jacobian matrix remains triangular, allowing for fast determinant calculation and inversion.

To effectively capture the relationships between different pixels in an image, the authors of RealNVP proposed two specific masking patterns: checkerboard masking and channel-wise masking.

In checkerboard masking, the spatial dimensions of the image are masked in an alternating pattern that resembles a standard chess board. A pixel assigned a mask value of 1 is directly adjacent to pixels with a mask value of 0. This arrangement forces the model to use the immediate spatial context surrounding a pixel to predict its transformation parameters.

In channel-wise masking, the spatial dimensions remain entirely intact. The split occurs along the depth dimension of the image tensor. If a specific intermediate tensor contains 64 channels, the first 32 channels are assigned a mask value of 1, and the remaining 32 channels are assigned a mask value of 0.

A single coupling layer only updates a fraction of the input data. If you use the exact same mask for every layer in your network, the elements assigned a 1 will never be transformed. To solve this, RealNVP architectures stack multiple coupling layers and reverse the binary mask between operations. If a pixel passes through unchanged in the first layer, the inverted mask in the second layer ensures it gets transformed, using the newly updated pixels as its conditioning context.

Alternating mask operation across two consecutive coupling layers to ensure all input variables are eventually transformed.

The functions sss and ttt are typically implemented as deep convolutional neural networks, often utilizing residual blocks. An important property of the RealNVP architecture is that these internal networks do not need to be invertible themselves. The invertibility of the entire model is mathematically guaranteed by the coupling architecture alone. This allows you to use standard, highly expressive neural network layers for sss and ttt without worrying about their individual mathematical properties.

When training these architectures, numerical stability requires careful attention. The affine coupling equation relies on exponentiating the output of the scaling network, exp⁡(s(x))\exp(s(x))exp(s(x)). If the network sss outputs a large positive number, the exponential function will explode, causing gradient calculations to fail. Implementations often apply a tanh activation function to the output of sss, which constrains the values between -1 and 1. This constrained output is then scaled by a learnable parameter to allow the model sufficient flexibility while preventing explosive numerical values during maximum likelihood estimation.

© 2026 ApX Machine Learning