GAN Fundamentals Revisited

Generative Adversarial Networks (GANs), introduced by Ian Goodfellow and colleagues in 2014, represent a powerful class of generative models. Instead of explicitly modeling the probability distribution of the data, GANs learn to generate samples from that distribution through an adversarial process involving two competing neural networks: the Generator and the Discriminator. The core components of GANs and their interaction are explained.

The Generator (G)

The Generator network, denoted as $G$, acts as the creative engine. Its primary function is to synthesize data samples that resemble the real data distribution. Typically, $G$ takes a random noise vector $z$ drawn from a simple prior distribution (like a Gaussian or uniform distribution), $z \sim p_z(z)$, as input. It then processes this noise through a series of transformations, often implemented using deep convolutional layers (specifically, transposed convolutions or "deconvolutions" for image generation), to produce a candidate data sample $G(z)$. The goal of $G$ is to learn the mapping from the latent space (the space of $z$) to the data space such that the generated samples $G(z)$ become indistinguishable from real data samples $x$.

The Discriminator (D)

The Discriminator network, $D$, acts as the critic or judge. Its role is to evaluate the authenticity of a given data sample. $D$ is essentially a binary classifier, usually implemented as a standard feedforward or convolutional neural network. It takes a data sample (either a real sample $x$ from the training dataset or a fake sample $G(z)$ produced by the Generator) as input and outputs a scalar probability $D(x)$ representing the likelihood that the input sample is real (not generated). A value close to 1 suggests the Discriminator believes the sample is real, while a value close to 0 suggests it believes the sample is fake.

The Adversarial Training Game

The training process orchestrates a competitive game between $G$ and $D$. This game can be understood as follows:

1. Discriminator Training: $D$ is trained to improve its ability to distinguish real data from generated data. It is presented with a batch containing both real samples $x$ from the dataset ($x \sim p_{data}(x)$) and fake samples $G(z)$ produced by the current Generator (using random noise $z \sim p_z(z)$). $D$'s weights are updated via gradient ascent to maximize its classification accuracy, effectively pushing $D(x)$ towards 1 and $D(G(z))$ towards 0. During this phase, the Generator's weights are held constant.
2. Generator Training: $G$ is trained to improve its ability to fool the Discriminator. It generates a batch of fake samples $G(z)$. These samples are then passed through the Discriminator. The Generator's weights are updated via gradient descent based on the Discriminator's output $D(G(z))$, aiming to make the Discriminator classify these fake samples as real (pushing $D(G(z))$ towards 1). During this phase, the Discriminator's weights are held constant.

These two steps are alternated iteratively. Over time, $G$ gets better at producing realistic samples, making the task harder for $D$. Simultaneously, $D$ gets better at spotting fakes, pushing $G$ to generate even more convincing outputs. This dynamic competition ideally leads to a state where $G$ generates samples that are statistically indistinguishable from the real data, and $D$ is forced to guess randomly ($D(x) \approx 0.5$).

Flow diagram illustrating the relationship between the Generator, Discriminator, random noise, real data, and generated data in a GAN framework.

The Minimax Objective Function

The adversarial game is formally described by a minimax objective function $V(D, G)$:

$$\min_G \max_D V(D, G) = E_{x \sim p_{data}(x)}[\log D(x)] + E_{z \sim p_z(z)}[\log(1 - D(G(z)))]$$

Let's break this down:

• $E_{x \sim p_{data}(x)}[\log D(x)]$: This term represents the expected output of the Discriminator for real data samples $x$ drawn from the true data distribution $p_{data}(x)$. The Discriminator $D$ aims to maximize this term, meaning it wants $D(x)$ to be close to 1 (correctly identifying real samples).
• $E_{z \sim p_z(z)}[\log(1 - D(G(z)))]$: This term represents the expected output for fake data samples $G(z)$ generated from noise $z$. The Discriminator $D$ also aims to maximize this part of the objective, which corresponds to minimizing $D(G(z))$, pushing it towards 0 (correctly identifying fake samples).
• $\max_D$: The Discriminator tries to maximize the entire expression $V(D, G)$ by becoming better at distinguishing real from fake.
• $\min_G$: The Generator cannot directly influence the first term ($\log D(x)$), so it tries to minimize the overall objective by affecting the second term. Minimizing $E_{z \sim p_z(z)}[\log(1 - D(G(z)))]$ means $G$ aims to make $D(G(z))$ close to 1, effectively fooling the Discriminator into thinking the generated samples are real.

In practice, training $G$ to minimize $\log(1 - D(G(z)))$ can lead to vanishing gradients early in training when $D$ is strong and rejects $G$'s samples with high confidence ($D(G(z))$ is close to 0). A common alternative is to modify the Generator's objective to maximize $\log D(G(z))$ instead. This alternative objective provides stronger gradients early on but maintains the same fixed point in the adversarial game.

This fundamental framework forms the basis for the diverse range of GAN architectures and applications we will explore, including the specific models discussed later in this chapter. Understanding this core adversarial dynamic is essential for diagnosing training issues and appreciating the innovations in more advanced GAN variants.