While the minimax game between the generator ( $G$ ) and discriminator ( $D$ ) provides a powerful theoretical framework, achieving a stable equilibrium in practice is notoriously difficult. Unlike typical supervised learning problems where we minimize a single, well-behaved loss function, GAN training involves finding a Nash equilibrium in a complex, high-dimensional, non-convex game. This dynamic interplay often leads to several common training instabilities that plagued early GAN development and continue to be areas of active research. Understanding these issues is fundamental before exploring the advanced techniques designed to mitigate them.

Mode Collapse

Perhaps the most frequently encountered and discussed instability is mode collapse. This occurs when the generator fails to capture the full diversity of the real data distribution ( $P_{data}$ ) and instead produces only a limited subset of possible outputs, sometimes collapsing to a single output type regardless of the input noise vector $z$ .

Imagine training a GAN on the MNIST dataset of handwritten digits. Severe mode collapse might result in a generator that only ever produces images resembling the digit '1', ignoring '0', '2' through '9' entirely. Less severe collapse might produce only a few different digits.

Why does it happen? Mode collapse often arises when the generator finds a few specific outputs that are particularly effective at fooling the current discriminator. If the discriminator becomes temporarily too strong or if the optimization dynamics push the generator towards these "safe zones," the generator might over-optimize for these specific outputs. It learns that producing, for example, a passable '1' is less likely to be penalized by the discriminator than attempting a more complex digit like '8' and failing. Once the generator fixates on these limited modes, it can be difficult for the training process to encourage exploration of other parts of the data distribution. The generator essentially gives up on diversity to minimize its immediate loss against the discriminator.

Illustration of mode collapse. The generator ( $G$ ) maps the diverse latent space ( $Z$ ) to only a small subset (a single mode shown here) of the variations present in the real data distribution ( $P_{data}$ ).

The consequence of mode collapse is a generator that produces low-diversity, repetitive samples, failing the primary goal of modeling the true underlying data distribution.

Vanishing Gradients

Another significant challenge, particularly prevalent in the original GAN formulation using a sigmoid cross-entropy loss, is the problem of vanishing gradients. This occurs when the discriminator becomes too proficient too quickly.

Consider the generator's loss function, often related to minimizing $\log(1 - D(G(z)))$ . If the discriminator $D$ becomes very effective, it can easily distinguish real samples from fake ones. For fake samples $G(z)$ , $D(G(z))$ will be close to 0 (indicating "fake" with high confidence). As $D(G(z))$ approaches 0, the function $\log(1 - D(G(z)))$ saturates. That is, its value changes very little even if $D(G(z))$ changes slightly.

Mathematically, the gradient of this loss term with respect to the generator's parameters becomes extremely small.

\nabla_{\theta_g} \log(1 - D_{\phi}(G_{\theta_g}(z)))

When $D_{\phi}(G_{\theta_g}(z)) \approx 0$ , this gradient approaches zero.

What's the impact? When gradients vanish, the generator receives virtually no informative signal from the discriminator about how to improve its outputs. Even if the generated samples are poor, the small gradients mean the generator's weights are updated minimally, effectively halting the learning process for the generator. The discriminator might continue to improve, making the problem even worse.

This is distinct from vanishing gradients sometimes seen in very deep networks due to activation functions or initialization; here, it's a direct consequence of the minimax game dynamics and the choice of loss function when one player significantly outperforms the other.

Non-Convergence and Oscillation

Instead of smoothly converging to an equilibrium where the generator produces realistic samples and the discriminator is unsure ( $D(x) \approx 0.5$ , $D(G(z)) \approx 0.5$ ), GAN training can exhibit oscillatory behavior or fail to converge altogether.

The losses of the generator and discriminator might fluctuate wildly over training iterations, with improvements in one network potentially causing instability or performance degradation in the other. This is because the optimization landscape is not fixed; as the generator updates, it changes the problem the discriminator is trying to solve, and vice-versa. Finding a stable point (a Nash equilibrium) in this constantly shifting landscape is difficult.

Representation of oscillating loss values during GAN training, where improvements in one network might correspond to increased loss in the other, preventing stable convergence.

Factors like learning rates, optimizer choices, and network architectures can significantly influence convergence behavior. Simply monitoring the loss values is often insufficient to diagnose GAN training progress, as lower loss doesn't always correlate directly with better sample quality or stability in this adversarial setting. Qualitative evaluation of generated samples and the use of specialized evaluation metrics (discussed in Chapter 5) become necessary.

These instabilities highlight that training GANs requires more than just applying standard deep learning optimization recipes. The adversarial nature necessitates careful consideration of the game dynamics, loss functions, network architectures, and optimization strategies, motivating the development of the advanced techniques we will explore in subsequent chapters.