Classical Generative Models Review

Before we examine quantum approaches to generative modeling, let's briefly review two influential classical frameworks: Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs). Understanding their core ideas provides valuable context for their quantum counterparts. The goal in both cases is to learn a model distribution $p_{\text{model}}(x)$ that approximates an unknown data distribution $p_{\text{data}}(x)$, allowing us to generate new samples resembling the original data.

Generative Adversarial Networks (GANs)

GANs, introduced by Ian Goodfellow and colleagues in 2014, employ a clever adversarial training process involving two neural networks:

1. Generator (G): This network takes a random noise vector $z$, typically sampled from a simple distribution like a Gaussian or uniform distribution ($z \sim p_z(z)$), as input. Its goal is to transform this noise into a data sample $G(z)$ that looks like it came from the real data distribution $p_{\text{data}}(x)$.
2. Discriminator (D): This network acts as a classifier. It takes a sample $x$ (which could be a real sample from the training data or a fake sample produced by the Generator) and outputs a probability $D(x)$ representing the likelihood that $x$ is a real sample, not a fake one.

The training process is formulated as a minimax game. The Discriminator is trained to maximize its accuracy in distinguishing real samples from fake samples generated by $G$. Simultaneously, the Generator is trained to minimize the probability that the Discriminator correctly identifies its outputs as fake. Essentially, $G$ tries to fool $D$.

The objective function for this game is typically expressed as:

$$\min_G \max_D V(D, G) = \mathbb{E}_{x \sim p_{\text{data}}(x)}[\log D(x)] + \mathbb{E}_{z \sim p_z(z)}[\log(1 - D(G(z)))]$$

Here, $\mathbb{E}$ denotes expectation. The Discriminator $D$ wants to maximize this value (making $D(x)$ close to 1 for real $x$ and $D(G(z))$ close to 0 for fake samples). The Generator $G$ wants to minimize this value (by making $D(G(z))$ close to 1, effectively fooling the Discriminator).

Basic architecture of a Generative Adversarial Network (GAN), illustrating the interaction between the Generator and Discriminator.

Training GANs can be challenging due to potential instabilities like mode collapse (where the generator produces only a limited variety of samples) or vanishing gradients. However, when successful, GANs are known for generating high-fidelity samples, particularly in domains like image generation.

Variational Autoencoders (VAEs)

VAEs, introduced by Diederik Kingma and Max Welling in 2013, approach generative modeling from a probabilistic perspective, combining ideas from variational inference and autoencoders. A VAE consists of two main parts, often implemented as neural networks:

1. Encoder (Recognition Model): This network, denoted $q_\phi(z|x)$, takes an input data point $x$ and outputs parameters (typically the mean $\mu$ and variance $\sigma^2$) for a probability distribution in a lower-dimensional latent space $Z$. This distribution represents an approximation of the true posterior $p(z|x)$. Often, $q_\phi(z|x)$ is chosen to be a Gaussian: $\mathcal{N}(z | \mu_\phi(x), \sigma^2_\phi(x)I)$.
2. Decoder (Generative Model): This network, denoted $p_\theta(x|z)$, takes a point $z$ sampled from the latent space (either from the approximate posterior $q_\phi(z|x)$ during training or from a prior distribution $p(z)$, usually $\mathcal{N}(0, I)$, during generation) and reconstructs the data point $x$. It defines a distribution over the data space conditioned on the latent variable.

The goal of training a VAE is to maximize the log-likelihood of the data, $\log p(x)$. Since this is often intractable, VAEs maximize a lower bound instead, known as the Evidence Lower Bound (ELBO):

$$\mathcal{L}(\theta, \phi; x) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{KL}(q_\phi(z|x) || p(z))$$

The ELBO consists of two terms:

• Reconstruction Loss: $\mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)]$ encourages the decoder to accurately reconstruct the input $x$ from its latent representation $z$. For continuous data, this often corresponds to minimizing mean squared error; for binary data, binary cross-entropy is common.
• KL Divergence Regularizer: $D_{KL}(q_\phi(z|x) || p(z))$ measures the Kullback Leibler divergence between the approximate posterior distribution learned by the encoder and a predefined prior distribution $p(z)$ over the latent space (e.g., a standard Gaussian). This term acts as a regularizer, forcing the encoded distributions to stay close to the prior, which helps in generating new samples by drawing $z$ directly from $p(z)$.

A specific technique called the "reparameterization trick" is used to allow gradients to flow back through the sampling process within the encoder, enabling training via standard backpropagation.

Basic architecture of a Variational Autoencoder (VAE), showing the encoding, latent sampling, and decoding stages.

VAEs typically provide more stable training compared to GANs and offer an explicit probabilistic model with a well-defined likelihood objective (the ELBO). However, the samples generated by VAEs are sometimes perceived as less sharp or realistic than those from state-of-the-art GANs, potentially due to the simplifying assumptions made (e.g., Gaussian posteriors) and the influence of the reconstruction term.

These classical models establish fundamental concepts in generative modeling: representing distributions implicitly (GANs) or explicitly via latent variables (VAEs), defining objective functions, and navigating training dynamics. As we move into the quantum domain, we'll see how quantum circuits can be used as components within similar frameworks, potentially offering new ways to capture complex data distributions.