Adversarial Training: Principles and Variations

Adversarial training stands as one of the most effective and widely adopted strategies for improving the robustness of machine learning models, particularly deep neural networks, against evasion attacks. The idea behind this method is straightforward: to make a model resilient against adversarial examples, it should be explicitly trained using them. Instead of only training on clean data, the training set is augmented with adversarial examples generated on the fly.

This process directly addresses the vulnerability exploited by attacks like PGD. It forces the model to learn decision boundaries that are less sensitive to small, malicious perturbations in the input space.

The Minimax Optimization Viewpoint

As mentioned in the chapter introduction, adversarial training can be framed formally as a minimax optimization problem. The goal is to find model parameters $\theta$ that minimize the expected loss, even when facing the strongest possible adversary within defined constraints. The objective function is:

$$\min_{\theta} \mathbb{E}_{(x,y) \sim \mathcal{D}} \left[ \max_{\delta \in S} L(\theta, x+\delta, y) \right]$$

Let's break this down:

• $\min_{\theta}$: We want to find the optimal model parameters $\theta$.
• $\mathbb{E}_{(x,y) \sim \mathcal{D}}$: This indicates the expectation over the true data distribution $\mathcal{D}$. In practice, we approximate this by averaging over mini-batches sampled from the training dataset.
• $\max_{\delta \in S}$: This is the inner maximization problem, representing the adversary's goal. The adversary tries to find the perturbation $\delta$ within an allowed set $S$ (e.g., an $L_p$-norm ball like $S = \{\delta \mid ||\delta||_p \leq \epsilon\}$) that maximizes the loss function $L$. This corresponds to finding the "worst-case" adversarial example near the clean input $x$.
• $L(\theta, x+\delta, y)$: This is the loss function (e.g., cross-entropy for classification) evaluated on the perturbed input $x+\delta$ with the true label $y$, using the current model parameters $\theta$.

Essentially, the outer minimization seeks model parameters that perform well (low loss) even after the inner maximization finds the most effective adversarial perturbation for each data point. This creates a dynamic where the model learns to defend against the attacks it's subjected to during training.

Practical Implementation: PGD Adversarial Training

Solving the exact inner maximization $\max_{\delta \in S} L(\theta, x+\delta, y)$ is often intractable. In practice, it's approximated using iterative gradient-based attack methods. Projected Gradient Descent (PGD) is the most common choice, leading to what's known as PGD Adversarial Training (PGD-AT).

The PGD-AT process within a single training step for a mini-batch typically looks like this:

1. Sample a mini-batch: Get a set of clean examples $(x, y)$ from the training data.
2. Generate Adversarial Examples (Inner Loop - Attack): For each clean example $x$ in the mini-batch:
   • Initialize a small random perturbation $\delta_0$.
   • Iteratively update the perturbation for $k$ steps using PGD: $$\delta_{t+1} = \Pi_S \left( \delta_t + \alpha \cdot \text{sign}(\nabla_\delta L(\theta, x+\delta_t, y)) \right)$$ where $\alpha$ is the step size, $\nabla_\delta L$ is the gradient of the loss with respect to the perturbation $\delta$, and $\Pi_S$ is the projection operator that ensures $\delta_{t+1}$ stays within the allowed set $S$ (e.g., clipping values to remain within an $L_\infty$ ball of radius $\epsilon$).
   • The final perturbation $\delta_k$ yields the adversarial example $x' = x + \delta_k$.
3. Update Model Parameters (Outer Loop - Training): Compute the loss using the generated adversarial examples $x'$ and their true labels $y$. Calculate the gradient of this loss with respect to the model parameters $\theta$, and update $\theta$ using an optimizer (like SGD or Adam).

This cycle repeats for many epochs. The strength of the PGD attack used in the inner loop (controlled by $\epsilon$, $\alpha$, and $k$) is a significant hyperparameter. A stronger attack during training generally leads to a more stable model, but also increases computational cost and can sometimes make training difficult.

Variations on Adversarial Training

While PGD-AT is a standard, several variations and improvements exist:

• FGSM Adversarial Training (FGSM-AT): A faster alternative where the inner loop uses only a single step of the Fast Gradient Sign Method (FGSM) instead of PGD. While computationally cheaper, it often results in less robust models compared to PGD-AT. Sometimes, it suffers from a phenomenon called "catastrophic overfitting" where robustness against multi-step PGD attacks suddenly collapses during training, even though robustness against the single-step FGSM attack improves.
• Fast Adversarial Training Methods: Techniques like "Free AT" and "You Only Propagate Once (YOPO)" aim to reduce the computational overhead. They cleverly reuse gradient computations, updating both the perturbation $\delta$ and the model parameters $\theta$ with a single backward pass per step, significantly speeding up training compared to standard PGD-AT.
• TRADES (TRadeoff-inspired Adversarial DEfense via Surrogate-loss): This method explicitly addresses the common trade-off between standard accuracy (on clean data) and adversarial robustness. It modifies the loss function to include a regularization term based on the Kullback-Leibler (KL) divergence between the model's output distribution for the clean input and the adversarial input: $$L_{TRADES}(\theta, x, y) = L_{CE}(\theta, x, y) + \beta \cdot KL(p_{\theta}(x) || p_{\theta}(x'))$$ Here, $L_{CE}$ is the standard cross-entropy loss on the clean example, $x'$ is the adversarial example generated via an inner PGD loop (optimizing the KL term), $p_{\theta}(x)$ is the model's predicted probability distribution for input $x$, and $\beta$ is a hyperparameter controlling the balance between standard accuracy and robustness. TRADES encourages the model to produce similar outputs for clean and adversarial inputs.

Understanding the Effects

Adversarial training fundamentally changes how the model learns. Instead of just fitting the training data, it must also learn to be invariant to small perturbations designed to fool it. This often leads to:

• Smoother Decision Boundaries: The model becomes less sensitive to tiny input changes near data points.
• More Perceptually Aligned Gradients: Gradients might become less noisy and potentially align better with human perception, although this is an area of ongoing research.
• Feature Learning: The model may learn features that are inherently more stable and representative of the true class, rather than relying on superficial correlations easily exploited by adversaries.

The Accuracy-Robustness Trade-off

A common observation with adversarial training is a trade-off: increasing robustness against adversarial attacks often leads to a decrease in accuracy on clean, unperturbed examples. Standard training optimizes solely for clean accuracy, while adversarial training optimizes for worst-case performance within a perturbation radius, which can pull the decision boundary away from some clean examples.

Models trained with stronger adversarial methods (like PGD-AT) typically achieve higher robustness but may sacrifice some standard accuracy compared to models with standard training or weaker defenses. Techniques like TRADES aim to find a better balance.

Challenges

While effective, adversarial training presents challenges:

• Computational Cost: Generating adversarial examples for every batch significantly increases training time (often by a factor of 5-10x or more compared to standard training).
• Hyperparameter Tuning: The effectiveness depends heavily on the choice of attack parameters used during training ($\epsilon$, $\alpha$, $k$). These need careful tuning. Using too weak an attack results in insufficient robustness, while too strong an attack might hinder convergence or excessively degrade clean accuracy.
• Generalization: Robustness learned against one type of attack (e.g., $L_\infty$ PGD) might not fully generalize to other attack types or perturbation norms.

Despite these challenges, adversarial training, particularly PGD-AT and its sophisticated variants like TRADES, remains a foundation technique for building more secure machine learning models against evasion attacks. It provides a principled way to directly incorporate robustness into the learning process.