Physical Adversarial Attacks

Most adversarial attacks we've discussed so far operate in the digital domain. An attacker manipulates pixel values directly, often assuming perfect knowledge of the input the model will receive. However, many AI systems interact with the physical environment through sensors like cameras. This transition from digital bits to physical objects introduces a new set of significant challenges and opportunities for attackers. Physical adversarial attacks aim to create perturbations that remain effective when deployed in the real environment, captured by a sensor, and then processed by the target model.

The Digital-to-Physical Gap

Creating effective physical attacks is considerably harder than crafting digital ones. The core difficulty lies in the digital-to-physical gap: a perturbation optimized digitally might fail in the real environment due to numerous uncontrolled factors.

Consider an attack on an image classifier. A digital attack optimizes a perturbation pattern $\delta$ such that $f(x + \delta)$ results in misclassification, where $x$ is the digital image and $f$ is the model. In practice, the attacker creates a physical object (e.g., a printed sticker, modified glasses) embodying the perturbation. This object is then placed in the environment and captured by a camera. The process looks more like:

$physical\_object \rightarrow environment \xrightarrow{\text{capture}} sensor \rightarrow preprocessing \rightarrow x' \rightarrow f(x')$

Several factors can degrade or alter the adversarial pattern during this process:

1. Environmental Conditions: Lighting variations (intensity, color temperature, shadows), weather effects (rain, fog), and occlusions can drastically change how the object appears.
2. Sensor Noise and Processing: Camera sensors introduce noise. Internal camera software might apply transformations like color correction, gamma correction, or compression (e.g., JPEG), which are often unknown and non-differentiable.
3. Viewpoint Variations: The object will be viewed from different angles, distances, and perspectives, causing geometric distortions (affine, projective transformations).
4. Fabrication Constraints: Physical reproduction isn't perfect. Printers have limited color gamuts (printable colors), resolution limits, and material properties affect reflectance and appearance. Applying a perturbation might involve stickers, paint, or 3D printing, each with its own constraints.

Techniques for Physical Attacks

To overcome these challenges, physical attacks must be strong to a wide range of transformations and environmental conditions. A prominent technique for achieving this is Expectation Over Transformation (EOT).

Expectation Over Transformation (EOT)

Introduced by Athalye et al. (2018), EOT aims to create adversarial examples that are effective under a distribution of transformations $T$. Instead of optimizing the perturbation $\delta$ for a single input $x$, EOT optimizes it such that the expected loss over a set of transformations $t \in T$ is maximized.

The objective function becomes:

$\max_{\delta} \mathbb{E}_{t \sim T} [ \mathcal{L}(f(t(x + \delta)), y_{target}) ]$

Here, $\mathcal{L}$ is the loss function (e.g., cross-entropy towards a target class $y_{target}$), $f$ is the model, $x$ is the original input (or a representation of the object being perturbed), $\delta$ is the perturbation, and $t$ represents a transformation sampled from the distribution $T$.

The distribution $T$ typically includes transformations expected in the physical setting:

• Rotations and translations
• Scaling (zoom)
• Brightness and contrast adjustments
• Perspective changes
• Addition of noise

By optimizing over this distribution, the resulting perturbation $\delta$ is more likely to survive these variations when deployed physically. EOT is often implemented by sampling transformations $t$ at each step of an iterative optimization process (like PGD).

Examples of Physical Attacks

Research has demonstrated successful physical attacks across various computer vision tasks:

1. Attacking Image Classifiers (e.g., Traffic Signs): Elovic et al. (2017) showed that carefully crafted stickers placed on a stop sign could cause classifiers to misinterpret it as a speed limit sign or other objects, even when viewed from different angles and distances. These attacks often use techniques like EOT.

   Diagram illustrating the process of a physical attack on a stop sign classifier using an adversarial sticker.

2. Fooling Facial Recognition: Adversarial patterns can be incorporated into eyeglass frames (Sharif et al., 2016) or worn as patches (Thys et al., 2019) to cause misidentification (dodging recognition) or impersonation. These often require careful optimization considering facial landmarks and 3D structure.

3. Attacking Object Detectors: Adversarial patches can be designed to make detectors fail to recognize specific objects (e.g., making a person invisible to a YOLO detector) or to cause hallucinated object detections. These patches often need to be relatively large and carefully placed.

Challenges in Creating and Evaluating Physical Attacks

While techniques like EOT improve robustness, physical attacks still face hurdles:

• Optimization Complexity: Including physical transformations (especially non-differentiable ones like sensor noise or complex lighting) makes optimization much harder. Surrogates or approximations are often needed.
• Fabrication Fidelity: Accurately reproducing the optimized digital perturbation is challenging. Color differences between digital specification and printed output (the "color gamut" problem) are common.
• Evaluation Cost and Effort: Rigorous evaluation requires extensive real-world testing under diverse conditions, which is time-consuming and resource-intensive compared to digital evaluations. Simulators can help but may not fully capture real-world complexities.
• Detection and Mitigation: Physical perturbations are often visually noticeable (e.g., large, strangely patterned stickers), making them potentially detectable by humans or specialized defense mechanisms. However, more subtle physical attacks are an active area of research.

Physical adversarial attacks highlight the importance of evaluating model security not just in simulation but also under realistic deployment conditions. They represent a significant step towards understanding the practical security implications of machine learning in systems interacting with the physical environment. As AI deployment in areas like autonomous driving and surveillance increases, understanding and defending against these attacks becomes increasingly important.