Generating Adversarial Text: Practice

Generating adversarial examples for text presents unique challenges compared to the continuous domain of images. Text is inherently discrete, composed of characters, words, and sentences. Perturbations must often operate at the word or character level, making direct gradient application difficult. Furthermore, effective adversarial text often needs to preserve semantic meaning and grammatical fluency to be convincing or to bypass human moderation filters. Practical guidance and examples are provided for crafting adversarial text targeting NLP models.

We will focus on common strategies and demonstrate how to implement them, often leveraging specialized libraries designed for NLP adversarial attacks. Assume you have a working Python environment with standard machine learning libraries (transformers, torch or tensorflow, datasets) and potentially an NLP attack library like TextAttack.

Core Strategies for Text Perturbation

Adversarial text generation typically involves modifying an input text $x$ to create a perturbed version $x_{adv}$ such that a target model $f$ misclassifies $x_{adv}$ (e.g., changing sentiment from positive to negative), while $x_{adv}$ remains semantically similar to $x$ and often grammatically correct. Common perturbation strategies operate at different granularities:

1. Word-Level Modifications

These are among the most popular techniques due to their potential to significantly alter model predictions while maintaining readability.

• Synonym Substitution: Replace words with synonyms. The core idea is that synonyms should preserve the original meaning but might have different embeddings or impacts on the model's internal representations.

  • Process: Identify candidate words (often based on importance scores or simply non-stopwords). Find synonyms (using resources like WordNet or nearest neighbors in embedding space). Replace the original word with a synonym, often checking constraints like part-of-speech (POS) tagging consistency.
  • Example: "The film was excellent." -> "The film was superb."
  • Challenge: Ensuring the chosen synonym fits the context and doesn't subtly change the meaning in a way unintended by the attacker. Embeddings might find synonyms that are technically close but contextually wrong.

• Word Insertion/Deletion: Add or remove words that seem insignificant to a human reader but can disrupt the model's processing.

  • Process: Insert neutral words (e.g., ' DUMMY ', ' oh ') or delete seemingly unimportant words (e.g., punctuation, some adjectives/adverbs).
  • Example: "The movie was good." -> "The movie, oh, was good."
  • Challenge: Maintaining grammatical correctness and avoiding nonsensical sentences, especially with insertions.

• Word Reordering: Change the order of words or phrases. This can sometimes confuse models, especially older sequence models, without significantly altering meaning for humans.

  • Example: "It was not good, actually bad." -> "Actually bad, it was not good."
  • Challenge: Often results in ungrammatical sentences if not done carefully.

2. Character-Level Modifications

These attacks introduce small changes at the character level, akin to typos. They can be effective against models sensitive to surface forms or character-level CNNs/RNNs.

• Common Operations: Character insertion, deletion, substitution (typos), swap (adjacent characters).
• Example: "The movie was great." -> "The movie was graet." or "The movie was grea t."
• Challenge: Can make text look unnatural or be easily caught by spell checkers. Effectiveness depends heavily on the target model's architecture and training data.

Search Algorithms for Finding Adversarial Text

Since we cannot directly compute gradients with respect to discrete word choices, finding the best sequence of perturbations requires search algorithms. The goal is typically to minimize the number of changes (perturbation distance) while maximizing the attack success (e.g., causing a misclassification).

• Greedy Search (Word Substitution): This is a common approach.

  1. Score words in the input text based on their importance (e.g., how much deleting them changes the model's prediction probability for the correct class).
  2. Start with the highest-scoring word.
  3. Try replacing it with candidate synonyms.
  4. Choose the replacement that most effectively degrades the model's confidence in the original label (or increases confidence in a target label), while satisfying constraints (POS match, semantic similarity threshold).
  5. Repeat for the next most important word until the attack succeeds or a perturbation budget is exhausted.

• Beam Search: Keeps track of multiple candidate sequences (beams) of perturbations at each step, potentially finding better solutions than a purely greedy approach but at a higher computational cost.

• Genetic Algorithms: Use concepts like population, mutation (applying perturbations), crossover (combining perturbation strategies), and fitness (attack success/semantic similarity) to evolve effective adversarial examples.

Practical Implementation with TextAttack

Frameworks like TextAttack significantly simplify the process of launching these attacks. TextAttack provides pre-built components:

• Models: Wrappers for models (e.g., from Hugging Face transformers).
• Datasets: Tools for loading standard NLP datasets.
• Transformations: Implementations of perturbation methods (e.g., WordSwapWordNet, CharacterDeletion).
• Constraints: Rules to ensure perturbed text quality (e.g., MaxWordsPerturbed, WordEmbeddingDistance, PartOfSpeech).
• Goal Functions: Define the attack objective (e.g., UntargetedClassification, TargetedClassification).
• Search Methods: Algorithms like GreedyWordSwapWIR (Greedy Word Swap with Word Importance Ranking).
• Attack Recipes: Pre-packaged combinations of the above components, often replicating published attack methods (e.g., TextFoolerJin2019, DeepWordBugGao2018).

Here's an example using TextAttack:

# Python code using TextAttack
# Note: This is illustrative pseudocode, not directly runnable without setup

from textattack.attack_recipes import TextFoolerJin2019
from textattack.datasets import HuggingFaceDataset
from textattack.models.wrappers import HuggingFaceModelWrapper
from textattack import Attacker
from transformers import AutoModelForSequenceClassification, AutoTokenizer

# 1. Load Model and Tokenizer
model_name = "distilbert-base-uncased-finetuned-sst-2-english"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
model_wrapper = HuggingFaceModelWrapper(model, tokenizer)

# 2. Load Dataset (Example: first 10 samples of SST-2 validation)
# Typically you'd load specific data points you want to attack
dataset = HuggingFaceDataset("glue", "sst2", split="validation")
# subset_for_demo = dataset[:10] # Just for illustration

# 3. Choose an Attack Recipe
# TextFooler uses word embeddings for synonym substitution and word importance ranking
attack = TextFoolerJin2019.build(model_wrapper)

# 4. Set up the Attacker
# This dataset provides (text, label) pairs
attacker = Attacker(attack, dataset)

# 5. Run the Attack
results = attacker.attack_dataset()

# 6. Analyze Results
# results is an iterable; each item contains original text, perturbed text,
# original output, perturbed output, etc.
for result in results:
    print(result)
    # Example: Check if successful
    # print(result.perturbed_result.perturbed_text)
    # print(f"Original Output: {result.original_result.output}")
    # print(f"Perturbed Output: {result.perturbed_result.output}")

This workflow allows you to apply sophisticated attacks like TextFooler with minimal code by leveraging the framework's pre-built components.

Evaluating Adversarial Text Quality

Simply causing a misclassification is not enough. A good evaluation considers:

• Attack Success Rate (ASR): Percentage of inputs successfully perturbed to fool the model.
• Perturbation Rate: How much the text was changed (e.g., percentage of words modified, Levenshtein distance). Lower is generally better.
• Semantic Similarity: Does the adversarial text still mean the same thing? Measured computationally using sentence embeddings (e.g., cosine similarity between USE vectors) or through human judgment.
• Grammaticality and Fluency: Is the text grammatically correct and readable? Often assessed using language models (perplexity) or human evaluation.

Flow of generating and evaluating adversarial text using an attack framework. The attack recipe iteratively queries the target model to guide the search for perturbations that fool the model while meeting quality constraints.

Hands-on Practice Suggestions

1. Set up TextAttack: Install the library (pip install textattack).
2. Run a Pre-built Recipe: Use the example above (or TextAttack's documentation examples) to attack a standard model (like distilbert-base-uncased-finetuned-sst-2-english) on a few examples from a known dataset (like GLUE's SST-2).
3. Examine the Outputs: Look closely at the generated adversarial examples. Are they successful? How much were they changed? Do they preserve the meaning? Are they fluent?
4. Try Different Recipes: Experiment with other recipes like DeepWordBugGao2018 (character-level) or BERTAttackLi2020. Compare their effectiveness and the nature of the perturbations.
5. Modify Constraints: Take an existing recipe and adjust its constraints. For example, limit the percentage of words perturbed (MaxModificationRate) or enforce stricter semantic similarity (SentenceEncoder constraint from textattack.constraints.semantics.sentence_encoders). Observe how this affects the ASR and the quality of the generated text.
6. Target a Different Model: Apply the same attack to a different model (e.g., RoBERTa, another fine-tuned BERT variant) and see if the attack transfers or requires adaptation.

This practice will provide concrete experience with the nuances of generating adversarial examples in the challenging domain of natural language processing. Remember that balancing attack effectiveness with semantic preservation and fluency remains an active area of research.