Generating Adversarial Text for NLP Models

While adversarial examples are well-studied in computer vision, manipulating text data presents unique challenges and requires different approaches. Unlike the continuous pixel values in images, text is composed of discrete units like characters or words (tokens). This discrete nature means we cannot simply add small, gradient-calculated perturbations as we do with images using methods like FGSM or PGD. A tiny change to a word embedding vector doesn't necessarily correspond to another valid word.

Therefore, generating adversarial text involves modifying the discrete sequence of tokens while aiming to achieve two primary goals:

1. Induce Misclassification: Cause the target NLP model (e.g., sentiment analyzer, text classifier, translation model) to produce an incorrect output.
2. Maintain Utility/Plausibility: Ensure the modified text remains grammatically correct, semantically similar to the original (to fool humans or downstream tasks), and fluent.

Achieving both goals simultaneously is difficult. Aggressive modifications might easily fool a model but become nonsensical to a human reader. Subtle changes might preserve meaning but fail to alter the model's prediction.

Attack Strategies for Text

Adversarial attacks on NLP models typically operate at the character, word, or sentence level.

Character-Level Perturbations

These attacks involve making small changes to the characters within words. Common techniques include:

• Insertion: Adding characters (e.g., "great" -> "greaat").
• Deletion: Removing characters (e.g., "positive" -> "postive").
• Substitution: Swapping characters, often adjacent ones (e.g., "review" -> "reveiw") or visually similar ones (e.g., replacing 'l' with '1' or 'o' with '0').
• Unicode Homoglyphs: Using visually identical characters from different Unicode blocks.

Character-level attacks can be subtle but may be easily caught by spell-checkers or simple preprocessing steps. They can sometimes disrupt the tokenization process, potentially impacting downstream model performance significantly.

Word-Level Perturbations

These are currently the most common and often more effective strategies. They involve replacing, inserting, or deleting entire words.

• Synonym Substitution: This is a widely used technique. Words in the original text are replaced with synonyms. The core challenge lies in selecting synonyms that:

  • Are semantically appropriate in the context.
  • Preserve the original meaning sufficiently.
  • Are effective in changing the model's prediction.

  Finding suitable candidates often involves resources like WordNet or leveraging word embeddings (e.g., Word2Vec, GloVe, BERT embeddings) to find words close in the embedding space. However, closeness in embedding space doesn't always guarantee contextual appropriateness or semantic equivalence.

• Word Insertion/Deletion: Adding neutral words (e.g., "the", "a", "is") or deleting seemingly unimportant words can sometimes be enough to flip a model's prediction, especially if the model relies heavily on specific keywords or sequence patterns.

• Word Reordering: Changing the order of words, although this often impacts grammar and meaning significantly.

A typical word-level attack workflow involves:

1. Identify Important Words: Determine which words in the input text have the most influence on the model's prediction. This can sometimes be approximated using gradient-based saliency methods (calculating gradients with respect to word embeddings) or occlusion-based methods (measuring prediction change when a word is removed).
2. Generate Candidate Replacements: For the identified important words, find potential substitutes (e.g., synonyms, nearby words in embedding space).
3. Select Best Candidates: Choose replacements that maximize the chance of misclassification while minimizing semantic deviation and maintaining grammaticality. This often involves a search strategy.

Sentence-Level Perturbations

These attacks modify the structure or content at the sentence level.

• Paraphrasing: Generating alternative sentences or phrases that convey the same meaning but use different wording and structure. This can be done using rule-based systems, back-translation (translating to another language and back), or generative models trained for paraphrasing. These attacks are often highly effective and stealthy but computationally more expensive to generate.
• Sentence Insertion/Deletion: Adding or removing entire sentences, potentially altering the context significantly.

Search and Optimization

Since the search space for text modifications is immense and discrete, finding the optimal perturbation often requires heuristic search algorithms.

• Greedy Search: Iteratively make the single best modification (e.g., the synonym replacement that most increases the target incorrect class probability) until the attack succeeds or a limit is reached.
• Beam Search: Maintain multiple promising modification sequences (beams) at each step, exploring a wider part of the search space than greedy search.

The objective function during the search usually balances the model's prediction score for the target (incorrect) class against constraints related to semantic similarity, grammaticality, and the number of modifications.

A simplified example of a word-level attack using synonym substitution to flip the predicted sentiment of a sentence. The attacker identifies an influential word ("fantastic"), finds synonyms (including one with opposite sentiment, "terrible"), and substitutes it to create an adversarial example.

Constraints and Evaluation

Unlike image perturbations measured by $L_p$ norms, evaluating text perturbations requires different metrics:

• Perturbation Size: Number or percentage of modified characters/words.
• Semantic Similarity: Measured using embedding cosine similarity (e.g., Universal Sentence Encoder) or human evaluation.
• Grammaticality: Assessed using automated tools or human judgment.
• Attack Success Rate: Percentage of examples successfully causing misclassification.

Frameworks like TextAttack provide implementations of various text-based adversarial attacks and evaluation tools, facilitating research and benchmarking in this area. Generating effective and inconspicuous adversarial text remains a significant challenge due to the discrete, structured, and semantic nature of language.