Adversarial Training for Transformer‑Based Natural Language Models: Methods, Variants, and Experimental Results

In this talk, the speaker (a Ph.D. student from the University of Maryland) introduces the work done during a summer internship at Microsoft on improving natural‑language‑understanding models through adversarial training.

Adversarial training overview : Small perturbations to inputs can cause mis‑classification in both image and text models. Originally used for image classifiers, the concept extends to NLP where adversarial examples are generated by word substitutions, synonym replacements, or character‑level changes.

For image classifiers, adversarial training involves generating perturbations δ that maximize loss while keeping ‖δ‖<ε, typically solved by Projected Gradient Descent (PGD). This improves robustness but may slightly reduce clean‑data accuracy.

Adversarial training for NLP : Generating adversarial text examples is harder; common approaches replace words with synonyms or apply semantic‑equivalent rules (SEARs). These methods preserve meaning but may be less effective than image attacks. Back‑translation scores can filter unrealistic examples but are computationally expensive.

Standard PGD and variants : The baseline uses K‑step PGD (K‑PGD) on word embeddings. FreeAT performs a gradient‑ascent step on the input and simultaneously updates model parameters, reducing the number of forward‑backward passes. YOPO fixes the gradient of the first layer and performs multiple inner steps, achieving similar adversarial strength with fewer passes.

Proposed FreeLB method : The authors adopt K‑step gradient ascent on embeddings, accumulate the gradients, and update parameters only after K steps, avoiding the stale‑parameter issue of FreeAT. This approach resembles parallel optimization and yields higher GLUE scores (≈1.1% on BERT‑base, 0.3% on RoBERTa).

Experimental results : Comparisons with YOPO show that inner‑step mechanisms do not consistently improve performance. FreeLB consistently outperforms other adversarial training variants across multiple GLUE tasks, demonstrating better robustness without sacrificing clean‑data accuracy.

Conclusion : Adversarial training applied during fine‑tuning significantly enhances the generalization and robustness of transformer‑based language models. Future work aims to design efficient adversarial training for the pre‑training phase, potentially using large‑batch strategies.