Adversarial Examples for Evaluating Reading Comprehension Systems

2017/07/23 by Robin Jia, Percy Liang, Jia, Robin +1 · 89 citations
Computer Science · #Adversarial Robustness in Machine Learning #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1707.07328

openalex publication_date 2017/07/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Standard accuracy metrics indicate that reading comprehension systems are making rapid progress, but the extent to which these systems truly understand language remains unclear. To reward systems with real language understanding abilities, we propose an adversarial evaluation scheme for the Stanford Question Answering Dataset (SQuAD). Our method tests whether systems can answer questions about paragraphs that contain adversarially inserted sentences, which are automatically generated to distract computer systems without changing the correct answer or misleading humans. In this adversarial setting, the accuracy of sixteen published models drops from an average of 75% F1 score to 36%; when the adversary is allowed to add ungrammatical sequences of words, average accuracy on four models decreases further to 7%. We hope our insights will motivate the development of new models that understand language more precisely.

Citations

• Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering
• Structural Embedding of Syntactic Trees for Machine Comprehension

Cited by

• Beyond Context: Large Language Models' Failure to Grasp Users' Intent
• Bounding Hallucinations: Information-Theoretic Guarantees for RAG Systems via Merlin-Arthur Protocols
• Universal Adversarial Suffixes Using Calibrated Gumbel-Softmax Relaxation
• DAMASHA: Detecting AI in Mixed Adversarial Texts via Segmentation with Human-interpretable Attribution
• Analyzing and Mitigating Negation Artifacts using Data Augmentation for Improving ELECTRA-Small Model Accuracy
• Cache Mechanism for Agent RAG Systems
• FPT-Noise: Dynamic Scene-Aware Counterattack for Test-Time Adversarial Defense in Vision-Language Models
• CMT-Bench: Cricket Multi-Table Generation Benchmark for Probing Robustness in Large Language Models
• Can Transformer Memory Be Corrupted? Investigating Cache-Side Vulnerabilities in Large Language Models
• Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answering
• Adversarial Robustness in One-Stage Learning-to-Defer
• Rethinking RL Evaluation: Can Benchmarks Truly Reveal Failures of RL Methods?
• ConDABench: Interactive Evaluation of Language Models for Data Analysis
• Evaluating the Robustness of a Production Malware Detection System to Transferable Adversarial Attacks
• Probing Pre-trained Language Models on Code Changes: Insights from ReDef, a High-Confidence Just-in-Time Defect Prediction Dataset
• SALMAN: Stability Analysis of Language Models Through the Maps Between Graph-based Manifolds
• MultiWikiQA: A Reading Comprehension Benchmark in 300+ Languages
• Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder
• Can Out-of-Distribution Evaluations Uncover Reliance on Shortcuts? A Case Study in Question Answering
• Can NLI Models Verify QA Systems' Predictions?
• Adversarial Attack on Graph Structured Data
• How Causal Abstraction Underpins Computational Explanation
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• Special-Character Adversarial Attacks on Open-Source Language Model
• Semantics Preserving Adversarial Learning
• Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classification
• HeQ: a Large and Diverse Hebrew Reading Comprehension Benchmark
• Adversarial Defence without Adversarial Defence: Enhancing Language Model Robustness via Instance-level Principal Component Removal
• ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion
• The Importance of Being Recurrent for Modeling Hierarchical Structure
• Exploring Neural Models for Parsing Natural Language into First-Order Logic
• Adversarial machine learning :
• Verification of Neural Network Control Policy Under Persistent Adversarial Perturbation
• Counterfactual Variable Control for Robust and Interpretable Question Answering
• Efficient and Robust Question Answering from Minimal Context over Documents
• Small Edits, Big Consequences: Telling Good from Bad Robustness in Large Language Models
• A Survey on Neural Machine Reading Comprehension
• Robust Multilingual Part-of-Speech Tagging via Adversarial Training
• Visual Referring Expression Recognition: What Do Systems Actually Learn?
• Mitigating Shortcut Learning with InterpoLated Learning
• PredCoin: Defense against Query-based Hard-label Attack
• Answer Generation through Unified Memories over Multiple Passages
• An Empirical Evaluation of Adversarial Robustness under Transfer Learning
• Robust Reading Comprehension with Linguistic Constraints via Posterior Regularization
• Imbalanced Sentiment Classification Enhanced with Discourse Marker
• Towards Fast Computation of Certified Robustness for ReLU Networks
• Adversarial Attack Classification and Robustness Testing for Large Language Models for Code
• TAI3: Testing Agent Integrity in Interpreting User Intent
• Golden Grain: Building a Secure and Decentralized Model Marketplace for MLaaS
• Commonsense knowledge adversarial dataset that challenges ELECTRA
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Reinforced Mnemonic Reader for Machine Reading Comprehension
• Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification
• SeqPE: Transformer with Sequential Position Encoding
• Attention-based Adversarial Robust Distillation in Radio Signal Classifications for Low-Power IoT Devices
• Adversarial Attacks for Multi-view Deep Models
• Normative Conflicts and Shallow AI Alignment
• SUCEA: Reasoning-Intensive Retrieval for Adversarial Fact-checking through Claim Decomposition and Editing
• Interactive Language Learning by Question Answering
• Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
• Daedalus: Breaking Non-Maximum Suppression in Object Detection via Adversarial Examples
• Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
• Adversarially Robust Generalization Requires More Data
• Artificial Intelligence Strategies for National Security and Safety Standards
• Spurious Correlations and Beyond: Understanding and Mitigating Shortcut Learning in SDOH Extraction with Large Language Models
• Evaluating Gender Bias in Natural Language Inference
• TRAPDOC: Deceiving LLM Users by Injecting Imperceptible Phantom Tokens into Documents
• Knowing More About Questions Can Help: Improving Calibration in Question Answering
• Geometry matters: Exploring language examples at the decision boundary
• Unshuffling Data for Improved Generalization
• Tunneling Neural Perception and Logic Reasoning through Abductive Learning
• Evaluating the Retrieval Robustness of Large Language Models
• Generating Semantically Valid Adversarial Questions for TableQA
• Look Within or Look Beyond? A Theoretical Comparison Between Parameter-Efficient and Full Fine-Tuning
• Exploring the Robustness of NMT Systems to Nonsensical Inputs
• On VLMs for Diverse Tasks in Multimodal Meme Classification
• Using Adversarial Attacks to Reveal the Statistical Bias in Machine Reading Comprehension Models
• Robust Machine Comprehension Models via Adversarial Training
• Mapping to Declarative Knowledge for Word Problem Solving
• Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• YESciEval: Robust LLM-as-a-Judge for Scientific Question Answering
• Investigating the Vulnerability of LLM-as-a-Judge Architectures to Prompt-Injection Attacks
• SPIRIT: Patching Speech Language Models against Jailbreak Attacks
• Beyond Single-Point Judgment: Distribution Alignment for LLM-as-a-Judge
• Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviors
• A Study of the Tasks and Models in Machine Reading Comprehension
• Can Edge Probing Tasks Reveal Linguistic Knowledge in QA Models?
• QAInfomax: Learning Robust Question Answering System by Mutual Information Maximization
• Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models

Related

• Power in Numbers: Robust reading comprehension by finetuning with four adversarial sentences per example
• The Impacts of Unanswerable Questions on the Robustness of Machine Reading Comprehension Models
• Using Adversarial Attacks to Reveal the Statistical Bias in Machine Reading Comprehension Models
• Robust Machine Comprehension Models via Adversarial Training
• Trick Me If You Can: Human-in-the-loop Generation of Adversarial Examples for Question Answering