An Analysis of Negation in Natural Language Understanding Corpora

2022/03/16 by Md Mosharaf Hossain, Hossain, Md Mosharaf, Dhivya Chinnappa +3 · 5 citations
Computer Science · #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.2203.08929

To appear in the proceedings of ACL 2022 (main conference)

arxiv created 2022/03/16 · arxiv updated 2022/03/18

Abstract

This paper analyzes negation in eight popular corpora spanning six natural language understanding tasks. We show that these corpora have few negations compared to general-purpose English, and that the few negations in them are often unimportant. Indeed, one can often ignore negations and still make the right predictions. Additionally, experimental results show that state-of-the-art transformers trained with these corpora obtain substantially worse results with instances that contain negation, especially if the negations are important. We conclude that new corpora accounting for negation are needed to solve natural language understanding tasks when negation is present.

Cited by

• What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Features
• Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?
• The Impact of Negated Text on Hallucination with Large Language Models
• Investigating Counterclaims in Causality Extraction from Text
• Thunder-NUBench: A Benchmark for LLMs' Sentence-Level Negation Understanding

Related

• This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models
• Negation: A Pink Elephant in the Large Language Models' Room?
• Developmental Negation Processing in Transformer Language Models
• It's not a Non-Issue: Negation as a Source of Error in Machine Translation
• This is not correct! Negation-aware Evaluation of Language Generation Systems