LLMs Struggle with NLI for Perfect Aspect: A Cross-Linguistic Study in Chinese and Japanese

2025/08/16 by Jie Lu, Lu, Jie, Du Jin +3
Social Sciences · Arts and Humanities · Computer Science · #Artificial Intelligence in Law #Translation Studies and Practices #Library Science and Information Systems

paper · doi:10.48550/arxiv.2508.11927

Abstract

Unlike English, which uses distinct forms (e.g., had, has, will have) to mark the perfect aspect across tenses, Chinese and Japanese lack separate grammatical forms for tense within the perfect aspect, which complicates Natural Language Inference (NLI). Focusing on the perfect aspect in these languages, we construct a linguistically motivated, template-based NLI dataset (1,350 pairs per language). Experiments reveal that even advanced LLMs struggle with temporal inference, particularly in detecting subtle tense and reference-time shifts. These findings highlight model limitations and underscore the need for cross-linguistic evaluation in temporal semantics. Our dataset is available at https://github.com/Lujie2001/CrossNLI.

Citations

• Compositional Evaluation on Japanese Textual Entailment and Similarity
• OCNLI: Original Chinese Natural Language Inference
• Temporal and Aspectual Entailment
• Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference

Related

• Jamp: Controlled Japanese Temporal Inference Dataset for Evaluating Generalization Capacity of Language Models
• Acquiring Temporal Meanings Without Tense Morphology: The Case of L2 Mandarin Chinese
• Compositional Semantics and Inference System for Temporal Order based on Japanese CCG
• Probing Language Models for Understanding of Temporal Expressions
• Online processing of the grammatical aspect marker by L2 Chinese learners