A Logic-Driven Framework for Consistency of Neural Models

2019/08/31 by Tao Li, Li, Tao, Vivek Gupta +5 · 8 citations
Computer Science · Mathematics · #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Bayesian Modeling and Causal Inference #Computation and Language (cs.CL) #Computer science #Consistency (knowledge bases) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Generalization #Generalization error #Inference #Leverage (statistics) #Machine Learning (cs.LG) #Machine learning #Mathematics #Rule of inference #Topic Modeling #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.1909.00126

published in arXiv (Cornell University) (Cornell University) · Accepted in EMNLP 2019; Extra footnote after camera ready; Addressing R-fuzzy and S-fuzzy logic + extra acknowledgement

openalex publication_date 2019/08/31 · arxiv created 2019/09/13 · arxiv updated 2019/09/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

While neural models show remarkable accuracy on individual predictions, their internal beliefs can be inconsistent across examples. In this paper, we formalize such inconsistency as a generalization of prediction error. We propose a learning framework for constraining models using logic rules to regularize them away from inconsistency. Our framework can leverage both labeled and unlabeled examples and is directly compatible with off-the-shelf learning schemes without model redesign. We instantiate our framework on natural language inference, where experiments show that enforcing invariants stated in logic can help make the predictions of neural models both accurate and consistent.

Citations

• Bidirectional Attention Flow for Machine Comprehension

Cited by

• Temporal Referential Consistency: Do LLMs Favor Sequences Over Absolute Time References?
• Improving Consistency in Retrieval-Augmented Systems with Group Similarity Rewards
• Database Views as Explanations for Relational Deep Learning
• Constraints-Guided Diffusion Reasoner for Neuro-Symbolic Learning
• The Need for Verification in AI-Driven Scientific Discovery
• Found in Translation: Measuring Multilingual LLM Consistency as Simple as Translate then Evaluate
• LogiCoL: Logically-Informed Contrastive Learning for Set-based Dense Retrieval
• Consistency in Language Models: Current Landscape, Challenges, and Future Directions

Related

• Training Machine Learning Models by Regularizing their Explanations
• Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language?
• Logic of Machine Learning
• Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models
• Logically Consistent Language Models via Neuro-Symbolic Integration