Prospects for inconsistency detection using large language models and sheaves

2024/01/30 by Steve Huntsman, Michael Robinson, Huntsman, Steve +3 · 4 citations
Computer Science · Decision Sciences · #Algebraic Topology (math.AT) #Computation and Language (cs.CL) #Computers and Society (cs.CY) #Data Quality and Management #FOS: Computer and information sciences #FOS: Mathematics #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2401.16713

openalex publication_date 2024/01/30 · openalex created_date 2024/02/01 · openalex updated_date 2026/07/28

Abstract

We demonstrate that large language models can produce reasonable numerical ratings of the logical consistency of claims. We also outline a mathematical approach based on sheaf theory for lifting such ratings to hypertexts such as laws, jurisprudence, and social media and evaluating their consistency globally. This approach is a promising avenue to increasing consistency in and of government, as well as to combating mis- and disinformation and related ills.

Cited by

• On Finding Inconsistencies in Documents
• LLM-Generated Counterfactual Stress Scenarios for Portfolio Risk Simulation via Hybrid Prompt-RAG Pipeline
• Automatic coherence-driven inference on arguments
• Coherence-driven inference for cybersecurity

Related

• A Sheaf Model of Contradictions and Disagreements. Preliminary Report and Discussion
• Automatic coherence-driven inference on arguments
• ECon: On the Detection and Resolution of Evidence Conflicts
• Misleading through Inconsistency: A Benchmark for Political Inconsistencies Detection
• On Finding Inconsistencies in Documents