Measuring Association Between Labels and Free-Text Rationales

2020/10/24 by Sarah Wiegreffe, Ana Marasović, Wiegreffe, Sarah +3 · 23 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #cs.CL

paper · pdf · doi:10.48550/arxiv.2010.12762

Revision to EMNLP 2021 camera-ready; corrects simulatability terminology and clarifies computation of rationale quality metric (no results changed). For a detailed explanation of changes, see https://github.com/allenai/label_rationale_association

arxiv created 2022/08/29 · arxiv updated 2022/08/31

Abstract

In interpretable NLP, we require faithful rationales that reflect the model's decision-making process for an explained instance. While prior work focuses on extractive rationales (a subset of the input words), we investigate their less-studied counterpart: free-text natural language rationales. We demonstrate that pipelines, existing models for faithful extractive rationalization on information-extraction style tasks, do not extend as reliably to "reasoning" tasks requiring free-text rationales. We turn to models that jointly predict and rationalize, a class of widely used high-performance models for free-text rationalization whose faithfulness is not yet established. We define label-rationale association as a necessary property for faithfulness: the internal mechanisms of the model producing the label and the rationale must be meaningfully correlated. We propose two measurements to test this property: robustness equivalence and feature importance agreement. We find that state-of-the-art T5-based joint models exhibit both properties for rationalizing commonsense question-answering and natural language inference, indicating their potential for producing faithful free-text rationales.

Cited by

• Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalization
• The Scaling Properties of Implicit Deductive Reasoning in Transformers
• Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
• Everything is Plausible: Investigating the Impact of LLM Rationales on Human Notions of Plausibility
• Self-Filtered Distillation with LLMs-generated Trust Indicators for Reliable Patent Classification
• Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations
• Pun Unintended: LLMs and the Illusion of Humor Understanding
• LLMs Don't Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations
• Let's Roleplay: Examining LLM Alignment in Collaborative Dialogues
• Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Models
• NeuroFaith: Evaluating LLM Self-Explanation Faithfulness via Internal Representation Alignment
• Step-Level Visual Grounding Faithfulness Predicts Out-of-Distribution Generalization in Long-Horizon Vision-Language Models
• Towards Large Language Models with Self-Consistent Natural Language Explanations
• Argument-Based Consistency in Toxicity Explanations of LLMs
• Iterative Self-Improvement of Vision Language Models for Image Scoring and Self-Explanation
• A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models
• Self-Critique and Refinement for Faithful Natural Language Explanations
• Frictional Agent Alignment Framework: Slow Down and Don't Break Things
• Explaining Sources of Uncertainty in Automated Fact-Checking
• Truth or Twist? Optimal Model Selection for Reliable Label Flipping Evaluation in LLM-based Counterfactuals
• Improving Language Model Personas via Rationalization with Psychological Scaffolds
• Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability
• AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computation

Related

• RORA: Robust Free-Text Rationale Evaluation
• REV: Information-Theoretic Evaluation of Free-Text Rationales
• FRAME: Evaluating Rationale-Label Consistency Metrics for Free-Text Rationales
• Flexible Instance-Specific Rationalization of NLP Models
• The Irrationality of Neural Rationale Models