Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Models

2024/08/21 by Sepehr Kamahi, Kamahi, Sepehr, Yadollah Yaghoobzadeh +1 · 2 citations
Computer Science · Decision Sciences · #Topic Modeling #activated carbon and charcoal #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2408.11252

Abstract

Despite the widespread adoption of autoregressive language models, explainability evaluation research has predominantly focused on span infilling and masked language models. Evaluating the faithfulness of an explanation method -- how accurately it explains the inner workings and decision-making of the model -- is challenging because it is difficult to separate the model from its explanation. Most faithfulness evaluation techniques corrupt or remove input tokens deemed important by a particular attribution (feature importance) method and observe the resulting change in the model's output. However, for autoregressive language models, this approach creates out-of-distribution inputs due to their next-token prediction training objective. In this study, we propose a technique that leverages counterfactual generation to evaluate the faithfulness of attribution methods for autoregressive language models. Our technique generates fluent, in-distribution counterfactuals, making the evaluation protocol more reliable.

Cited by

• Explaining the Reasoning of Large Language Models Using Attribution Graphs
• Learning from Sufficient Rationales: Analysing the Relationship Between Explanation Faithfulness and Token-level Regularisation Strategies

Related

• Counterfactual Evaluation for Explainable AI
• Incorporating Attribution Importance for Improving Faithfulness Metrics
• The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models
• Connecting Attributions and QA Model Behavior on Realistic Counterfactuals
• Noiser: Bounded Input Perturbations for Attributing Large Language Models