Do Transformer Attention Heads Provide Transparency in Abstractive Summarization?

2019/07/01 by Joris Baan, Maartje ter Hoeve, Baan, Joris +7
Computer Science · #Advanced Text Analysis Techniques #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1907.00570

openalex publication_date 2019/07/01 · openalex created_date 2022/07/28 · openalex updated_date 2026/07/28

Abstract

Learning algorithms become more powerful, often at the cost of increased complexity. In response, the demand for algorithms to be transparent is growing. In NLP tasks, attention distributions learned by attention-based deep learning models are used to gain insights in the models' behavior. To which extent is this perspective valid for all NLP tasks? We investigate whether distributions calculated by different attention heads in a transformer architecture can be used to improve transparency in the task of abstractive summarization. To this end, we present both a qualitative and quantitative analysis to investigate the behavior of the attention heads. We show that some attention heads indeed specialize towards syntactically and semantically distinct input. We propose an approach to evaluate to which extent the Transformer model relies on specifically learned attention distributions. We also discuss what this implies for using attention distributions as a means of transparency.

Citations

• Teaching Machines to Read and Comprehend
• Bidirectional recurrent neural networks
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Towards A Rigorous Science of Interpretable Machine Learning
• Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned
• RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism

Related

• Understanding Multi-Head Attention in Abstractive Summarization
• Attention Head Masking for Inference Time Content Selection in Abstractive Summarization
• Human Guided Exploitation of Interpretable Attention Patterns in Summarization and Topic Segmentation
• Mechanistic Interpretability of GPT-like Models on Summarization Tasks
• Analyzing the Structure of Attention in a Transformer Language Model