Inverse Reinforcement Learning for Text Summarization

2022/12/19 by Fu, Yu, Xiong, Deyi, Dong, Yue
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2212.09917

Abstract

We introduce inverse reinforcement learning (IRL) as an effective paradigm for training abstractive summarization models, imitating human summarization behaviors. Our IRL model estimates the reward function using a suite of important sub-rewards for summarization and concurrently optimizes the policy network. Experimental results across datasets in different domains (CNN/DailyMail and WikiHow) and various model sizes (BART-base and BART-large) demonstrate the superiority of our proposed IRL model for summarization over MLE and RL baselines. The resulting summaries exhibit greater similarity to human-crafted gold references, outperforming MLE and RL baselines on metrics such as ROUGE, coverage, novelty, compression ratio, factuality, and human evaluations.

Related

• Better Rewards Yield Better Summaries: Learning to Summarise Without\n References
• Multi-Reward Reinforced Summarization with Saliency and Entailment
• Learning to summarize from human feedback
• Answers Unite! Unsupervised Metrics for Reinforced Summarization Models
• Ranking Sentences for Extractive Summarization with Reinforcement Learning