Utilizing BERT Intermediate Layers for Aspect Based Sentiment Analysis and Natural Language Inference

2020/02/12 by Youwei Song, Jiahai Wang, Song, Youwei +7
Computer Science · #Advanced Text Analysis Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Sentiment Analysis and Opinion Mining #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2002.04815

openalex publication_date 2020/02/12 · openalex created_date 2020/02/24 · openalex updated_date 2026/07/28

Abstract

Aspect based sentiment analysis aims to identify the sentimental tendency towards a given aspect in text. Fine-tuning of pretrained BERT performs excellent on this task and achieves state-of-the-art performances. Existing BERT-based works only utilize the last output layer of BERT and ignore the semantic knowledge in the intermediate layers. This paper explores the potential of utilizing BERT intermediate layers to enhance the performance of fine-tuning of BERT. To the best of our knowledge, no existing work has been done on this research. To show the generality, we also apply this approach to a natural language inference task. Experimental results demonstrate the effectiveness and generality of the proposed approach.

Citations

• Adam: A Method for Stochastic Optimization
• Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
• Multi-Task Deep Neural Networks for Natural Language Understanding
• BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis

Related

• Does BERT Understand Sentiment? Leveraging Comparisons Between Contextual and Non-Contextual Embeddings to Improve Aspect-Based Sentiment Models
• Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence
• Exploiting BERT for End-to-End Aspect-based Sentiment Analysis
• BERT-ASC: Auxiliary-Sentence Construction for Implicit Aspect Learning in Sentiment Analysis
• Understanding Pre-trained BERT for Aspect-based Sentiment Analysis