Sentiment analysis is not solved! Assessing and probing sentiment\n classification

2019/06/13 by Jeremy Barnes, Barnes, Jeremy, Lilja Øvrelid +3
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Sentiment Analysis and Opinion Mining #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1906.05887

openalex publication_date 2019/06/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Neural methods for SA have led to quantitative improvements over previous\napproaches, but these advances are not always accompanied with a thorough\nanalysis of the qualitative differences. Therefore, it is not clear what\noutstanding conceptual challenges for sentiment analysis remain. In this work,\nwe attempt to discover what challenges still prove a problem for sentiment\nclassifiers for English and to provide a challenging dataset. We collect the\nsubset of sentences that an (oracle) ensemble of state-of-the-art sentiment\nclassifiers misclassify and then annotate them for 18 linguistic and\nparalinguistic phenomena, such as negation, sarcasm, modality, etc. The dataset\nis available at https://github.com/ltgoslo/assessing_and_probing_sentiment.\nFinally, we provide a case study that demonstrates the usefulness of the\ndataset to probe the performance of a given sentiment classifier with respect\nto linguistic phenomena.\n

Related

• Comprehensive Study on Sentiment Analysis: From Rule-based to modern LLM based system
• Linguistically Regularized LSTMs for Sentiment Classification
• Sentiment Analysis in the Era of Large Language Models: A Reality Check
• BESSTIE: A Benchmark for Sentiment and Sarcasm Classification for Varieties of English
• Do Large Language Models Possess Sensitive to Sentiment?