Applying a Pre-trained Language Model to Spanish Twitter Humor Prediction

2019/07/06 by Farzin, Bobak, Czapla, Piotr, Howard, Jeremy
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.1907.03187

Abstract

Our entry into the HAHA 2019 Challenge placed 3^rd in the classification task and 2^nd in the regression task. We describe our system and innovations, as well as comparing our results to a Naive Bayes baseline. A large Twitter based corpus allowed us to train a language model from scratch focused on Spanish and transfer that knowledge to our competition model. To overcome the inherent errors in some labels we reduce our class confidence with label smoothing in the loss function. All the code for our project is included in a GitHub repository for easy reference and to enable replication by others.

Related

• RoBERTuito: a pre-trained language model for social media text in\n Spanish
• A Crowd-Annotated Spanish Corpus for Humor Analysis
• BERTuit: Understanding Spanish language in Twitter through a native transformer
• UG18 at SemEval-2018 Task 1: Generating Additional Training Data for Predicting Emotion Intensity in Spanish
• TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twitter