2020/10/16 by Tim Nugent, Nicole Stelea, Nugent, Tim +4
Computer Science · Decision Sciences · Social Sciences · #Computation and Language (cs.CL) #Computational and Text Analysis Methods #FOS: Computer and information sciences #I.2.7 #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Stock Market Forecasting Methods #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2010.08319
openalex publication_date 2020/10/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Despite recent advances in deep learning-based language modelling, many natural language processing (NLP) tasks in the financial domain remain challenging due to the paucity of appropriately labelled data. Other issues that can limit task performance are differences in word distribution between the general corpora - typically used to pre-train language models - and financial corpora, which often exhibit specialized language and symbology. Here, we investigate two approaches that may help to mitigate these issues. Firstly, we experiment with further language model pre-training using large amounts of in-domain data from business and financial news. We then apply augmentation approaches to increase the size of our dataset for model fine-tuning. We report our findings on an Environmental, Social and Governance (ESG) controversies dataset and demonstrate that both approaches are beneficial to accuracy in classification tasks.