L3iTC at the FinLLM Challenge Task: Quantization for Financial Text Classification & Summarization

2024/08/06 by Elvys Linhares Pontes, Pontes, Elvys Linhares, Carlos-Emiliano González-Gallardo +7 · 4 citations
Computer Science · Decision Sciences · #Advanced Text Analysis Techniques #Computation and Language (cs.CL) #Computational Engineering #FOS: Computer and information sciences #Finance #Stock Market Forecasting Methods #Text and Document Classification Technologies #and Science (cs.CE)

paper · pdf · doi:10.48550/arxiv.2408.03033

openalex publication_date 2024/08/06 · openalex created_date 2024/10/16 · openalex updated_date 2026/07/28

Abstract

This article details our participation (L3iTC) in the FinLLM Challenge Task 2024, focusing on two key areas: Task 1, financial text classification, and Task 2, financial text summarization. To address these challenges, we fine-tuned several large language models (LLMs) to optimize performance for each task. Specifically, we used 4-bit quantization and LoRA to determine which layers of the LLMs should be trained at a lower precision. This approach not only accelerated the fine-tuning process on the training data provided by the organizers but also enabled us to run the models on low GPU memory. Our fine-tuned models achieved third place for the financial classification task with an F1-score of 0.7543 and secured sixth place in the financial summarization task on the official test datasets.

Cited by

• Financial News Summarization: Can extractive methods still offer a true alternative to LLMs?
• A symbolic Perl algorithm for the unification of Nahuatl word spellings
• LLM Bias Detection and Mitigation through the Lens of Desired Distributions
• HARLF: Hierarchical Reinforcement Learning and Lightweight LLM-Driven Sentiment Integration for Financial Portfolio Optimization

Related

• FinLoRA: Finetuning Quantized Financial Large Language Models Using Low-Rank Adaptation
• 'Finance Wizard' at the FinLLM Challenge Task: Financial Text Summarization
• CatMemo at the FinLLM Challenge Task: Fine-Tuning Large Language Models using Data Fusion in Financial Applications
• FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets
• FinLoRA: Benchmarking LoRA Methods for Fine-Tuning LLMs on Financial Datasets