A Simple Ensemble Strategy for LLM Inference: Towards More Stable Text Classification

2025/04/26 by Junichiro Niimi, Niimi, Junichiro · 4 citations
Computer Science · #Artificial Intelligence (cs.AI) #Authorship Attribution and Profiling #Computation and Language (cs.CL) #FOS: Computer and information sciences #Sentiment Analysis and Opinion Mining #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2504.18884

openalex publication_date 2025/04/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

With the advance of large language models (LLMs), LLMs have been utilized for the various tasks. However, the issues of variability and reproducibility of results from each trial of LLMs have been largely overlooked in existing literature while actual human annotation uses majority voting to resolve disagreements among annotators. Therefore, this study introduces the straightforward ensemble strategy to a sentiment analysis using LLMs. As the results, we demonstrate that the ensemble of multiple inference using medium-sized LLMs produces more robust and accurate results than using a large model with a single attempt with reducing RMSE by 18.6%.

Cited by

• Impact of Label Noise from Large Language Model–generated Annotations on Evaluation of Diagnostic Model Performance
• Stable LLM Ensemble: Interaction between Example Representativeness and Diversity
• Harnessing Consistency for Robust Test-Time LLM Ensemble
• Reference Points in LLM Sentiment Analysis: The Role of Structured Context

Related

• Harnessing Multiple Large Language Models: A Survey on LLM Ensemble
• Majority Rules: LLM Ensemble is a Winning Approach for Content Categorization
• Examining Independence in Ensemble Sentiment Analysis: A Study on the Limits of Large Language Models Using the Condorcet Jury Theorem
• Can Unconfident LLM Annotations Be Used for Confident Conclusions?
• Not All Layers of LLMs Are Necessary During Inference