Effect of Selection Format on LLM Performance

2025/03/10 by Han, Yuchen, Wu, Yucheng, Willard, Jeffrey · 2 citations
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computational Engineering #Emerging Technologies (cs.ET) #FOS: Computer and information sciences #Finance #Machine Learning (cs.LG) #and Science (cs.CE)

paper · doi:10.48550/arxiv.2503.06926

Abstract

This paper investigates a critical aspect of large language model (LLM) performance: the optimal formatting of classification task options in prompts. Through an extensive experimental study, we compared two selection formats -- bullet points and plain English -- to determine their impact on model performance. Our findings suggest that presenting options via bullet points generally yields better results, although there are some exceptions. Furthermore, our research highlights the need for continued exploration of option formatting to drive further improvements in model performance.

Cited by

• The Hidden Structure -- Improving Legal Document Understanding Through Explicit Text Formatting
• Isotropy Cliffs: The Geometric Signature of Decision-Making in Large Language Models

Related

• Does Prompt Formatting Have Any Impact on LLM Performance?
• Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
• LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs
• Reducing Selection Bias in Large Language Models
• Towards LLMs Robustness to Changes in Prompt Format Styles