Efficient MAP Estimation of LLM Judgment Performance with Prior Transfer

2025/04/17 by Hui Qu, Qu, Huaizhi, In-Young Choi +13 · 6 citations
Computer Science · #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Imbalanced Data Classification Techniques #Machine Learning (cs.LG) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2504.12589

openalex publication_date 2025/04/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

LLM ensembles are widely used for LLM judges. However, how to estimate their accuracy, especially in an efficient way, is unknown. In this paper, we present a principled maximum a posteriori (MAP) framework for an economical and precise estimation of the performance of LLM ensemble judgment. We first propose a mixture of Beta-Binomial distributions to model the judgment distribution, revising from the vanilla Binomial distribution. Next, we introduce a conformal prediction-driven approach that enables adaptive stopping during iterative sampling to balance accuracy with efficiency. Furthermore, we design a prior transfer mechanism that utilizes learned distributions on open-source datasets to improve estimation on a target dataset when only scarce annotations are available. Finally, we present BetaConform, a framework that integrates our distribution assumption, adaptive stopping, and the prior transfer mechanism to deliver a theoretically guaranteed distribution estimation of LLM ensemble judgment with minimum labeled samples. BetaConform is also validated empirically. For instance, with only 10 samples from the TruthfulQA dataset, for a Llama ensembled judge, BetaConform gauges its performance with error margin as small as 3.37%.

Citations

• The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs
• Labeling Free-text Data using Language Model Ensembles
• The Lessons of Developing Process Reward Models in Mathematical Reasoning
• Qwen2.5 Technical Report
• VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models
• From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
• JudgeBench: A Benchmark for Evaluating LLM-based Judges
• CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?
• The Llama 3 Herd of Models
• Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
• OffsetBias: Leveraging Debiased Data for Tuning Evaluators
• "Seeing the Big through the Small": Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations?
• Evaluating the Performance of Large Language Models via Debates
• HalluDial: A Large-Scale Benchmark for Automatic Dialogue-Level Hallucination Evaluation
• Auto-Arena: Automating LLM Evaluations with Agent Peer Battles and Committee Discussions
• Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators
• InternLM2 Technical Report
• RewardBench: Evaluating Reward Models for Language Modeling
• KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models
• Language Models with Conformal Factuality Guarantees
• R-Judge: Benchmarking Safety Risk Awareness for LLM Agents
• Self-Rewarding Language Models
• A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators
• Evaluating Large Language Models at Evaluating Instruction Following
• Benchmarking Cognitive Biases in Large Language Models as Evaluators
• RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Let's Verify Step by Step
• Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
• Large Language Models are not Fair Evaluators
• FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
• ICE-Score: Instructing Large Language Models to Evaluate Code
• Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
• Language Models are Few-Shot Learners
• A tutorial on conformal prediction

Cited by

• Separate the Wheat from the Chaff: Winnowing Down Divergent Views in Retrieval Augmented Generation
• Multi-Agent Debate for LLM Judges with Adaptive Stability Detection
• AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Prediction

Related

• Auto-Prompt Ensemble for LLM Judge
• Beyond Single-Point Judgment: Distribution Alignment for LLM-as-a-Judge
• Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation
• Better Instruction-Following Through Minimum Bayes Risk
• Improving LLM-as-a-Judge Inference with the Judgment Distribution