The Generative AI Paradox on Evaluation: What It Can Solve, It May Not Evaluate

2024/02/09 by Juhyun Oh, Eun‐Su Kim, Oh, Juhyun +5 · 2 citations
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #Cognitive Science and Mapping #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Impact of AI and Big Data on Business and Society

paper · pdf · doi:10.48550/arxiv.2402.06204

openalex publication_date 2024/02/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper explores the assumption that Large Language Models (LLMs) skilled in generation tasks are equally adept as evaluators. We assess the performance of three LLMs and one open-source LM in Question-Answering (QA) and evaluation tasks using the TriviaQA (Joshi et al., 2017) dataset. Results indicate a significant disparity, with LLMs exhibiting lower performance in evaluation tasks compared to generation tasks. Intriguingly, we discover instances of unfaithful evaluation where models accurately evaluate answers in areas where they lack competence, underscoring the need to examine the faithfulness and trustworthiness of LLMs as evaluators. This study contributes to the understanding of "the Generative AI Paradox" (West et al., 2023), highlighting a need to explore the correlation between generative excellence and evaluation proficiency, and the necessity to scrutinize the faithfulness aspect in model evaluations.

Cited by

• A Principle-based Framework for the Development and Evaluation of Large Language Models for Health and Wellness
• Benchmark Profiling: Mechanistic Diagnosis of LLM Benchmarks

Related

• Large Language Models are not Fair Evaluators
• Style Over Substance: Evaluation Biases for Large Language Models
• Language Models can Evaluate Themselves via Probability Discrepancy
• Evaluating Open-QA Evaluation
• Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Tasks