Curiosity-Driven LLM-as-a-judge for Personalized Creative Judgment

2025/10/01 by Vanya Bannihatti Kumar, D. P. Goyal, Kumar, Vanya Bannihatti +5
Computer Science · Social Sciences · #Artificial Intelligence Applications #Artificial Intelligence in Law #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2510.05135

openalex publication_date 2025/10/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Modern large language models (LLMs) excel at objective tasks such as evaluating mathematical reasoning and factual accuracy, yet they falter when faced with the nuanced, subjective nature of assessing creativity. In this work, we propose a novel curiosity-driven LLM-as-a-judge for evaluating creative writing which is personlized to each individual's creative judgments. We use the Torrance Test of Creative Thinking(TTCW) benchmark introduced in Chakrabarty et al. (2024), which has stories annotated by expert humans across various subjective dimensions like Originality, to test our hypothesis. We show that our method enables models across various sizes, to learn the nuanced creative judgments of different individuals, by showing improvements over baseline supervised finetuning(SFT) method across various evaluation metrics like Pearson correlation, Cohen's and F1 values. Our method is especially useful in subjective evaluations where not all the annotators agree with each other.

Citations

• Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward
• Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?
• Self-Preference Bias in LLM-as-a-Judge
• Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both
• Robust Preference Optimization through Reward Model Distillation
• LLM Evaluators Recognize and Favor Their Own Generations
• Assessing and Understanding Creativity in Large Language Models
• Qwen Technical Report
• Art or Artifice? Large Language Models and the False Promise of Creativity
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Training Verifiers to Solve Math Word Problems
• Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations
• Exploration Based Language Learning for Text-Based Games
• Exploration by Random Network Distillation
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• Curiosity-driven Exploration by Self-supervised Prediction
• Unifying Count-Based Exploration and Intrinsic Motivation
• ImageNet Large Scale Visual Recognition Challenge
• ImageNet Large Scale Visual Recognition Challenge
• A Coefficient of Agreement for Nominal Scales
• Creativity: Yesterday, Today and Tomorrow

Related

• Steering Large Language Models to Evaluate and Amplify Creativity
• Art or Artifice? Large Language Models and the False Promise of Creativity
• CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity
• Do LLMs Agree on the Creativity Evaluation of Alternative Uses?
• Automated Creativity Evaluation for Large Language Models: A Reference-Based Approach