Is Temperature the Creativity Parameter of Large Language Models?

2024/05/01 by Max Peeperkorn, Tom Kouwenhoven, Peeperkorn, Max +5 · 1 voice · 48 citations
Computer Science · Psychology · #Creativity #Linguistics #Philosophy #Psychology #Social psychology #Topic Modeling #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2405.00492

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2024/05/01 · arxiv published 2024/05/01 · arxiv updated 2024/05/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Large language models (LLMs) are applied to all sorts of creative tasks, and their outputs vary from beautiful, to peculiar, to pastiche, into plain plagiarism. The temperature parameter of an LLM regulates the amount of randomness, leading to more diverse outputs; therefore, it is often claimed to be the creativity parameter. Here, we investigate this claim using a narrative generation task with a predetermined fixed context, model and prompt. Specifically, we present an empirical analysis of the LLM output for different temperature values using four necessary conditions for creativity in narrative generation: novelty, typicality, cohesion, and coherence. We find that temperature is weakly correlated with novelty, and unsurprisingly, moderately correlated with incoherence, but there is no relationship with either cohesion or typicality. However, the influence of temperature on creativity is far more nuanced and weak than suggested by the "creativity parameter" claim; overall results suggest that the LLM generates slightly more novel outputs as temperatures get higher. Finally, we discuss ideas to allow more controlled LLM creativity, rather than relying on chance via changing the temperature parameter.

Cited by

• Can LLMs Cook Jamaican Couscous? A Study of Cultural Novelty in Recipe Generation
• Exploiting the Randomness of Large Language Models (LLM) in Text Classification Tasks: Locating Privileged Documents in Legal Matters
• General Intelligence-based Fragmentation (GIF): A framework for peak-labeled spectra simulation
• On the Creativity of AI Agents
• Optimizing Diversity and Quality through Base-Aligned Model Collaboration
• Political censorship in large language models originating from China
• Mapping and Comparing Climate Equity Policy Practices Using RAG LLM-Based Semantic Analysis and Recommendation Systems
• QuanBench: Benchmarking Quantum Code Generation with Large Language Models
• Automated Extraction of Protocol State Machines from 3GPP Specifications with Domain-Informed Prompts and LLM Ensembles
• An Evaluation of the Performance of OpenAI-o1 and GPT-4o in the Japanese National Examination for Physical Therapists
• From Editorial Records to Structured Provenance Information: Documenting Warrant in Knowledge Organization Systems Using Large Language Models
• Adding LLMs to the psycholinguistic norming toolbox: A practical guide to getting the most out of human ratings
• Jointly Reinforcing Diversity and Quality in Language Model Generations
• An Agile Method for Implementing Retrieval Augmented Generation Tools in Industrial SMEs
• Cognitive Agents Powered by Large Language Models for Agile Software Project Management
• Improving Diversity in Language Models: When Temperature Fails, Change the Loss
• From Legacy to Standard: LLM-Assisted Transformation of Cybersecurity Playbooks into CACAO Format
• Evaluation of GPT-based large language generative AI models as study aids for the national licensure examination for registered dietitians in Japan
• Introducing HALC: A general pipeline for finding optimal prompting strategies for automated coding with LLMs in the computational social sciences
• Geometry of Knowledge Allows Extending Diversity Boundaries of Large Language Models
• Combinatorial Creativity: A New Frontier in Generalization Abilities
• Quantifying Narrative Similarity Across Languages
• An Evaluation of Large Language Models on Text Summarization Tasks Using Prompt Engineering Techniques
• Agent-Based Detection and Resolution of Incompleteness and Ambiguity in Interactions with Large Language Models
• A Survey of LLM Inference Systems
• Large Language Models Acing Chartered Accountancy
• When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs
• Exploring the Impact of Temperature on Large Language Models:Hot or Cold?
• Probability Bounding: Post-Hoc Calibration via Box-Constrained Softmax
• Defining and Quantifying Creative Behavior in Popular Image Generators
• MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinations
• GIFARC: Synthetic Dataset for Leveraging Human-Intuitive Analogies to Elevate AI Reasoning
• EarthSE: A Benchmark for Evaluating Earth Scientific Exploration Capability of LLMs
• The production of meaning in the processing of natural language
• AGC-Bench: Measuring Artificial General Creativity
• Inducing Sustained Creativity and Diversity in Large Language Models
• NoveltyBench: Evaluating Language Models for Humanlike Diversity
• CAPTURe: Evaluating Spatial Reasoning in Vision Language Models via Occluded Object Counting
• Has the Creativity of Large-Language Models peaked? An analysis of inter- and intra-LLM variability
• Automated Business Process Analysis: An LLM-Based Approach to Value Assessment

Discussions

• There’s an interesting explanation of why AIs don’t produce novel art styles. It is said that unpredictable outputs that aren’t incoherent are hard to identify for a system designed to create coherent [bsky, 1 points, 1 comments]

Related

• Steering Large Language Models to Evaluate and Amplify Creativity
• The Paradox of Stochasticity: Limited Creativity and Computational Decoupling in Temperature-Varied LLM Outputs of Structured Fictional Data
• Evaluating Creative Short Story Generation in Humans and Large Language Models
• Evaluating Large Language Model Creativity from a Literary Perspective
• Style Over Story: A Process-Oriented Study of Authorial Creativity in Large Language Models