MillStone: How Open-Minded Are LLMs?

2025/09/15 by Harold Triedman, Vitaly Shmatikov, Triedman, Harold +1
Computer Science · #Benchmark (surveying) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Library Science and Information Systems #Machine Learning (cs.LG) #Measure (data warehouse) #Risk assessment #Selection (genetic algorithm) #Semantic Web and Ontologies #The Internet

paper · pdf · doi:10.48550/arxiv.2509.11967

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/09/15 · openalex created_date 2025/10/18 · openalex updated_date 2026/08/05

Abstract

Large language models equipped with Web search, information retrieval tools, and other agentic capabilities are beginning to supplant traditional search engines. As users start to rely on LLMs for information on many topics, including controversial and debatable issues, it is important to understand how the stances and opinions expressed in LLM outputs are influenced by the documents they use as their information sources. In this paper, we present MillStone, the first benchmark that aims to systematically measure the effect of external arguments on the stances that LLMs take on controversial issues (not all of them political). We apply MillStone to nine leading LLMs and measure how ``open-minded'' they are to arguments supporting opposite sides of these issues, whether different LLMs agree with each other, which arguments LLMs find most persuasive, and whether these arguments are the same for different LLMs. In general, we find that LLMs are open-minded on most issues. An authoritative source of information can easily sway an LLM's stance, highlighting the importance of source selection and the risk that LLM-based information retrieval and search systems can be manipulated.

Citations

• Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attack
• Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs
• IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistance
• Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models
• FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Models
• LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals
• ConflictBank: A Benchmark for Evaluating the Influence of Knowledge Conflicts in LLM
• Are Large Language Models Consistent over Value-laden Questions?
• Phantom: General Backdoor Attacks on Retrieval Augmented Language Generation
• Knowledge Conflicts for LLMs: A Survey
• Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models
• What Evidence Do Language Models Find Convincing?
• GEO: Generative Engine Optimization
• Jailbreaking Black Box Large Language Models in Twenty Queries
• More human than human: measuring ChatGPT political bias
• From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models
• Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
• Out of One, Many: Using Language Models to Simulate Human Samples
• Fact-Saboteurs: A Taxonomy of Evidence Manipulation Attacks against Fact-Verification Systems
• Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies
• Attacking Open-domain Question Answering by Injecting Misinformation
• Aligning AI With Shared Human Values
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings

Related

• In Agents We Trust, but Who Do Agents Trust? Latent Source Preferences Steer LLM Generations
• The Political Preferences of LLMs
• The Chameleon Nature of LLMs: Quantifying Multi-Turn Stance Instability in Search-Enabled Language Models
• Whose Facts Win? LLM Source Preferences under Knowledge Conflicts
• Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seeking