Large Language Models Reflect the Ideology of their Creators

2024/10/24 by Maarten Buyl, Alexander Rogiers, Buyl, Maarten +20 · 14 voices · 23 citations
Social Sciences · #Computational and Text Analysis Methods

paper · pdf · doi:10.48550/arxiv.2410.18417

Abstract

Large language models (LLMs) are trained on vast amounts of data to generate natural language, enabling them to perform tasks like text summarization and question answering. These models have become popular in artificial intelligence (AI) assistants like ChatGPT and already play an influential role in how humans access information. However, the behavior of LLMs varies depending on their design, training, and use. In this paper, we prompt a diverse panel of popular LLMs to describe a large number of prominent personalities with political relevance, in all six official languages of the United Nations. By identifying and analyzing moral assessments reflected in their responses, we find normative differences between LLMs from different geopolitical regions, as well as between the responses of the same LLM when prompted in different languages. Among only models in the United States, we find that popularly hypothesized disparities in political views are reflected in significant normative differences related to progressive values. Among Chinese models, we characterize a division between internationally- and domestically-focused models. Our results show that the ideological stance of an LLM appears to reflect the worldview of its creators. This poses the risk of political instrumentalization and raises concerns around technological and regulatory efforts with the stated aim of making LLMs ideologically 'unbiased'.

Cited by

• AI-Powered Browsers Are Broadly Accurate News Summarizers That Reduce Political Bias and Negative Affect
• The Ethics of Autonomous AI Agents for Offensive Security
• REGARD: Regional Affective Differences in Large Language Models
• AuAu: A Benchmark for Auditing Authoritarian Alignment in Large Language Models
• Grokipedia vs Wikipedia: An LLM-Based Audit of Political Neutrality along Ideologies
• AI-Mediated Communication Can Steer Collective Opinion
• Ideological bias in the production of research findings
• Information suppression in large language models: Auditing, quantifying, and characterizing censorship in DeepSeek
• The Homogenizing Effect of Large Language Models on Human Expression and Thought
• Estimating the Geopolitical Preferences of Large Language Models from United Nations Voting Data
• Prompt Governance? On Governing Technologies Governed by Natural Language
• Position: Universal Aesthetic Alignment Narrows Artistic Expression
• Multilingual Large Language Models do not comprehend all natural languages to equal degrees
• Politically Speaking: LLMs on Changing International Affairs
• Deliberative Dynamics and Value Alignment in LLM Debates
• Reward Model Perspectives: Whose Opinions Do Reward Models Reward?
• SocialHarmBench: Revealing LLM Vulnerabilities to Socially Harmful Requests
• Exploring Chain-of-Thought Reasoning for Steerable Pluralistic Alignment
• Assessing political bias in AI systems: a framework for disentangling viewpoint preferences from epistemic integrity
• POW: Political Overton Windows of Large Language Models
• Building and Measuring Trust between Large Language Models
• Understanding and Mitigating Political Stance Cross-topic Generalization in Large Language Models
• Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs

Discussions

• Artigo mostrando que LLMs carregam vieses do criador delas. Eu falei isso quando ChatGPT surgiu em 2022. Sabe como sabia disso? Pq só crianças acreditam que computadores ou ciência não tem vieses. [bsky, 127 points, 3 comments]
• 🤖 "Large Language Models Reflect the Ideology of their Creators" "We find normative differences between LLMs from different geopolitical regions, as well as between the responses of the same LLM when [bsky, 11 points, 0 comments]
• Really shocking numbers! Was reading this earlier which just makes an (albeit by now) obvious point even more horrifying arxiv.org/abs/2410.18417 [bsky, 9 points, 0 comments]
• the choice of LLM is not value neutral. Different LLMs reflect the ideology of its creator. Not so surprising across countries (Chinese vs French LLM, for example); more surprising for different LLMs [bsky, 7 points, 1 comments]
• Look, there was research about AI vibes last year... or let's call it the default political stance of large language models > Our results show that the ideological stance of an LLM appears to reflect [bsky, 6 points, 1 comments]
• Large Language Models Reflect the Ideology of Their Creators [hn, 4 points, 2 comments]
• Not sure about the app, but when running the model locally, it happily told me all about Tiananmen Square :D. Read a really nice paper on this last year: arxiv.org/abs/2410.18417 [bsky, 2 points, 1 comments]
• "Sins of the Father…" @arxiv #llm #bias arxiv.org/abs/2410.18417 - "This raises important concerns around technological and regulatory efforts with the stated aim of making LLMs ideologically `unbias [bsky, 1 points, 0 comments]
• 💡What an interesting new study on the ideologies of AI language models, which are made in the image and likeness of their creators 👇 arxiv.org/abs/2410.18417 [bsky, 1 points, 1 comments]
• Large Language Models Reflect the Ideology of their Creators | arXiv arxiv.org/abs/2410.18417 #openaccess [bsky, 0 points, 0 comments]
• arxiv.org/pdf/2410.18417 [bsky, 0 points, 0 comments]
• Large Language Models Reflect the Ideology of their Creators. This raises important concerns around technological and regulatory efforts with the stated aim of making LLMs ideologically `unbiased', a [bsky, 0 points, 0 comments]
• ¿Tiene ideología ChatGpt? arxiv.org/abs/2410.18417 [bsky, 0 points, 0 comments]
• El documento que analiza el sesgo ideológico de las AI: ChatGPT (OpenAI) vs Gemini (Google) vía @elpais.com arxiv.org/pdf/2410.18417 [bsky, 0 points, 0 comments]

Related

• Echoes of Power: Investigating Geopolitical Bias in US and China Large Language Models
• Decoding the Mind of Large Language Models: A Quantitative Evaluation of Ideology and Biases
• Red AI? Inconsistent Responses from GPT3.5 Models on Political Issues in the US and China
• Large Language Models
• Language-Dependent Political Bias in AI: A Study of ChatGPT and Gemini