Large Language Models Can Be Used to Estimate the Latent Positions of Politicians

2023/03/21 by Patrick Y. Wu, Jonathan Nagler, Wu, Patrick Y. +5 · 7 citations
Social Sciences · #Computation and Language (cs.CL) #Computational and Text Analysis Methods #Computers and Society (cs.CY) #Electoral Systems and Political Participation #FOS: Computer and information sciences #Media Influence and Politics

paper · pdf · doi:10.48550/arxiv.2303.12057

openalex publication_date 2023/03/21 · openalex created_date 2023/03/25 · openalex updated_date 2026/07/31

Abstract

Existing approaches to estimating politicians' latent positions along specific dimensions often fail when relevant data is limited. We leverage the embedded knowledge in generative large language models (LLMs) to address this challenge and measure lawmakers' positions along specific political or policy dimensions. We prompt an instruction/dialogue-tuned LLM to pairwise compare lawmakers and then scale the resulting graph using the Bradley-Terry model. We estimate novel measures of U.S. senators' positions on liberal-conservative ideology, gun control, and abortion. Our liberal-conservative scale, used to validate LLM-driven scaling, strongly correlates with existing measures and offsets interpretive gaps, suggesting LLMs synthesize relevant data from internet and digitized media rather than memorizing existing measures. Our gun control and abortion measures -- the first of their kind -- differ from the liberal-conservative scale in face-valid ways and predict interest group ratings and legislator votes better than ideology alone. Our findings suggest LLMs hold promise for solving complex social science measurement problems.

Cited by

• Scaling Open-Ended Survey Responses Using LLM-Paired Comparisons
• Reimagining Agent-based Modeling with Large Language Model Agents via Shachi
• Topic modeling of video and image data: a visual semantic unsupervised approach
• Mapping (A)Ideology: A Taxonomy of European Parties Using Generative LLMs as Zero-Shot Learners
• Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models
• Be.FM: Open Foundation Models for Human Behavior
• Large language models as a substitute for human experts in annotating political text

Related

• Measurement in the Age of LLMs: An Application to Ideological Scaling
• A Structural Text-Based Scaling Model for Analyzing Political Discourse
• Linear Representations of Political Perspective Emerge in Large Language Models
• Positioning Political Texts with Large Language Models by Asking and Averaging
• Positioning Political Texts with Large Language Models by Asking and Averaging