LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings
2025/10/09 by Benjamin F. Maier, Ulf Aslak, Maier, Benjamin F. +15 · 8 voices
Computer Science · Social Sciences · Decision Sciences · #Sentiment Analysis and Opinion Mining #Computational and Text Analysis Methods #Forecasting Techniques and Applications
paper · pdf · doi:10.48550/arxiv.2510.08338
Abstract
Consumer research costs companies billions annually yet suffers from panel biases and limited scale. Large language models (LLMs) offer an alternative by simulating synthetic consumers, but produce unrealistic response distributions when asked directly for numerical ratings. We present semantic similarity rating (SSR), a method that elicits textual responses from LLMs and maps these to Likert distributions using embedding similarity to reference statements. Testing on an extensive dataset comprising 57 personal care product surveys conducted by a leading corporation in that market (9,300 human responses), SSR achieves 90% of human test-retest reliability while maintaining realistic response distributions (KS similarity > 0.85). Additionally, these synthetic respondents provide rich qualitative feedback explaining their ratings. This framework enables scalable consumer research simulations while preserving traditional survey metrics and interpretability.
Cited by
Discussions
- A great example of doing shit because it is easy, rather than because it is effective. Rather than concluding "stop doing focus groups because the data is useless" AI dorks decided that we should simp [bsky, 17 points, 3 comments]
- Paper: arxiv.org/abs/2510.08338 [bsky, 9 points, 0 comments]
- LLMs Reproduce Human Purchase Intent via Semantic Similarity of Likert Ratings [hn, 4 points, 0 comments]
- Billionaires and their daft LLMs have really broken…everything… Likely irreparably… "LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings" https://arxiv.org/abs/2 [bsky, 2 points, 0 comments]
- LLMs Reproduce Human Purchase Intent via Semantic Similarity [hn, 2 points, 1 comments]
- If you're in consumer marketing, your job just became much more threatened by AI. I'd suggest you read this thread and paper.
Thread: threadreaderapp.com/thread/20110...
Paper: arxiv.org/abs/2510.083 [bsky, 0 points, 0 comments]
- Using LLMs to predict actual purchase intent without talking to actual customers: arxiv.org/pdf/2510.08338 [bsky, 0 points, 0 comments]
- It’s a pretty cool look at how we might run thousands of 'interviews' in seconds, without losing the human nuance and still getting reliable numerical outputs to work with. Here's the paper: arxiv.org [bsky, 0 points, 0 comments]
Related