Demystifying Prompts in Language Models via Perplexity Estimation

2022/12/08 by Hila Gonen, Srini Iyer, Gonen, Hila +7 · 28 citations
Computer Science · #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2212.04037

openalex publication_date 2022/12/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Language models can be prompted to perform a wide variety of zero- and few-shot learning problems. However, performance varies significantly with the choice of prompt, and we do not yet understand why this happens or how to pick the best prompts. In this work, we analyze the factors that contribute to this variance and establish a new empirical hypothesis: the performance of a prompt is coupled with the extent to which the model is familiar with the language it contains. Over a wide range of tasks, we show that the lower the perplexity of the prompt is, the better the prompt is able to perform the task. As a result, we devise a method for creating prompts: (1) automatically extend a small seed set of manually written prompts by paraphrasing using GPT3 and backtranslation and (2) choose the lowest perplexity prompts to get significant gains in performance.

Cited by

• MIRAGE: Misleading Retrieval-Augmented Generation via Black-box and Query-agnostic Poisoning Attacks
• DETAIL Matters: Measuring the Impact of Prompt Specificity on Reasoning in Large Language Models
• EmoRAG: Evaluating RAG Robustness to Symbolic Perturbations
• Can LLMs Threaten Human Survival? Benchmarking Potential Existential Threats from LLMs via Prefix Completion
• SnapAudit: Active Auditing of Differentially Private In-Context Learning via Snapshot-Based Simulation
• AMaPO: Adaptive Margin-attached Preference Optimization for Language Model Alignment
• CompressionAttack: Exploiting Prompt Compression as a New Attack Surface in LLM-Powered Agents
• Foundation of Intelligence: Review of Math Word Problems from Human Cognition Perspective
• Do Prompts Reshape Representations? An Empirical Study of Prompting Effects on Embeddings
• Beyond Single Models: Mitigating Multimodal Hallucinations via Adaptive Token Ensemble Decoding
• LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblance
• ADMIT: Few-shot Knowledge Poisoning Attacks on RAG-based Fact Checking
• Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
• Prompts Generalize with Low Data: Non-vacuous Generalization Bounds for Optimizing Prompts with More Informative Priors
• Learning to Rewrite Prompts for Bootstrapping LLMs on Downstream Tasks
• Fine-Grained Emotion Recognition via In-Context Learning
• The Impact of Role Design in In-Context Learning for Large Language Models
• Protocode: Prototype-Driven Interpretability for Code Generation in LLMs
• PANICL: Mitigating Over-Reliance on Single Prompt in Visual In-Context Learning
• Predicting LLM Reasoning Performance with Small Proxy Model
• LLMZ+: Contextual Prompt Whitelist Principles for Agentic LLMs
• A method for improving multilingual quality and diversity of instruction fine-tuning datasets
• ALLabel: Three-stage Active Learning for LLM-based Entity Recognition using Demonstration Retrieval
• Online Anti-sexist Speech: Identifying Resistance to Gender Bias in Political Discourse
• Fact2Fiction: Targeted Poisoning Attack to Agentic Fact-checking System
• Language Model Guided Reinforcement Learning in Quantitative Trading
• Practical, Generalizable and Robust Backdoor Attacks on Text-to-Image Diffusion Models
• DICE: Dynamic In-Context Example Selection in LLM Agents via Efficient Knowledge Transfer

Related

• Are Prompt-based Models Clueless?
• What Makes Pre-trained Language Models Better Zero-shot Learners?
• DETAIL Matters: Measuring the Impact of Prompt Specificity on Reasoning in Large Language Models
• Prompt Injection: Parameterization of Fixed Inputs
• Do Prompt-Based Models Really Understand the Meaning of their Prompts?