PRL: Prompts from Reinforcement Learning

2025/05/20 by Paweł Batorski, Batorski, Paweł, Adrian Kosmala +3 · 4 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2505.14412

openalex publication_date 2025/05/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Effective prompt engineering remains a central challenge in fully harnessing the capabilities of LLMs. While well-designed prompts can dramatically enhance performance, crafting them typically demands expert intuition and a nuanced understanding of the task. Moreover, the most impactful prompts often hinge on subtle semantic cues, ones that may elude human perception but are crucial for guiding LLM behavior. In this paper, we introduce PRL (Prompts from Reinforcement Learning), a novel RL-based approach for automatic prompt generation. Unlike previous methods, PRL can produce novel few-shot examples that were not seen during training. Our approach achieves state-of-the-art performance across a range of benchmarks, including text classification, simplification, and summarization. On the classification task, it surpasses prior methods by 2.58% over APE and 1.00% over EvoPrompt. Additionally, it improves the average ROUGE scores on the summarization task by 4.32 over APE and by 2.12 over EvoPrompt and the SARI score on simplification by 6.93 over APE and by 6.01 over EvoPrompt. Our code is available at https://github.com/Batorskq/prl .

Citations

• No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization
• ZERA: Zero-init Instruction Evolving Refinement Agent -- From Zero Instructions to Structured Prompts via Principle-based Optimization
• Spurious Rewards: Rethinking Training Signals in RLVR
• Benchmarking Prompt Sensitivity in Large Language Models
• s1: Simple test-time scaling
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• Qwen2.5 Technical Report
• ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs
• CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generation
• POSIX: A Prompt Sensitivity Index For Large Language Models
• SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning
• Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together
• What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering
• Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs
• A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• SGLang: Efficient Execution of Structured Language Model Programs
• Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
• Graph of Thoughts: Solving Elaborate Problems with Large Language Models
• Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection
• PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
• Let's Verify Step by Step
• LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• Automatic Prompt Optimization with "Gradient Descent" and Beam Search
• Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
• Large Language Models Are Human-Level Prompt Engineers
• RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
• Large Language Models are Zero-Shot Reasoners
• OPT: Open Pre-trained Transformer Language Models
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Training Verifiers to Solve Math Word Problems
• Cross-Task Generalization via Natural Language Crowdsourcing Instructions
• Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
• The Power of Scale for Parameter-Efficient Prompt Tuning
• Calibrate Before Use: Improving Few-Shot Performance of Language Models
• Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
• Language Models are Few-Shot Learners
• ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification\n Models with Multiple Rewriting Transformations
• Character-level Convolutional Networks for Text Classification
• Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
• A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts
• EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• Prefix-Tuning: Optimizing Continuous Prompts for Generation

Cited by

• PIAST: Rapid Prompting with In-context Augmentation for Scarce Training data
• GPS: General Per-Sample Prompter
• How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation
• WST: Weak-to-Strong Knowledge Transfer via Reinforcement Learning

Related

• PRewrite: Prompt Rewriting with Reinforcement Learning
• GRL-Prompt: Towards Knowledge Graph based Prompt Optimization via Reinforcement Learning
• Prompt Curriculum Learning for Efficient LLM Post-Training
• RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
• PromptFlow: Training Prompts Like Neural Networks