A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT
2023/02/21 by Jules White, Quchen Fu, White, Jules +15 · 8 voices · 114 citations
Computer Science · #Software Engineering Research #Software Engineering Techniques and Practices #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2302.11382
Abstract
Prompt engineering is an increasingly important skill set needed to converse effectively with large language models (LLMs), such as ChatGPT. Prompts are instructions given to an LLM to enforce rules, automate processes, and ensure specific qualities (and quantities) of generated output. Prompts are also a form of programming that can customize the outputs and interactions with an LLM. This paper describes a catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs. Prompt patterns are a knowledge transfer method analogous to software patterns since they provide reusable solutions to common problems faced in a particular context, i.e., output generation and interaction when working with LLMs. This paper provides the following contributions to research on prompt engineering that apply LLMs to automate software development tasks. First, it provides a framework for documenting patterns for structuring prompts to solve a range of problems so that they can be adapted to different domains. Second, it presents a catalog of patterns that have been applied successfully to improve the outputs of LLM conversations. Third, it explains how prompts can be built from multiple patterns and illustrates prompt patterns that benefit from combination with other prompt patterns.
Cited by
- Vibe Coding: An Experiment with Test-Driven Development
- Towards Automated Formal Verification of zkEVMs Using LLM-Guided Constraint Synthesis
- Evaluating Large Language Models for Symbolic Security Protocol Analysis
- A Grey Literature Review of AI-Native Applications
- Persona-as-Configuration: Generative Stakeholder Reporting for Agricultural Floods
- Anthropomorphic Behaviors of AI
- Automatically Evolving Prompt Guidelines for Task-Specific Optimization
- How Human is AI? Examining the Impact of Emotional Prompts on Artificial and Human and Responsiveness
- Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering
- ChatGPT in thematic analysis: Can AI become a research assistant in qualitative research?
- Mitigating Social Desirability Bias in Random Silicon Sampling
- Expert-Grounded Automatic Prompt Engineering for Extracting Lattice Constants of High-Entropy Alloys from Scientific Publications using Large Language Models
- Generative Artificial Intelligence for Software Engineering -- A Research Agenda
- The Impact of Prompt Programming on Function-Level Code Generation
- ContractHIL-HLS: Contract-Aligned Multi-Agent Workflow with Hardware-in-the-Loop Feedback for HLS Design
- Visual prompt engineering for video models
- How Well Can AI Generate Backlogs from App Mockups?
- AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation
- ARdena: Scenario-driven control of real-time LLM agents
- What Makes a GitHub Issue Ready for Copilot?
- Scrum Sprint Planning: LLM-based and algorithmic solutions
- CienaLLM: Generative Climate-Impact Extraction from News Articles with Autoregressive LLMs
- LLMs on Drugs: Language Models Are Few-Shot Consumers
- A systematic assessment of Large Language Models for constructing two-level fractional factorial designs
- TIB AIssistant: a Platform for AI-Supported Research Across Research Life Cycles
- SeBERTis: A Framework for Producing Classifiers of Security-Related Issue Reports
- Neurosymbolic Inference On Foundation Models For Remote Sensing Text-to-image Retrieval With Complex Queries
- Towards Interactive Intelligence for Digital Humans
- Living the Novel: A System for Generating Self-Training Timeline-Aware Conversational Agents from Novels
- Decoding the Black Box: Discerning AI Rhetorics About and Through Poetic Prompting
- Classifying German Language Proficiency Levels Using Large Language Models
- Trusted AI Agents in the Cloud
- Everything is Context: Agentic File System Abstraction for Context Engineering
- Empirical Prompt Engineering for Construct Identification with Large Language Models
- TaleFrame: An Interactive Story Generation System with Fine-Grained Control and Large Language Models
- promptolution: A Unified, Modular Framework for Prompt Optimization
- An Empirical Study of Agent Developer Practices in AI Agent Frameworks
- LAURA: Enhancing Code Review Generation with Context-Enriched Retrieval-Augmented LLM
- A Comparison of Human and ChatGPT Classification Performance on Complex Social Media Data
- LLM-Driven Transient Stability Assessment: From Automated Simulation to Neural Architecture Design
- Emotion-Aware Conversational Recommender Systems: a Case Study
- Liberating Logic in the Age of AI: Going Beyond Programming with Computational Thinking
- RAG-Driven Data Quality Governance for Enterprise ERP Systems
- Performance and interpretability analysis of code generation large language models
- Prompt-Based Value Steering of Large Language Models
- Socrates-Mol: Self-Oriented Cognitive Reasoning through Autonomous Trial-and-Error with Empirical-Bayesian Screening for Molecules
- Owlgorithm: Supporting Self-Regulated Learning in Competitive Programming through LLM-Driven Reflection
- SCRIBE: Structured Chain Reasoning for Interactive Behaviour Explanations using Tool Calling
- Large Language Model-assisted Autonomous Vehicle Recovery from Immobilization
- Beware of botshit: How to manage the epistemic risks of generative chatbots
- AI-generated stories favour stability over change: homogeneity and cultural stereotyping in narratives generated by gpt-4o-mini
- The effects of generative AI agents and scaffolding on enhancing students’ comprehension of visual learning analytics
- Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
- Collaborative LLM Agents for C4 Software Architecture Design Automation
- Code-enabled language models can outperform reasoning models on diverse tasks
- VAPU: System for Autonomous Legacy Code Modernization
- Enhancing Hotel Recommendations with AI: LLM-Based Review Summarization and Query-Driven Insights
- A Systematic Literature Review of the Use of GenAI Assistants for Code Comprehension: Implications for Computing Education Research and Practice
- Can ChatGPT Evaluate Plans?
- Show Your Title! A Scoping Review on Verbalization in Software Engineering with LLM-Assisted Screening
- A Survey on Evaluation of Large Language Models
- How Students Use Generative AI for Software Testing: An Observational Study
- Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMs
- Quantifying uncert-AI-nty: Testing the accuracy of LLMs’ confidence judgments
- Taxonomy of User Needs and Actions
- Prompting in Practice: Investigating Software Practitioners' Use of Generative AI Tools
- User Prompting Strategies and ChatGPT Contextual Adaptation Shape Conversational Information-Seeking Experiences
- Topic classification of case law using a large language model and a new taxonomy for UK law: AI insights into summary judgment
- Using Instruction-Tuned Large Language Models to Identify Indicators of Vulnerability in Police Incident Narratives
- No More Manual Guides: Automatic and Scalable Generation of High-Quality Excel Tutorials
- What makes prompts a graph: necessary and sufficient conditions for prompt graph engineering
- Using degree apprenticeships to shape the future of traditional undergraduate degrees
- Asymmetric Communication: Large Language Models and Language Games
- Security and Privacy Challenges of Large Language Models: A Survey
- Generative artificial intelligence and engineering education
- Summary of ChatGPT-Related research and perspective towards the future of large language models
- On the Soundness and Consistency of LLM Agents for Executing Test Cases Written in Natural Language
- LAD-VF: LLM-Automatic Differentiation Enables Fine-Tuning-Free Robot Planning from Formal Methods Feedback
- A Taxonomy of Prompt Defects in LLM Systems
- Generating Energy-Efficient Code via Large-Language Models -- Where are we now?
- LearnLens: An AI-Enhanced Dashboard to Support Teachers in Open-Ended Classrooms
- Vibe Check: Understanding the Effects of LLM-Based Conversational Agents' Personality and Alignment on User Perceptions in Goal-Oriented Tasks
- PromptGuard: An Orchestrated Prompting Framework for Principled Synthetic Text Generation for Vulnerable Populations using LLMs with Enhanced Safety, Fairness, and Controllability
- Calibrating MLLM-as-a-judge via Multimodal Bayesian Prompt Ensembles
- Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing
- ChatGPT for Code Refactoring: Analyzing Topics, Interaction, and Effective Prompts
- Talking to an AI Mirror: Designing Self-Clone Chatbots for Enhanced Engagement in Digital Mental Health Support
- DRF: LLM-AGENT Dynamic Reputation Filtering Framework
- SATORI: Static Test Oracle Generation for REST APIs
- LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documents
- Assessing prompting frameworks for enhancing literature reviews among university students using ChatGPT
- LLM-HyPZ: Hardware Vulnerability Discovery using an LLM-Assisted Hybrid Platform for Zero-Shot Knowledge Extraction and Refinement
- An Agile Method for Implementing Retrieval Augmented Generation Tools in Industrial SMEs
- Principled Personas: Defining and Measuring the Intended Effects of Persona Prompting on Task Performance
- Synthetic Replacements for Human Survey Data? The Perils of Large Language Models
- Towards Scalable and Interpretable Mobile App Risk Analysis via Large Language Models
- Fine-tuning and prompt engineering for large language models-based code review automation
- The Prompting Brain: Neurocognitive Markers of Expertise in Guiding Large Language Models
- Learning to Use AI for Learning: Teaching Responsible Use of AI Chatbot to K-12 Students Through an AI Literacy Module
- Towards Generalizable Human Activity Recognition: A Survey
- Uncovering Systematic Failures of LLMs in Verifying Code Against Natural Language Specifications
- Automated Building Heritage Assessment Using Street-Level Imagery
- Considering the ethics of large machine learning models in the chemical sciences
- Agentic Graph Neural Networks for Wireless Communications and Networking Towards Edge General Intelligence: A Survey
- Description and Comparative Analysis of QuRE: A New Industrial Requirements Quality Dataset
- The Cost of Thinking: Increased Jailbreak Risk in Large Language Models
- Conversational Inoculation to Enhance Resistance to Misinformation
- Pragmatics beyond humans: meaning, communication, and LLMs
- Large Language Models for Oral History Understanding with Text Classification and Sentiment Analysis
- Experimental Analysis of Productive Interaction Strategy with ChatGPT: User Study on Function and Project-level Code Generation Tasks
- From Legacy to Standard: LLM-Assisted Transformation of Cybersecurity Playbooks into CACAO Format
- Analysing nontraditional students' <scp>ChatGPT</scp> interaction, engagement, self‐efficacy and performance: A mixed‐methods approach
- Applying Psychometrics to Large Language Model Simulated Populations: Recreating the HEXACO Personality Inventory Experiment with Generative Agents
- AI-generated stories favour stability over change: homogeneity and cultural stereotyping in narratives generated by gpt-4o-mini
Discussions
- A prompt pattern catalog to enhance prompt engineering with ChatGPT [hn, 130 points, 25 comments]
- A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT [hn, 6 points, 0 comments]
- A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT [hn, 3 points, 0 comments]
- A Prompt Pattern Catalog for ChatGPT [hn, 3 points, 0 comments]
- A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT [hn, 2 points, 0 comments]
- arxiv.org/pdf/2302.11382 [bsky, 0 points, 0 comments]
- This basic course also suggested a pattern-oriented prompting approach, with a pattern catalog as a useful prompt technique reminder for me (A Prompt Pattern Catalog to Enhance Prompt Engineering with [bsky, 0 points, 1 comments]
- arxiv.org/pdf/2302.11382 [bsky, 0 points, 0 comments]
Related