2023/07/13 by Hong Sun, Xue Li, Sun, Hong +13 · 10 citations
Computer Science · #Annotation #Artificial Intelligence (cs.AI) #Artificial intelligence #Computation and Language (cs.CL) #Computer science #Computer security #Context (archaeology) #FOS: Computer and information sciences #Key (lock) #Machine Learning and Data Classification #Machine learning #Natural Language Processing Techniques #Quality (philosophy) #Shot (pellet) #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2307.07415
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/07/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper presents AutoHint, a novel framework for automatic prompt engineering and optimization for Large Language Models (LLM). While LLMs have demonstrated remarkable ability in achieving high-quality annotation in various tasks, the key to applying this ability to specific tasks lies in developing high-quality prompts. Thus we propose a framework to inherit the merits of both in-context learning and zero-shot learning by incorporating enriched instructions derived from input-output demonstrations to optimize original prompt. We refer to the enrichment as the hint and propose a framework to automatically generate the hint from labeled data. More concretely, starting from an initial prompt, our method first instructs a LLM to deduce new hints for selected samples from incorrect predictions, and then summarizes from per-sample hints and adds the results back to the initial prompt to form a new, enriched instruction. The proposed method is evaluated on the BIG-Bench Instruction Induction dataset for both zero-shot and few-short prompts, where experiments demonstrate our method is able to significantly boost accuracy for multiple tasks.