Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models

2023/10/11 by Sun, Yuchong, Liu, Che, Zhou, Kun +6 · 4 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2310.07301

Abstract

Humans often interact with large language models (LLMs) in multi-turn interaction to obtain desired answers or more information. However, most existing studies overlook the multi-turn instruction following ability of LLMs, in terms of training dataset, training method, and evaluation benchmark. In this paper, we introduce Parrot, a solution aiming to enhance multi-turn instruction following for LLMs. First, we introduce an efficient but effective method for collecting multi-turn instructions that feature human-like queries, such as anaphora and ellipsis. Second, we propose a context-aware preference optimization strategy to further enhance LLMs for complex queries in multi-turn interaction. Moreover, to quantitatively evaluate LLMs in multi-turn instruction following, we manually build a multi-turn benchmark derived from existing ones. Extensive experiments show that Parrot improves current LLMs by up to 7.2% in multi-turn instruction following. Our dataset and codes will be open-sourced to facilitate future research.

Cited by

• RefineBench: Evaluating Refinement Capability of Language Models via Checklists
• DynaKV: Enabling Accurate and Efficient Long-Sequence LLM Decoding on Smartphones
• Test-Time Policy Adaptation for Enhanced Multi-Turn Interactions with LLMs
• KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues

Related

• ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback
• Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models
• MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models
• Helping Language Models Learn More: Multi-dimensional Task Prompt for Few-shot Tuning
• Fine-tuning Large Language Models with Sequential Instructions