Language Models are Few-Shot Butlers

2021/04/16 by Micheli, Vincent, Fleuret, François · 3 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2104.07972

Abstract

Pretrained language models demonstrate strong performance in most NLP tasks when fine-tuned on small task-specific datasets. Hence, these autoregressive models constitute ideal agents to operate in text-based environments where language understanding and generative capabilities are essential. Nonetheless, collecting expert demonstrations in such environments is a time-consuming endeavour. We introduce a two-stage procedure to learn from a small set of demonstrations and further improve by interacting with an environment. We show that language models fine-tuned with only 1.2% of the expert demonstrations and a simple reinforcement learning algorithm achieve a 51% absolute improvement in success rate over existing methods in the ALFWorld environment.

Cited by

• NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied Reasoning
• MCTS-EP: Empowering Embodied Planning with Online Preference Optimization
• Generalizability of Large Language Model-Based Agents: A Comprehensive Survey

Related

• Language Models are Few-Shot Learners
• Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
• Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
• Making Pre-trained Language Models Better Few-shot Learners
• Large Language Model Programs