BabyAI: A Platform to Study the Sample Efficiency of Grounded Language\n Learning

2018/10/18 by Maxime Chevalier-Boisvert, Chevalier-Boisvert, Maxime, Dzmitry Bahdanau +11 · 2 voices · 69 citations
Chemistry · Computer Science · Psychology · #Artificial intelligence #Artificial neural network #Baseline (sea) #Chemistry #Computer science #Data science #Grounded theory #Heuristic #Human–computer interaction #Language acquisition #Language model #Machine learning #Mathematics education #Natural Language Processing Techniques #Natural language processing #Psychology #Qualitative research #Sample (material) #Social science #Sociology #Speech and dialogue systems #Suite #Topic Modeling #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.1810.08272

published in arXiv (Cornell University) (Cornell University) · Accepted at ICLR 2019

arxiv published 2018/10/18 · arxiv created 2019/12/19 · arxiv updated 2019/12/20

Abstract

Allowing humans to interactively train artificial agents to understand\nlanguage instructions is desirable for both practical and scientific reasons,\nbut given the poor data efficiency of the current learning methods, this goal\nmay require substantial research efforts. Here, we introduce the BabyAI\nresearch platform to support investigations towards including humans in the\nloop for grounded language learning. The BabyAI platform comprises an\nextensible suite of 19 levels of increasing difficulty. The levels gradually\nlead the agent towards acquiring a combinatorially rich synthetic language\nwhich is a proper subset of English. The platform also provides a heuristic\nexpert agent for the purpose of simulating a human teacher. We report baseline\nresults and estimate the amount of human involvement that would be required to\ntrain a neural network-based agent on some of the BabyAI levels. We put forward\nstrong evidence that current deep learning methods are not yet sufficiently\nsample efficient when it comes to learning a language with compositional\nproperties.\n

Cited by

• Learning General Policies with Policy Gradient Methods
• Observer, Not Player: Simulating Theory of Mind in LLMs through Game Observation
• Benchmarking the Generality of Vision-Language-Action Models
• An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
• Independent Density Estimation
• Fed-SE: Federated Self-Evolution for Privacy-Constrained Multi-Environment LLM Agents
• Language-conditioned world model improves policy generalization by reading environmental descriptions
• Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
• Beyond Fixed Tasks: Unsupervised Environment Design for Task-Level Pairs
• AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress
• CoFineLLM: Conformal Finetuning of LLMs for Language-Instructed Robot Planning
• Bridging the Imitation Gap by Adaptive Insubordination
• Adapting Interleaved Encoders with PPO for Language-Guided Reinforcement Learning in BabyAI
• HRM-Agent: Training a recurrent reasoning model in dynamic environments using reinforcement learning
• DAIL: Beyond Task Ambiguity for Language-Conditioned Reinforcement Learning
• Heterogeneous Adversarial Play in Interactive Environments
• One Life to Learn: Inferring Symbolic World Models for Stochastic Environments from Unguided Exploration
• FOSSIL: Harnessing Feedback on Suboptimal Samples for Data-Efficient Generalisation with Imitation Learning for Embodied Vision-and-Language Tasks
• Can RL Improve Generalization of LLM Agents? An Empirical Study
• Towards Neurocognitive-Inspired Intelligence: From AI's Structural Mimicry to Human-Like Functional Cognition
• Guiding Exploration in Reinforcement Learning Through LLM-Augmented Observations
• GRACE: A Language Model Framework for Explainable Inverse Reinforcement Learning
• BuilderBench: The Building Blocks of Intelligent Agents
• TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL
• Polychromic Objectives for Reinforcement Learning
• SocialAI 0.1: Towards a Benchmark to Stimulate Research on Socio-Cognitive Abilities in Deep Reinforcement Learning Agents
• Generalizable End-to-End Tool-Use RL with Synthetic CodeGym
• Instance based Generalization in Reinforcement Learning
• TCPO: Thought-Centric Preference Optimization for Effective Embodied Decision-making
• AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learning
• The NetHack Learning Environment
• Language Conditioned Imitation Learning over Unstructured Data
• Succeed or Learn Slowly: Sample Efficient Off-Policy Reinforcement Learning for Mobile App Control
• GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games
• CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
• MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models
• YES SIR!Optimizing Semantic Space of Negatives with Self-Involvement Ranker
• Human-In-The-Loop Document Layout Analysis
• Systematic Generalization on gSCAN: What is Nearly Solved and What is Next?
• Grounding 'Grounding' in NLP
• ELLA: Exploration through Learned Language Abstraction
• Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Data
• An Open-Source Software Toolkit & Benchmark Suite for the Evaluation and Adaptation of Multimodal Action Models
• Gearshift Fellowship: A Next-Generation Neurocomputational Game Platform to Model and Train Human-AI Adaptability
• FloorplanQA: A Benchmark for Spatial Reasoning in LLMs using Structured Representations
• OmniReflect: Discovering Transferable Constitutions for LLM agents via Neuro-Symbolic Reflections
• Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Study
• WordCraft: An Environment for Benchmarking Commonsense Agents
• Enhancing Decision-Making of Large Language Models via Actor-Critic
• Agentic Episodic Control
• ChatPD: An LLM-driven Paper-Dataset Networking System
• Large Language Models for Planning: A Comprehensive and Systematic Survey
• Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments
• Long-Horizon Embodied Decision-Making via Multimodal Memory Compression
• When Replanning Becomes the Bottleneck: Budgeted Replanning for Embodied Agents
• From Failures to Supervision: DynamicEnvPlan for Robust Long-Horizon Embodied Planning
• LLM-BABYBENCH: Understanding and Evaluating Grounded Planning and Reasoning in LLMs
• CrafText Benchmark: Advancing Instruction Following in Complex Multimodal Open-Ended World
• ImagineBench: Evaluating Reinforcement Learning with Large Language Model Rollouts
• CrystalMem: Elastic Memory for Self-Evolving LLM Agents via Knowledge Crystallization
• Unnatural Language Processing: Bridging the Gap Between Synthetic and Natural Language Data
• Working Memory Graphs
• Exploration and Exploitation Errors Are Measurable for Language Model Agents
• Verifiable Memory: Learning Unified Memory Management with Local and Global Verifiers for Large Language Model Agents
• Hybrid LLM-Augmented Reinforcement Learning Agents for Complex Sequential Decision Tasks
• When Memory Lies: An Empirical Study of Spatial Memory Staleness in VLM Agents
• Tree of Thoughts as a Classical Heuristic Search Problem: Formal Foundations and Design Patterns
• Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulness
• Executable Functional Abstractions: Inferring Generative Programs for Advanced Math Problems

Discussions

• BabyAI: Towards Grounded Language Learning with a Human in the Loop [pdf] [hn, 3 points, 0 comments]
• BabyAI: First Steps Towards Grounded Language Learning with a Human in the Loop [hn, 3 points, 1 comments]

Related

• BabyAI 1.1
• ELLA: Exploration through Learned Language Abstraction
• Towards Data-Efficient Language Models: A Child-Inspired Approach to Language Learning
• LLF-Bench: Benchmark for Interactive Learning from Language Feedback
• Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora