MaestroMotif: Skill Design from Artificial Intelligence Feedback

2024/12/11 by Martin Klissarov, Mikael Henaff, Klissarov, Martin +21 · 1 voice · 8 citations
Business, Management and Accounting · #AI and HR Technologies #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.2412.08542

openalex publication_date 2024/12/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Describing skills in natural language has the potential to provide an accessible way to inject human knowledge about decision-making into an AI system. We present MaestroMotif, a method for AI-assisted skill design, which yields high-performing and adaptable agents. MaestroMotif leverages the capabilities of Large Language Models (LLMs) to effectively create and reuse skills. It first uses an LLM's feedback to automatically design rewards corresponding to each skill, starting from their natural language description. Then, it employs an LLM's code generation abilities, together with reinforcement learning, for training the skills and combining them to implement complex behaviors specified in language. We evaluate MaestroMotif using a suite of complex tasks in the NetHack Learning Environment (NLE), demonstrating that it surpasses existing approaches in both performance and usability.

Cited by

• Zero-Shot Instruction Following in RL via Structured LTL Representations
• Beyond Fixed Tasks: Unsupervised Environment Design for Task-Level Pairs
• Automated Reward Design for Gran Turismo
• Multi-Agent Transactive Memory
• ARM-FM: Automated Reward Machines via Foundation Models for Compositional Reinforcement Learning
• One Life to Learn: Inferring Symbolic World Models for Stochastic Environments from Unguided Exploration
• SCOPE: Synthetic Conditional Objectives for Policy Evolution in Black-Box Combinatorial Optimization
• Scalable Option Learning in High-Throughput Environments

Discussions

• Can AI agents adapt zero-shot, to complex multi-step language instructions in open-ended environments? We present MaestroMotif, a method for skill design that produces highly capable and steerable hi [bsky, 21 points, 1 comments]

Related

• LLF-Bench: Benchmark for Interactive Learning from Language Feedback
• Automated Skill Discovery for Language Agents through Exploration and Iterative Feedback
• Better than Your Teacher: LLM Agents that learn from Privileged AI Feedback
• Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills
• Building AI Companions that Prioritise Learning over Performance