Dyna, an integrated architecture for learning, planning, and reacting

1991/07/01 by Richard S. Sutton · 9 citations

paper · doi:10.1145/122344.122377

crossref issued 1991/07/01 · crossref published 1991/07/01 · crossref published-online 1991/07/01 · crossref published-print 1991/07/01 · crossref created 2007/01/17 · crossref deposited 2022/12/28 · crossref indexed 2026/08/01

Abstract

Dyna is an AI architecture that integrates learning, planning, and reactive execution. Learning methods are used in Dyna both for compiling planning results and for updating a model of the effects of the agent's actions on the world. Planning is incremental and can use the probabilistic and ofttimes incorrect world models generated by learning processes. Execution is fully reactive in the sense that no planning intervenes between perception and action. Dyna relies on machine learning methods for learning from examples---these are among the basic building blocks making up the architecture---yet is not tied to any particular method. This paper briefly introduces Dyna and discusses its strengths and weaknesses with respect to other architectures.

Cited by

• A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures
• Solaris: Building a Multiplayer Video World Model in Minecraft
• Theoretical Foundations and Effective Algorithms for Policy-Aware Simulator Learning
• World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems
• On the Role of Computation in Reinforcement Learning
• Coupled Control, Structured Memory, and Verifiable Action in Agentic AI (SCRAT -- Stochastic Control with Retrieval and Auditable Trajectories): A Comparative Perspective from Squirrel Locomotion and Scatter-Hoarding
• Offline Robotic World Model: Learning Robotic Policies without a Physics Simulator
• MARFT: Multi-Agent Reinforcement Fine-Tuning
• Dream-MPC: Gradient-Based Model Predictive Control with Latent Imagination