Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation
2024/10/17 by Nam‐Young Kim, Chae, Hyungjoo, Kai Tzu-iunn Ong +12 · 45 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Mobile Agent-Based Network Management #Multi-Agent Systems and Negotiation
paper · pdf · doi:10.48550/arxiv.2410.13232
openalex publication_date 2024/10/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Abstract
Large language models (LLMs) have recently gained much attention in building autonomous agents. However, the performance of current LLM-based web agents in long-horizon tasks is far from optimal, often yielding errors such as repeatedly buying a non-refundable flight ticket. By contrast, humans can avoid such an irreversible mistake, as we have an awareness of the potential outcomes (e.g., losing money) of our actions, also known as the "world model". Motivated by this, our study first starts with preliminary analyses, confirming the absence of world models in current LLMs (e.g., GPT-4o, Claude-3.5-Sonnet, etc.). Then, we present a World-model-augmented (WMA) web agent, which simulates the outcomes of its actions for better decision-making. To overcome the challenges in training LLMs as world models predicting next observations, such as repeated elements across observations and long HTML inputs, we propose a transition-focused observation abstraction, where the prediction objectives are free-form natural language descriptions exclusively highlighting important state differences between time steps. Experiments on WebArena and Mind2Web show that our world models improve agents' policy selection without training and demonstrate our agents' cost- and time-efficiency compared to recent tree-search-based agents.
Cited by
- Scaling GUI Agents with Visual State Transitions
- Hybrid Analysis for Secure MCP Tool Use in LLM Agents
- From Word to World: Can Large Language Models be Implicit Text-based World Models?
- MobileWorldBench: Towards Semantic World Modeling For Mobile Agents
- Test-Time Adaptation for LLM Agents via Environment Interaction
- WebATLAS: An LLM Agent with Experience-Driven Memory and Action Simulation
- LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training
- MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents
- R-WoM: Retrieval-augmented World Model For Computer-use Agents
- Dyna-Mind: Learning to Simulate from Experience for Better AI Agents
- Agent Learning via Early Experience
- A Goal Without a Plan Is Just a Wish: Efficient and Effective Global Planner Training for Long-Horizon Agent Tasks
- Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI
- AutoContext: Instance-Level Context Learning for LLM Agents
- TAPO: Transition-Aware Policy Optimization for LLM Agents
- LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions
- ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization
- Combating the Memory Walls: Optimization Pathways for Long-Context Agentic LLM Inference
- Planning with Reasoning using Vision Language World Model
- UI-Venus Technical Report: Building High-performance UI Agents with RFT
- Designing Memory-Augmented AR Agents for Spatiotemporal Reasoning in Personalized Task Assistance
- Reinforcement Learning for Large Model: A Survey
- When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs
- OS-MAP: How Far Can Computer-Using Agents Go in Breadth and Depth?
- WebWorld: A Large-Scale World Model for Web Agent Training
- WebSynthesis: World-Model-Guided MCTS for Efficient WebUI-Trajectory Synthesis
- WebArXiv: Evaluating Multimodal Agents on Time-Invariant arXiv Tasks
- Beyond Syntax: Action Semantics Learning for App Agents
- MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents
- IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environment
- WebChoreArena: Evaluating Web Browsing Agents on Realistic Tedious Web Tasks
- Dyna-Think: Synergizing Reasoning, Acting, and World Model Simulation in AI Agents
- ToolHaystack: Stress-Testing Tool-Augmented Language Models in Realistic Long-Term Interactions
- WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning
- Web-Shepherd: Advancing PRMs for Reinforcing Web Agents
- RLVR-World: Training World Models with Reinforcement Learning
- Quo Vadis, World Modeling?
- Designing Agent-Ready Websites for AI Web Agents: A Framework for Machine Readability, Actionability, and Decision Reliability
- Internalizing the Future: A Unified Agentic Training Paradigm for World Model Planning
- Generative Visual Code Mobile World Models
- BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agents
- WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Model
- AppDeltaWorld: Transition-Grounded Delta Code World Model for Mobile GUI Agents
- ViMo: A Generative Visual GUI World Model for App Agents
- SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills
Related