2024/10/02 by Sam Earle, Earle, Sam, Samyak Parajuli +3 · 7 citations
Computer Science · Psychology · Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Games #Computation and Language (cs.CL) #Digital Games and Media #Educational Games and Gamification #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC)
paper · pdf · doi:10.48550/arxiv.2410.01791
openalex publication_date 2024/10/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Coding assistants are increasingly leveraged in game design, both generating code and making high-level plans. To what degree can these tools align with developer workflows, and what new modes of human-computer interaction can emerge from their use? We present DreamGarden, an AI system capable of assisting with the development of diverse game environments in Unreal Engine. At the core of our method is an LLM-driven planner, capable of breaking down a single, high-level prompt -- a dream, memory, or imagined scenario provided by a human user -- into a hierarchical action plan, which is then distributed across specialized submodules facilitating concrete implementation. This system is presented to the user as a garden of plans and actions, both growing independently and responding to user intervention via seed prompts, pruning, and feedback. Through a user study, we explore design implications of this system, charting courses for future work in semi-autonomous assistants and open-ended simulation design.