2025/10/22 by Archana Warrier, Dat Nguyen, Warrier, Archana +22 · 8 voices · 2 citations
Psychology · Computer Science · Neuroscience · #Social Robot Interaction and HRI #Reinforcement Learning in Robotics #Embodied and Extended Cognition
paper · pdf · doi:10.48550/arxiv.2510.19788
AutumnBench Dataset Release This dataset accompanies the paper “Benchmarking World-Model Learning with Environment-Level Queries.” It contains the benchmark release for AutumnBench, including task prompts, ground-truth answers, and environment programs used for evaluation. The release includes a `prompts/` folder containing the task prompts for each AutumnBench environment and challenge task. Files are named envnametaskname.json, where taskname is one of mfp, cd, or planning. It also includes an `answers/` folder containing the ground-truth answers for each task. For mfp, each file gives the index of the correct answer option (0-indexed). For cd, each file gives the condition after which the change occurs. For planning, each file gives the optimal number of steps required to reach the target. The `programs/` folder contains the Autumn programs for the original environments, as well as the modified environments used in the change-detection task. The Autumn interpreter for running the programs is available in the GitHub repository: https://github.com/BasisResearch/Autumn.cpp. Finally, the release includes a shared `colordict.yaml` file that maps numeric identifiers to color names. This release is intended to support reproducibility and further analysis of the AutumnBench benchmark described in “Benchmarking World-Model Learning with Environment-Level Queries.”