2026/03/04 by Dai, Yinpei, Yinpei Dai, Hongze Fu +7 · 1 voice · 1 citation
Computer Science · Engineering · Psychology · #Benchmark (surveying) #Benchmarking #Code (set theory) #Memory model #Multimodal Machine Learning Applications #Robot Manipulation and Learning #Social Robot Interaction and HRI #Suite #Taxonomy (biology) #cs.AI #cs.RO
paper · pdf · doi:10.48550/arxiv.2603.04639
openalex publication_date 2026/03/04 · arxiv published 2026/03/04 · openalex created_date 2026/03/07 · arxiv updated 2026/05/26 · openalex updated_date 2026/07/28
Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipulating objects that become temporarily occluded. Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms; however, their evaluations remain confined to narrow, non-standardized settings. This limits systematic understanding, comparison, and progress measurement. To address these challenges, we introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models in long-horizon, history-dependent scenarios. Our benchmark comprises 16 manipulation tasks constructed under a carefully designed taxonomy that evaluates temporal, spatial, object, and procedural memory. We further develop a suite of 14 memory-augmented VLA variants built on the π0.5 backbone to systematically explore different memory representations across multiple integration strategies. Experimental results show that the effectiveness of memory representations is highly task-dependent, with each design offering distinct advantages and limitations across different tasks. Videos and code can be found at our website https://robomme.github.io.