2025/11/11 by Miroslav Lžičař, Lžičař, Miroslav · 1 voice
Computer Science · Engineering · #68T07 #68T20 #Abstraction #Baseline (sea) #Benchmark (surveying) #Cellular Automata and Applications #Cellular automaton #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Generalization #I.2.6 #I.2.8 #I.5.1 #Machine Learning (cs.LG) #Machine Learning and Algorithms #Task (project management) #cs.LG
paper · pdf · doi:10.48550/arxiv.2511.07908
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2025/11/11 · arxiv published 2025/11/11 · arxiv updated 2025/11/11 · openalex created_date 2025/11/13 · openalex updated_date 2026/07/28
We introduce CellARC, a synthetic benchmark for abstraction and reasoning built from multicolor 1D cellular automata (CA). Each episode has five support pairs and one query serialized in 256 tokens, enabling rapid iteration with small models while exposing a controllable task space with explicit knobs for alphabet size k, radius r, rule family, Langton's lambda, query coverage, and cell entropy. We release 95k training episodes plus two 1k test splits (interpolation/extrapolation) and evaluate symbolic, recurrent, convolutional, transformer, recursive, and LLM baselines. CellARC decouples generalization from anthropomorphic priors, supports unlimited difficulty-controlled sampling, and enables reproducible studies of how quickly models infer new rules under tight budgets. Our strongest small-model baseline (a 10M-parameter vanilla transformer) outperforms recent recursive models (TRM, HRM), reaching 58.0%/32.4% per-token accuracy on the interpolation/extrapolation splits, while a large closed model (GPT-5 High) attains 62.3%/48.1% on subsets of 100 test tasks. An ensemble that chooses per episode between the Transformer and the best symbolic baseline reaches 65.4%/35.5%, highlighting neuro-symbolic complementarity. Leaderboard: https://cellarc.mireklzicar.com