CubeTR: Learning to Solve The Rubiks Cube Using Transformers

2021/11/11 by Mustafa Chasmai, Chasmai, Mustafa Ebrahim · 1 citation
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Artificial Intelligence (cs.AI) #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2111.06036

openalex publication_date 2021/11/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Since its first appearance, transformers have been successfully used in wide ranging domains from computer vision to natural language processing. Application of transformers in Reinforcement Learning by reformulating it as a sequence modelling problem was proposed only recently. Compared to other commonly explored reinforcement learning problems, the Rubiks cube poses a unique set of challenges. The Rubiks cube has a single solved state for quintillions of possible configurations which leads to extremely sparse rewards. The proposed model CubeTR attends to longer sequences of actions and addresses the problem of sparse rewards. CubeTR learns how to solve the Rubiks cube from arbitrary starting states without any human prior, and after move regularisation, the lengths of solutions generated by it are expected to be very close to those given by algorithms used by expert human solvers. CubeTR provides insights to the generalisability of learning algorithms to higher dimensional cubes and the applicability of transformers in other relevant sparse reward scenarios.

Citations

• High-Dimensional Continuous Control Using Generalized Advantage\n Estimation
• Playing Atari with Deep Reinforcement Learning
• Dueling Network Architectures for Deep Reinforcement Learning
• Computational Theories of Curiosity-Driven Learning
• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges.
• Decision Transformer: Reinforcement Learning via Sequence Modeling
• Highly accurate protein structure prediction with AlphaFold

Cited by

• CayleyPy Growth: Efficient growth computations and hundreds of new conjectures on Cayley graphs (Brief version)

Related

• Solving Rubik's Cube Without Tricky Sampling
• Towards Learning Rubik's Cube with N-tuple-based Reinforcement Learning
• Solving the Rubik's Cube Without Human Knowledge
• CubeRobot: Grounding Language in Rubik's Cube Manipulation via Vision-Language Model
• Self-Supervision is All You Need for Solving Rubik's Cube