2024/07/05 by Yiyang Huang, Huang, Yiyang, Yuhui Hao +19 · 4 citations
Computer Science · Engineering · #FOS: Computer and information sciences #Hardware Architecture (cs.AR) #Modular Robots and Swarm Intelligence #Robotic Path Planning Algorithms #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.2407.04292
openalex publication_date 2024/07/05 · openalex created_date 2024/07/09 · openalex updated_date 2026/07/28
Embodied AI robots have the potential to fundamentally improve the way human beings live and manufacture. Continued progress in the burgeoning field of using large language models to control robots depends critically on an efficient computing substrate, and this trend is strongly evident in manipulation tasks. In particular, today's computing systems for embodied AI robots for manipulation tasks are designed purely based on the interest of algorithm developers, where robot actions are divided into a discrete frame basis. Such an execution pipeline creates high latency and energy consumption. This paper proposes Corki\xspace, an algorithm-architecture co-design framework for real-time embodied AI-powered robotic manipulation applications. We aim to decouple LLM inference, robotic control, and data communication in the embodied AI robots' compute pipeline. Instead of predicting action for one single frame, Corki\xspace predicts the trajectory for the near future to reduce the frequency of LLM inference. The algorithm is coupled with a hardware that accelerates transforming trajectory into actual torque signals used to control robots and an execution pipeline that parallels data communication with computation. Corki\xspace largely reduces LLM inference frequency by up to 5.1×, resulting in up to 5.9× speed up. The success rate improvement can be up to 13.9%.