Learning to Reason over Scene Graphs: A Case Study of Finetuning GPT-2 into a Robot Language Model for Grounded Task Planning

2023/05/12 by Chalvatzaki, Georgia, Younes, Ali, Nandha, Daljeet +3 · 2 citations
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Robotics (cs.RO)

paper · doi:10.48550/arxiv.2305.07716

Abstract

Long-horizon task planning is essential for the development of intelligent assistive and service robots. In this work, we investigate the applicability of a smaller class of large language models (LLMs), specifically GPT-2, in robotic task planning by learning to decompose tasks into subgoal specifications for a planner to execute sequentially. Our method grounds the input of the LLM on the domain that is represented as a scene graph, enabling it to translate human requests into executable robot plans, thereby learning to reason over long-horizon tasks, as encountered in the ALFRED benchmark. We compare our approach with classical planning and baseline methods to examine the applicability and generalizability of LLM-based planners. Our findings suggest that the knowledge stored in an LLM can be effectively grounded to perform long-horizon task planning, demonstrating the promising potential for the future application of neuro-symbolic planning methods in robotics.

Cited by

• Intuitive Programming, Adaptive Task Planning, and Dynamic Role Allocation in Human-Robot Collaboration
• Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism

Related

• DELTA: Decomposed Efficient Long-Term Robot Task Planning using Large Language Models
• SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning
• Can only LLMs do Reasoning?: Potential of Small Language Models in Task Planning
• Can LLM-Reasoning Models Replace Classical Planning? A Benchmark Study
• Fast and Accurate Task Planning using Neuro-Symbolic Language Models and Multi-level Goal Decomposition