Integrating Quantized LLMs into Robotics Systems as Edge AI to Leverage their Natural Language Processing Capabilities

2025/06/11 by Miguel Á. González-Santamarta, Francisco J. Rodríguez-Lera, González-Santamarta, Miguel Á. +7
Computer Science · #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Robotics (cs.RO) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2506.09581

openalex publication_date 2025/06/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) have experienced great advancements in the last year resulting in an increase of these models in several fields to face natural language tasks. The integration of these models in robotics can also help to improve several aspects such as human-robot interaction, navigation, planning and decision-making. Therefore, this paper introduces llama_ros, a tool designed to integrate quantized Large Language Models (LLMs) into robotic systems using ROS 2. Leveraging llama.cpp, a highly optimized runtime engine, llama_ros enables the efficient execution of quantized LLMs as edge artificial intelligence (AI) in robotics systems with resource-constrained environments, addressing the challenges of computational efficiency and memory limitations. By deploying quantized LLMs, llama_ros empowers robots to leverage the natural language understanding and generation for enhanced decision-making and interaction which can be paired with prompt engineering, knowledge graphs, ontologies or other tools to improve the capabilities of autonomous robots. Additionally, this paper provides insights into some use cases of using llama_ros for planning and explainability in robotics.

Citations

• The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
• A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
• SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models
• Graph of Thoughts: Solving Elaborate Problems with Large Language Models
• Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• ChatGPT for Robotics: Design Principles and Model Abilities
• Robot Operating System 2: Design, architecture, and uses in the wild
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Training Verifiers to Solve Math Word Problems
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• Deep Learning with Low Precision by Half-wave Gaussian Quantization
• Fixed Point Quantization of Deep Convolutional Networks
• Integration of Large Language Models within Cognitive Architectures for Autonomous Robots

Related

• Deployment of Large Language Models to Control Mobile Robots at the Edge
• Large Language Models for Robotics: Opportunities, Challenges, and Perspectives
• LLM A*: Human in the Loop Large Language Models Enabled A* Search for Robotics
• Large Language Models for Robotics: A Survey
• Integration of Large Language Models within Cognitive Architectures for Autonomous Robots