Pushing Large Language Models to the 6G Edge: Vision, Challenges, and Opportunities

2023/09/28 by Zheng Lin, Guanqiao Qu, Lin, Zheng +9 · 19 citations
Computer Science · Engineering · Medicine · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Machine Learning (cs.LG) #Topic Modeling #Tracheal and airway disorders

paper · pdf · doi:10.48550/arxiv.2309.16739

openalex publication_date 2023/09/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs), which have shown remarkable capabilities, are revolutionizing AI development and potentially shaping our future. However, given their multimodality, the status quo cloud-based deployment faces some critical challenges: 1) long response time; 2) high bandwidth costs; and 3) the violation of data privacy. 6G mobile edge computing (MEC) systems may resolve these pressing issues. In this article, we explore the potential of deploying LLMs at the 6G edge. We start by introducing killer applications powered by multimodal LLMs, including robotics and healthcare, to highlight the need for deploying LLMs in the vicinity of end users. Then, we identify the critical challenges for LLM deployment at the edge and envision the 6G MEC architecture for LLMs. Furthermore, we delve into two design aspects, i.e., edge training and edge inference for LLMs. In both aspects, considering the inherent resource limitations at the edge, we discuss various cutting-edge techniques, including split learning/inference, parameter-efficient fine-tuning, quantization, and parameter-sharing inference, to facilitate the efficient deployment of LLMs. This article serves as a position paper for thoroughly identifying the motivation, challenges, and pathway for empowering LLMs at the 6G edge.

Cited by

• CryptoTensors: A Light-Weight Large Language Model File Format for Highly-Secure Model Distribution
• Joint Partitioning and Placement of Foundation Models for Real-Time Edge AI
• Agentic AI-Empowered Conversational Embodied Intelligence Networks in 6G
• LLM-Driven Stationarity-Aware Expert Demonstrations for Multi-Agent Reinforcement Learning in Mobile Systems
• Memory- and Latency-Constrained Inference of Large Language Models via Adaptive Split Computing
• TIGA: Trajectory-Injected Generative Attack against Black-box AIGC Detectors
• Efficient LLM Inference over Heterogeneous Edge Networks with Speculative Decoding
• Joint Communication Scheduling and Velocity Control for Multi-UAV-Assisted Post-Disaster Monitoring: An Attention-Based In-Context Learning Approach
• Adaptive Source-Channel Coding for Multi-User Semantic and Data Communications
• Generative Feature Imputing -- A Technique for Error-resilient Semantic Communication
• Otters: An Energy-Efficient SpikingTransformer via Optical Time-to-First-Spike Encoding
• Experience Scaling: Post-Deployment Evolution For Large Language Models
• LEED: A Highly Efficient and Scalable LLM-Empowered Expert Demonstrations Framework for Multi-Agent Reinforcement Learning
• Energy-Efficient Wireless LLM Inference via Uncertainty and Importance-Aware Speculative Decoding
• Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring
• Adaptive Source-Channel Coding for Semantic Communications
• Balancing Latency and Model Accuracy for Fluid Antenna-Assisted LM-Embedded MIMO Network
• RRTO: A High-Performance Transparent Offloading System for Model Inference in Mobile Edge Computing
• Talk with the Things: Integrating LLMs into IoT Networks

Related

• Mobile Edge Intelligence for Large Language Models: A Contemporary Survey
• TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computers
• When Large Language Model Agents Meet 6G Networks: Perception, Grounding, and Alignment
• Fine-Tuning Small Language Models for Domain-Specific AI: An Edge AI Perspective
• Toward Democratized Generative AI in Next-Generation Mobile Edge Networks