Integration of TinyML and LargeML: A Survey of 6G and Beyond
2025/05/20 by Vu, Thai-Hoc, Tu, Ngo Hoang, Huynh-The, Thien +4
#Artificial Intelligence (cs.AI) #Emerging Technologies (cs.ET) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multiagent Systems (cs.MA) #Networking and Internet Architecture (cs.NI)
paper · doi:10.48550/arxiv.2505.15854
Abstract
The transition from 5G networks to 6G highlights a significant demand for machine learning (ML). Deep learning models, in particular, have seen wide application in mobile networking and communications to support advanced services in emerging wireless environments, such as smart healthcare, smart grids, autonomous vehicles, aerial platforms, digital twins, and the metaverse. The rapid expansion of Internet-of-Things (IoT) devices, many with limited computational capabilities, has accelerated the development of tiny machine learning (TinyML) and resource-efficient ML approaches for cost-effective services. However, the deployment of large-scale machine learning (LargeML) solutions require major computing resources and complex management strategies to support extensive IoT services and ML-generated content applications. Consequently, the integration of TinyML and LargeML is projected as a promising approach for future seamless connectivity and efficient resource management.
Although the integration of TinyML and LargeML shows abundant potential, several challenges persist, including performance optimization, practical deployment strategies, effective resource management, and security considerations. In this survey, we review and analyze the latest research aimed at enabling the integration of TinyML and LargeML models for the realization of smart services and applications in future 6G networks and beyond. The paper concludes by outlining critical challenges and identifying future research directions for the holistic integration of TinyML and LargeML in next-generation wireless networks.
Citations
- Incentivizing Multi-Tenant Split Federated Learning for Foundation Models at the Network Edge
- Encrypted Large Model Inference: The Equivariant Encryption Paradigm
- Consolidating TinyML Lifecycle with Large Language Models: Reality, Illusion, or Opportunity?
- Large Model Empowered Metaverse: State-of-the-Art, Challenges and Opportunities
- Integrated Sensing and Edge AI: Realizing Intelligent Perception in 6G
- On Privacy, Security, and Trustworthiness in Distributed Wireless Large AI Models (WLAM)
- Efficient Federated Finetuning of Tiny Transformers with Resource-Constrained Devices
- FedPT: Federated Proxy-Tuning of Large Language Models on Resource-Constrained Edge Devices
- Leveraging Foundation Models for Efficient Federated Learning in Resource-restricted Edge Networks
- FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts
- SFPrompt: Communication-Efficient Split Federated Fine-Tuning for Large Pre-Trained Models over Resource-Limited Devices
- Toward Explainable Reasoning in 6G: A Proof of Concept Study on Radio Resource Allocation
- MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization
- Data Collection and Labeling Techniques for Machine Learning
- Synergizing Foundation Models and Federated Learning: A Survey
- A Survey on Intelligent Internet of Things: Applications, Security, Privacy, and Future Directions
- Applications of Generative AI (GAI) for Mobile and Wireless Networking: A Survey
- RIOT-ML: toolkit for over-the-air secure updates and performance evaluation of TinyML models
- UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs
- Heterogeneous Federated Learning with Splited Language Model
- Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey
- AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networks
- Adaptive Split Learning over Energy-Constrained Wireless Edge Networks
- Towards Intent-Based Network Management: Large Language Models for Intent Extraction in 5G Core Networks
- ESFL: Efficient Split Federated Learning over Resource-Constrained Heterogeneous Wireless Devices
- Resource-efficient Parallel Split Learning in Heterogeneous Edge Computing
- A Plug-in Tiny AI Module for Intelligent and Selective Sensor Data Transmission
- MP-SL: Multihop Parallel Split Learning
- When Large Language Model Agents Meet 6G Networks: Perception, Grounding, and Alignment
- Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models
- Thousands of AI Authors on the Future of AI
- Understanding LLMs: A Comprehensive Overview from Training to Inference
- Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Review
- FedSplitX: Federated Split Learning for Computationally-Constrained Heterogeneous Clients
- Applications of Distributed Machine Learning for the Internet-of-Things: A Comprehensive Survey
- A Machine Learning-oriented Survey on Tiny Machine Learning
- A Human-Centric Metaverse Enabled by Brain-Computer Interface: A Survey
- Explainability for Large Language Models: A Survey
- Large Language Models for Software Engineering: A Systematic Literature Review
- Large Language Models for Software Engineering: A Systematic Literature Review
- Optimal Resource Allocation for U-Shaped Parallel Split Learning
- Big AI Models for 6G Wireless Networks: Opportunities, Challenges, and Research Directions
- Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimization
- TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge
- TinyMetaFed: Efficient Federated Meta-Learning for TinyML
- A Survey on Evaluation of Large Language Models
- A Survey on Evaluation of Large Language Models
- When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions
- Split Learning in 6G Edge Networks
- Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation
- The ITU Vision and Framework for 6G: Scenarios, Capabilities and Enablers
- TinyReptile: TinyML with Federated Meta-Learning
- Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks
- Privacy and Efficiency of Communications in Federated Split Learning
- A Bargaining Game for Personalized, Energy Efficient Split Learning over Wireless Networks
- Federated Learning and Meta Learning: Approaches, Applications, and Directions
- On-Device Training Under 256KB Memory
- 6G Network AI Architecture for Everyone-Centric Customized Services
- Zero-Touch Network on Industrial IoT: An End-to-End Machine Learning Approach
- Split Learning over Wireless Networks: Parallel Design and Resource Management
- Incentive Mechanism Design for Joint Resource Allocation in Blockchain-based Federated Learning
- Incentive Mechanism Design for Joint Resource Allocation in Blockchain-Based Federated Learning
- Examining Scaling and Transfer of Language Model Architectures for Machine Translation
- GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
- Applications of Explainable AI for 6G: Technical Aspects, Use Cases, and Research Challenges
- Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
- Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
- Efficiently Modeling Long Sequences with Structured State Spaces
- Nine Challenges in Artificial Intelligence and Wireless Communications for 6G
- Splitfed learning without client-side synchronization: Analyzing client-side split network portion size to overall performance
- LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting
- Integrated Sensing and Communications: Towards Dual-functional Wireless Networks for 6G and Beyond
- Integrated Sensing and Communications: Toward Dual-Functional Wireless Networks for 6G and Beyond
- LoRA: Low-Rank Adaptation of Large Language Models
- Wireless Federated Learning (WFL) for 6G Networks -- Part I: Research\n Challenges and Future Trends
- Transfer Learning for Future Wireless Networks: A Comprehensive Survey
- Survey on Aerial Radio Access Networks: Toward a Comprehensive 6G Access Infrastructure
- TinyML for Ubiquitous Edge AI
- Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
- Advancements of federated learning towards privacy preservation: from federated learning to split learning
- TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems
- A Survey on Large-scale Machine Learning
- SplitFed: When Federated Learning Meets Split Learning
- Vision, Requirements, and Technology Trend of 6G: How to Tackle the Challenges of System Coverage, Capacity,User Data-Rate and Movement Speed
- Optimal Mini-Batch Size Selection for Fast Gradient Descent
- Learning in the Machine: To Share or Not to Share?
- FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare
- Learning What and Where to Transfer
- Adversarial attacks on medical machine learning
- Parameter-Efficient Transfer Learning for NLP
- Split learning for health: Distributed deep learning without sharing raw patient data
- Instance-based Deep Transfer Learning
- MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence
- To prune, or not to prune: exploring the efficacy of pruning for model compression
- Attention Is All You Need
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Membership Inference Attacks against Machine Learning Models
- Deep CORAL: Correlation Alignment for Deep Domain Adaptation
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Generalized Low Rank Models
- Domain Adaptive Neural Networks for Object Recognition
- A Survey on Transfer Learning
- Federated Domain Generalization with Data-free On-server Matching Gradient
- Advances and Open Challenges in Federated Foundation Models
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Related