GAPSL: A Gradient-Aligned Parallel Split Learning over Data-Heterogeneous Edge Computing Systems

2026/03/31 by Zheng Lin, Ons Aouedi, Zihan Fang +4
Computer Science · #cs.LG

paper · pdf

13 pages, 21 figures

arxiv created 2026/08/03 · arxiv updated 2026/08/04

Abstract

The increasing complexity of neural networks poses significant challenges for democratizing federated learning (FL) on resource-constrained edge devices. Parallel split learning (PSL) has emerged as a promising solution by offloading substantial computing workload to a server via model partitioning, shrinking client-side computing load, and eliminating the client-side model aggregation for reduced communication and deployment costs. However, the highly heterogeneous nature of client data in edge computing systems causes aggregation-free PSL to suffer from severe training divergence, stemming from gradient directional inconsistency across clients. To address this challenge, we propose GAPSL, a gradient-aligned PSL framework tailored for data-heterogeneous edge systems, which comprises two key components: leader gradient identification (LGI) and gradient direction alignment (GDA). LGI dynamically selects a set of directionally consistent device gradients to construct a leader gradient as a robust proxy for the global convergence trend. GDA employs a direction-aware regularization to align each client's gradient with the leader gradient, thereby mitigating inter-device gradient directional inconsistency and enhancing model convergence. We evaluate GAPSL on a prototype computing testbed. Extensive experiments demonstrate that GAPSL consistently outperforms state-of-the-art benchmarks in training accuracy, convergence latency, and system robustness under severe data heterogeneity.

Citations

• Data Heterogeneity and Forgotten Labels in Split Federated Learning
• Dynamic Uncertainty-aware Multimodal Fusion for Outdoor Health Monitoring
• Towards Collaborative Fairness in Federated Learning Under Imbalanced Covariate Shift
• HASFL: Heterogeneity-aware Split Federated Learning over Edge Computing Systems
• FSL-SAGE: Accelerating Federated Split Learning via Smashed Activation Gradient Estimation
• HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
• FedAWA: Adaptive Optimization of Aggregation Weights in Federated Learning Using Client Vectors
• FedMobile: Enabling Knowledge Contribution-aware Multi-modal Federated Learning with Incomplete Modalities
• LEO-Split: A Semi-Supervised Split Learning Framework over LEO Satellite Networks
• Unleashing the Power of Continual Learning on Non-Centralized Devices: A Survey
• Hierarchical Split Federated Learning: Convergence Analysis and System Optimization
• Optimizing Personalized Federated Learning through Adaptive Layer-Wise Learning
• Parallel Split Learning with Global Sampling
• Faster Convergence on Heterogeneous Federated Edge Learning: An Adaptive Clustered Data Sharing Approach
• Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data
• Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Models
• AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networks
• FedSN: A Federated Learning Framework over Heterogeneous LEO Satellite Networks
• FedCiR: Client-Invariant Representation Learning for Federated Non-IID Features
• Towards Personalized Federated Learning via Heterogeneous Model Reassembly
• FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence
• Split Learning in 6G Edge Networks
• Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks
• A Bargaining Game for Personalized, Energy Efficient Split Learning over Wireless Networks
• Split Learning over Wireless Networks: Parallel Design and Resource Management
• Actions at the Edge: Jointly Optimizing the Resources in Multi-access Edge Computing
• Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning
• Splitfed learning without client-side synchronization: Analyzing client-side split network portion size to overall performance
• Unleashing the Tiger: Inference Attacks on Split Learning
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• SplitFed: When Federated Learning Meets Split Learning
• Split learning for health: Distributed deep learning without sharing raw patient data
• Very Deep Convolutional Networks for Large-Scale Image Recognition