Towards Scheduling Federated Deep Learning using Meta-Gradients for Inter-Hospital Learning

2021/07/04 by Rasheed El-Bouri, Rasheed el-Bouri, el-Bouri, Rasheed +4
Computer Science · #Cryptography and Data Security #Cryptography and Security (cs.CR) #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Parallel #Privacy-Preserving Technologies in Data #and Cluster Computing (cs.DC) #cs.CR #cs.DC #cs.LG

paper · pdf · doi:10.48550/arxiv.2107.01707

11 pages, 8 figures

arxiv created 2021/07/04 · openalex publication_date 2021/07/04 · arxiv updated 2021/07/06 · openalex created_date 2021/07/19 · openalex updated_date 2026/07/28

Abstract

Given the abundance and ease of access of personal data today, individual privacy has become of paramount importance, particularly in the healthcare domain. In this work, we aim to utilise patient data extracted from multiple hospital data centres to train a machine learning model without sacrificing patient privacy. We develop a scheduling algorithm in conjunction with a student-teacher algorithm that is deployed in a federated manner. This allows a central model to learn from batches of data at each federal node. The teacher acts between data centres to update the main task (student) algorithm using the data that is stored in the various data centres. We show that the scheduler, trained using meta-gradients, can effectively organise training and as a result train a machine learning model on a diverse dataset without needing explicit access to the patient data. We achieve state-of-the-art performance and show how our method overcomes some of the problems faced in the federated learning such as node poisoning. We further show how the scheduler can be used as a mechanism for transfer learning, allowing different teachers to work together in training a student for state-of-the-art performance.

Citations

• Gradient-based learning applied to document recognition
• PhysioBank, PhysioToolkit, and PhysioNet
• Automated Curriculum Learning for Neural Networks
• Iterative Machine Teaching
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Towards Federated Learning at Scale: System Design
• DeepFM: A Factorization-Machine based Neural Network for CTR Prediction
• GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
• Federated Accelerated Stochastic Gradient Descent
• Learning Multiple Layers of Features from Tiny Images
• A Survey on Curriculum Learning

Related

• Preserving Patient Privacy while Training a Predictive Model of In-hospital Mortality
• Towards the Practical Utility of Federated Learning in the Medical Domain
• Split Learning for Distributed Collaborative Training of Deep Learning Models in Health Informatics
• Reliability and Performance Assessment of Federated Learning on Clinical Benchmark Data
• PMFL: Partial Meta-Federated Learning for heterogeneous tasks and its applications on real-world medical records