FedCCEA : A Practical Approach of Client Contribution Evaluation for Federated Learning

2021/06/04 by Sung Kuk Shyn, Shyn, Sung Kuk, Dong-Hee Kim +3 · 2 citations
Computer Science · Decision Sciences · Mathematics · #Advanced Causal Inference Techniques #Artificial Intelligence (cs.AI) #Computer Science and Game Theory (cs.GT) #Data Quality and Management #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #Privacy-Preserving Technologies in Data #and Cluster Computing (cs.DC)

paper · pdf · doi:10.48550/arxiv.2106.02310

openalex publication_date 2021/06/04 · openalex created_date 2021/06/22 · openalex updated_date 2026/07/28

Abstract

Client contribution evaluation, also known as data valuation, is a crucial approach in federated learning(FL) for client selection and incentive allocation. However, due to restrictions of accessibility of raw data, only limited information such as local weights and local data size of each client is open for quantifying the client contribution. Using data size from available information, we introduce an empirical evaluation method called Federated Client Contribution Evaluation through Accuracy Approximation(FedCCEA). This method builds the Accuracy Approximation Model(AAM), which estimates a simulated test accuracy using inputs of sampled data size and extracts the clients' data quality and data size to measure client contribution. FedCCEA strengthens some advantages: (1) enablement of data size selection to the clients, (2) feasible evaluation time regardless of the number of clients, and (3) precise estimation in non-IID settings. We demonstrate the superiority of FedCCEA compared to previous methods through several experiments: client contribution distribution, client removal, and robustness test to partial participation.

Citations

• Gradient-based learning applied to document recognition
• Towards Efficient Data Valuation Based on the Shapley Value
• Data Shapley: Equitable Valuation of Data for Machine Learning
• An Empirical Study of Example Forgetting during Deep Neural Network Learning
• Advances and Open Problems in Federated Learning
• Collaborative Fairness in Federated Learning

Cited by

• PQFed: A Privacy-Preserving Quality-Controlled Federated Learning Framework
• Owen Sampling Accelerates Contribution Estimation in Federated Learning

Related

• Contribution Evaluation in Federated Learning: Examining Current Approaches
• Towards Fair, Robust and Efficient Client Contribution Evaluation in Federated Learning
• Efficient Client Contribution Evaluation for Horizontal Federated Learning
• CoAst: Validation-Free Contribution Assessment for Federated Learning based on Cross-Round Valuation
• Owen Sampling Accelerates Contribution Estimation in Federated Learning