2023/07/27 by Connor Mclaughlin, Mclaughlin, Connor, Matthew Ding +5
Computer Science · Mathematics · #Artificial intelligence #Bayesian Modeling and Causal Inference #Byzantine fault tolerance #Computer network #Computer science #Distributed #Distributed Sensor Networks and Detection Algorithms #Distributed computing #FOS: Computer and information sciences #Fault tolerance #Machine Learning (cs.LG) #Machine learning #Network packet #Networking and Internet Architecture (cs.NI) #Parallel #Quantum Byzantine agreement #Random Matrices and Applications #and Cluster Computing (cs.DC)
paper · pdf · doi:10.48550/arxiv.2307.14952
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/07/27 · openalex created_date 2023/07/29 · openalex updated_date 2026/07/28
As the network scale increases, existing fully distributed solutions start to lag behind the real-world challenges such as (1) slow information propagation, (2) network communication failures, and (3) external adversarial attacks. In this paper, we focus on hierarchical system architecture and address the problem of non-Bayesian learning over networks that are vulnerable to communication failures and adversarial attacks. On network communication, we consider packet-dropping link failures. We first propose a hierarchical robust push-sum algorithm that can achieve average consensus despite frequent packet-dropping link failures. We provide a sparse information fusion rule between the parameter server and arbitrarily selected network representatives. Then, interleaving the consensus update step with a dual averaging update with Kullback-Leibler (KL) divergence as the proximal function, we obtain a packet-dropping fault-tolerant non-Bayesian learning algorithm with provable convergence guarantees. On external adversarial attacks, we consider Byzantine attacks in which the compromised agents can send maliciously calibrated messages to others (including both the agents and the parameter server). To avoid the curse of dimensionality of Byzantine consensus, we solve the non-Bayesian learning problem via running multiple dynamics, each of which only involves Byzantine consensus with scalar inputs. To facilitate resilient information propagation across sub-networks, we use a novel Byzantine-resilient gossiping-type rule at the parameter server.