Files and links (1)
Preprint (Author's original)arXiv.org - Non-exclusive license to distribute, Open
Preprint
Communication-Efficient Decentralized Federated Learning via One-Bit Compressive Sensing
arXiv.org
31 Aug 2023
Abstract
Decentralized federated learning (DFL) has gained popularity due to its
practicality across various applications. Compared to the centralized version,
training a shared model among a large number of nodes in DFL is more
challenging, as there is no central server to coordinate the training process.
Especially when distributed nodes suffer from limitations in communication or
computational resources, DFL will experience extremely inefficient and unstable
training. Motivated by these challenges, in this paper, we develop a novel
algorithm based on the framework of the inexact alternating direction method
(iADM). On one hand, our goal is to train a shared model with a sparsity
constraint. This constraint enables us to leverage one-bit compressive sensing
(1BCS), allowing transmission of one-bit information among neighbour nodes. On
the other hand, communication between neighbour nodes occurs only at certain
steps, reducing the number of communication rounds. Therefore, the algorithm
exhibits notable communication efficiency. Additionally, as each node selects
only a subset of neighbours to participate in the training, the algorithm is
robust against stragglers. Additionally, complex items are computed only once
for several consecutive steps and subproblems are solved inexactly using
closed-form solutions, resulting in high computational efficiency. Finally,
numerical experiments showcase the algorithm's effectiveness in both
communication and computation.
Metrics
5 Record Views
Details
- Title
- Communication-Efficient Decentralized Federated Learning via One-Bit Compressive Sensing
- Creators
- Shenglong ZhouKaidi XuGeoffrey Ye Li
- Publication Details
- arXiv.org
- Resource Type
- Preprint
- Language
- English
- Academic Unit
- Computer Science (Computing)
- Other Identifier
- 991021871349504721