Communication-efficient Federated Learning through Adaptive Weight Clustering and Server-side Distillation

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/CongresprocedureConferentiebijdrageAcademicpeer review

1 Citaat (Scopus)

Samenvatting

Federated Learning (FL) is a promising technique for the collaborative training of deep neural networks across multiple devices while preserving data privacy. Despite its potential benefits, FL is hindered by excessive communication costs due to repeated server-client communication during training. To address this challenge, model compression techniques, such as sparsification and weight clustering are applied, which often require modifying the underlying model aggregation schemes or involve cumbersome hyperparameter tuning, with the latter not only adjusts the model's compression rate but also limits model's potential for continuous improvement over growing data. In this paper, we propose FedCompress, a novel approach that combines dynamic weight clustering and server-side knowledge distillation to reduce communication costs while learning highly generalizable models. Through a comprehensive evaluation on diverse public datasets, we demonstrate the efficacy of our approach compared to baselines in terms of communication costs and inference speed. We will make our implementation public upon acceptance.

Originele taal-2Engels
Titel2024 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024
UitgeverijInstitute of Electrical and Electronics Engineers
Pagina's5805-5809
Aantal pagina's5
ISBN van elektronische versie979-8-3503-4485-1
DOI's
StatusGepubliceerd - 18 mrt. 2024
Evenement49th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024 - Seoul, Zuid-Korea
Duur: 14 apr. 202419 apr. 2024

Congres

Congres49th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2024
Land/RegioZuid-Korea
StadSeoul
Periode14/04/2419/04/24

Vingerafdruk

Duik in de onderzoeksthema's van 'Communication-efficient Federated Learning through Adaptive Weight Clustering and Server-side Distillation'. Samen vormen ze een unieke vingerafdruk.

Citeer dit