Towards federated unsupervised representation learning

B.R.D. van Berlo, Aaqib Saeed, Tanir Özçelebi

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Making deep learning models efficient at inferring nowadays requires training with an extensive number of labeled data that are gathered in a centralized system. However, gathering labeled data is an expensive and time-consuming process, centralized systems cannot aggregate an ever-increasing amount of data and aggregating user data is raising privacy concerns. Federated learning solves data volume and privacy issues by leaving user data on devices, but is limited to use cases where labeled data can be generated from user interaction. Unsupervised representation learning reduces the amount of labeled data required for model training, but previous work is limited to centralized systems. This work introduces federated unsupervised representation learning, a novel software architecture that uses unsupervised representation learning to pre-train deep neural networks using unlabeled data in a federated setting. Pre-trained networks can be used to extract discriminative features. The features help learn a down-stream task of interest with a reduced amount of labeled data. Based on representation performance experiments with human activity detection it is recommended to pre-train with unlabeled data originating from more users performing a bigger set of activities compared to data used with the down-stream task of interest. As a result, competitive or superior performance compared to supervised deep learning is achieved.
Original languageEnglish
Title of host publicationEdgeSys '20: Proceedings of the Third ACM International Workshop on Edge Systems, Analytics and Networking
Place of PublicationNew York, NY, United States
PublisherAssociation for Computing Machinery, Inc
Pages31-36
Number of pages6
ISBN (Electronic)978-1-4503-7132-2
DOIs
Publication statusPublished - 27 Apr 2020
EventThird ACM International Workshop on Edge Systems, Analytics and Networking - Heraklion, Greece
Duration: 27 Apr 202027 Apr 2020

Workshop

WorkshopThird ACM International Workshop on Edge Systems, Analytics and Networking
Abbreviated titleEdgeSys '20
CountryGreece
CityHeraklion
Period27/04/2027/04/20

Keywords

  • Deep learning
  • Federated learning
  • Human activity detection
  • Self-supervised learning
  • Unsupervised representation learning

Fingerprint Dive into the research topics of 'Towards federated unsupervised representation learning'. Together they form a unique fingerprint.

  • Cite this

    van Berlo, B. R. D., Saeed, A., & Özçelebi, T. (2020). Towards federated unsupervised representation learning. In EdgeSys '20: Proceedings of the Third ACM International Workshop on Edge Systems, Analytics and Networking (pp. 31-36). Association for Computing Machinery, Inc. https://doi.org/10.1145/3378679.3394530