Learning sparse neural networks for better generalization

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)
78 Downloads (Pure)

Abstract

Deep neural networks perform well on test data when they are highly overparameterized, which, however, also leads to large cost to train and deploy them. As a leading approach to address this problem, sparse neural networks have been widely used to significantly reduce the size of networks, making them more efficient during training and deployment, without compromising performance. Recently, sparse neural networks, either compressed from a pre-trained model or obtained by training from scratch, have been observed to be able to generalize as well as or even better than their dense counterparts. However, conventional techniques to find well fitted sparse sub-networks are expensive and the mechanisms underlying this phenomenon are far from clear. To tackle these problems, this Ph.D. research aims to study the generalization of sparse neural networks, and to propose more efficient approaches that can yield sparse neural networks with generalization bounds.

Original languageEnglish
Title of host publicationProceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020
EditorsChristian Bessiere
PublisherInternational Joint Conferences on Artificial Intelligence
Pages5190-5191
Number of pages2
ISBN (Electronic)9780999241165
Publication statusPublished - 2020
Event29th International Joint Conference on Artificial Intelligence - 17th Pacific Rim International Conference on Artificial Intelligence, IJCAI 2020, PRICAI 2020 - Pacifico Convention Plaza Yokohama, Yokohama, Japan
Duration: 11 Jul 202017 Jul 2020
Conference number: 29
https://ijcai20.org/

Conference

Conference29th International Joint Conference on Artificial Intelligence - 17th Pacific Rim International Conference on Artificial Intelligence, IJCAI 2020, PRICAI 2020
Abbreviated titleIJCAI-PRICAI 2020
Country/TerritoryJapan
CityYokohama
Period11/07/2017/07/20
Internet address

Bibliographical note

Publisher Copyright:
© 2020 Inst. Sci. inf., Univ. Defence in Belgrade. All rights reserved.

Keywords

  • Sparse neural networks
  • Generalization learning
  • sparse training

Fingerprint

Dive into the research topics of 'Learning sparse neural networks for better generalization'. Together they form a unique fingerprint.

Cite this