Learning Sparse Neural Networks for Better Generalization

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    58 Downloads (Pure)

    Abstract

    Deep neural networks perform well on test data when they are highly overparameterized, which, however, also leads to large cost to train and deploy them. As a leading approach to address this problem, sparse neural networks have been widely used to significantly reduce the size of networks, making them more efficient during training and deployment, without compromising performance. Recently, sparse neural networks, either compressed from a pre-trained model or obtained by training from scratch, have been observed to be able to generalize as well as or even better than their dense counterparts. However, conventional techniques to find well fitted sparse sub-networks are expensive and the mechanisms underlying this phenomenon are far from clear. To tackle these problems, this Ph.D. research aims to study the generalization of sparse neural networks, and to propose more efficient approaches that can yield sparse neural networks with generalization bounds.
    Original languageEnglish
    Title of host publicationInternational Joint Conferences on Artifical Intelligence (IJCAI), 2020
    Number of pages2
    Publication statusAccepted/In press - 20 Apr 2020
    Event29th International Joint Conference on Artificial Intelligence - 17th Pacific Rim International Conference on Artificial Intelligence. - Pacifico Convention Plaza Yokohama, Yokohama, Japan
    Duration: 11 Jul 202017 Jul 2020
    Conference number: 29
    https://ijcai20.org/

    Conference

    Conference29th International Joint Conference on Artificial Intelligence - 17th Pacific Rim International Conference on Artificial Intelligence.
    Abbreviated titleIJCAI-PRICAI 2020
    Country/TerritoryJapan
    CityYokohama
    Period11/07/2017/07/20
    Internet address

    Keywords

    • Sparse neural networks
    • Generalization learning
    • sparse training

    Fingerprint

    Dive into the research topics of 'Learning Sparse Neural Networks for Better Generalization'. Together they form a unique fingerprint.

    Cite this