Online function minimization with convex random relu expansions

Laurens Bliek, Michel Verhaegen, Sander Wahls

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

3 Citations (Scopus)

Abstract

We propose CDONE, a convex version of the DONE algorithm. DONE is a derivative-free online optimization algorithm that uses surrogate modeling with noisy measurements to find a minimum of objective functions that are expensive to evaluate. Inspired by their success in deep learning, CDONE makes use of rectified linear units, together with a nonnegativity constraint to enforce convexity of the surrogate model. This leads to a sparse and cheap to evaluate surrogate model of the unknown optimization objective that is still accurate and that can be minimized with convex optimization algorithms. The CDONE algorithm is demonstrated on a toy example and on the problem of hyper-parameter optimization for a deep learning example on handwritten digit classification.

Original languageEnglish
Title of host publication2017 IEEE International Workshop on Machine Learning for Signal Processing (MLSP)
EditorsNaonori Ueda, Jen-Tzung Chien, Tomoko Matsui, Jan Larsen, Shinji Watanabe
PublisherInstitute of Electrical and Electronics Engineers
Pages1-6
Number of pages6
ISBN (Electronic)978-1-5090-6341-3
DOIs
Publication statusPublished - 7 Dec 2017
Externally publishedYes
Event27th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2017 - Tokyo, Japan
Duration: 25 Sep 201728 Sep 2017
Conference number: 27

Conference

Conference27th IEEE International Workshop on Machine Learning for Signal Processing, MLSP 2017
Abbreviated titleMLSP 2017
Country/TerritoryJapan
CityTokyo
Period25/09/1728/09/17

Keywords

  • Bayesian optimization
  • Deep learning
  • Derivative-free optimization
  • Surrogate modeling

Fingerprint

Dive into the research topics of 'Online function minimization with convex random relu expansions'. Together they form a unique fingerprint.

Cite this