Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints

Laurens Bliek; Arthur Guijt; Sicco Verwer; Mathijs de Weerdt

doi:10.1145/3449726.3463136

Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints

Laurens Bliek, Arthur Guijt, Sicco Verwer, Mathijs de Weerdt

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

17 Citations (Scopus)

Abstract

A challenging problem in both engineering and computer science is that of minimising a function for which we have no mathematical formulation available, that is expensive to evaluate, and that contains continuous and integer variables, for example in automatic algorithm configuration. Surrogate-based algorithms are very suitable for this type of problem, but most existing techniques are designed with only continuous or only discrete variables in mind. Mixed-Variable ReLU-based Surrogate Modelling (MVRSM) is a surrogate-based algorithm that uses a linear combination of rectified linear units, defined in such a way that (local) optima satisfy the integer constraints. Unlike other methods, it also has a constant run-time per iteration. This method outperforms the state of the art on several synthetic benchmarks with up to 238 continuous and integer variables, and achieves competitive performance on two real-life benchmarks: XG-Boost hyperparameter tuning and Electrostatic Precipitator optimisation.

Original language	English
Title of host publication	GECCO'21
Subtitle of host publication	Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion
Publisher	Association for Computing Machinery, Inc.
Pages	1851-1859
Number of pages	9
ISBN (Electronic)	9781450383516
DOIs	https://doi.org/10.1145/3449726.3463136
Publication status	Published - 7 Jul 2021
Event	2021 Genetic and Evolutionary Computation Conference, GECCO 2021 - Virtual/Online, Lille, France Duration: 10 Jul 2021 → 14 Jul 2021 https://gecco-2021.sigevo.org/HomePage

Conference

Conference	2021 Genetic and Evolutionary Computation Conference, GECCO 2021
Abbreviated title	GECCO 2021
Country/Territory	France
City	Lille
Period	10/07/21 → 14/07/21
Internet address	https://gecco-2021.sigevo.org/HomePage

Funding

This work is part of the research programme Real-time data-driven maintenance logistics with project number 628.009.012, which is financed by the Dutch Research Council (NWO). The authors thank Erik Daxberger for providing the code for generating one of MiVaBO’s synthetic test functions, Frederik Rehbach for providing information on the ESP problem, and anonymous reviewers of an earlier version of this paper for providing constructive feedback.

Keywords

Bayesian optimisation
expensive optimisation
mixed-variable optimisation
surrogate models

Access to Document

10.1145/3449726.3463136

Cite this

@inproceedings{fa5ab869a71d4ae48cbfae246670fe1b,

title = "Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints",

abstract = "A challenging problem in both engineering and computer science is that of minimising a function for which we have no mathematical formulation available, that is expensive to evaluate, and that contains continuous and integer variables, for example in automatic algorithm configuration. Surrogate-based algorithms are very suitable for this type of problem, but most existing techniques are designed with only continuous or only discrete variables in mind. Mixed-Variable ReLU-based Surrogate Modelling (MVRSM) is a surrogate-based algorithm that uses a linear combination of rectified linear units, defined in such a way that (local) optima satisfy the integer constraints. Unlike other methods, it also has a constant run-time per iteration. This method outperforms the state of the art on several synthetic benchmarks with up to 238 continuous and integer variables, and achieves competitive performance on two real-life benchmarks: XG-Boost hyperparameter tuning and Electrostatic Precipitator optimisation. ",

keywords = "Bayesian optimisation, expensive optimisation, mixed-variable optimisation, surrogate models",

author = "Laurens Bliek and Arthur Guijt and Sicco Verwer and {de Weerdt}, Mathijs",

note = "Publisher Copyright: {\textcopyright} 2021 Owner/Author.; 2021 Genetic and Evolutionary Computation Conference, GECCO 2021 ; Conference date: 10-07-2021 Through 14-07-2021",

year = "2021",

month = jul,

day = "7",

doi = "10.1145/3449726.3463136",

language = "English",

pages = "1851--1859",

booktitle = "GECCO'21",

publisher = "Association for Computing Machinery, Inc.",

address = "United States",

url = "https://gecco-2021.sigevo.org/HomePage",

}

Bliek, L, Guijt, A, Verwer, S & de Weerdt, M 2021, Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints. in GECCO'21 : Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc., pp. 1851-1859, 2021 Genetic and Evolutionary Computation Conference, GECCO 2021, Lille, France, 10/07/21. https://doi.org/10.1145/3449726.3463136

Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints. / Bliek, Laurens; Guijt, Arthur; Verwer, Sicco et al.
GECCO'21 : Proceedings of the 2021 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc., 2021. p. 1851-1859.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints

AU - Bliek, Laurens

AU - Guijt, Arthur

AU - Verwer, Sicco

AU - de Weerdt, Mathijs

PY - 2021/7/7

Y1 - 2021/7/7

N2 - A challenging problem in both engineering and computer science is that of minimising a function for which we have no mathematical formulation available, that is expensive to evaluate, and that contains continuous and integer variables, for example in automatic algorithm configuration. Surrogate-based algorithms are very suitable for this type of problem, but most existing techniques are designed with only continuous or only discrete variables in mind. Mixed-Variable ReLU-based Surrogate Modelling (MVRSM) is a surrogate-based algorithm that uses a linear combination of rectified linear units, defined in such a way that (local) optima satisfy the integer constraints. Unlike other methods, it also has a constant run-time per iteration. This method outperforms the state of the art on several synthetic benchmarks with up to 238 continuous and integer variables, and achieves competitive performance on two real-life benchmarks: XG-Boost hyperparameter tuning and Electrostatic Precipitator optimisation.

AB - A challenging problem in both engineering and computer science is that of minimising a function for which we have no mathematical formulation available, that is expensive to evaluate, and that contains continuous and integer variables, for example in automatic algorithm configuration. Surrogate-based algorithms are very suitable for this type of problem, but most existing techniques are designed with only continuous or only discrete variables in mind. Mixed-Variable ReLU-based Surrogate Modelling (MVRSM) is a surrogate-based algorithm that uses a linear combination of rectified linear units, defined in such a way that (local) optima satisfy the integer constraints. Unlike other methods, it also has a constant run-time per iteration. This method outperforms the state of the art on several synthetic benchmarks with up to 238 continuous and integer variables, and achieves competitive performance on two real-life benchmarks: XG-Boost hyperparameter tuning and Electrostatic Precipitator optimisation.

KW - Bayesian optimisation

KW - expensive optimisation

KW - mixed-variable optimisation

KW - surrogate models

UR - http://www.scopus.com/inward/record.url?scp=85111000882&partnerID=8YFLogxK

U2 - 10.1145/3449726.3463136

DO - 10.1145/3449726.3463136

M3 - Conference contribution

AN - SCOPUS:85111000882

SP - 1851

EP - 1859

BT - GECCO'21

PB - Association for Computing Machinery, Inc.

T2 - 2021 Genetic and Evolutionary Computation Conference, GECCO 2021

Y2 - 10 July 2021 through 14 July 2021

ER -

Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints

Abstract

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this