An open source AutoML benchmark

Pieter Gijsbers, Erin LeDell, Sébastien Poirier, Janek Thomas, Bernd Bischl, Joaquin Vanschoren

Research output: Contribution to conference › Paper › Academic

12 Downloads (Pure)

Abstract

In recent years, an active field of research has developed around automated machine learning(AutoML). Unfortunately, comparing different AutoML systems is hard and often doneincorrectly. We introduce an open, ongoing, and extensible benchmark framework whichfollows best practices and avoids common mistakes. The framework is open-source, usespublic datasets and has a website with up-to-date results. We use the framework to conducta thorough comparison of 4 AutoML systems across 39 datasets and analyze the results.
Original languageEnglish
Number of pages8
Publication statusPublished - Jun 2019
Event6th ICML Workshop on Automated Machine Learning - Long Beach, United States
Duration: 14 Jun 2019 → 14 Jun 2019
https://sites.google.com/view/automl2019icml/home

Workshop

Workshop6th ICML Workshop on Automated Machine Learning
Abbreviated titleAutoML@ICML2019
CountryUnited States
CityLong Beach
Period14/06/19 → 14/06/19
Internet address

Fingerprint

Learning systems
Websites

Keywords

  • AutoML
  • benchmark
  • open source

Cite this

Gijsbers, P., LeDell, E., Poirier, S., Thomas, J., Bischl, B., & Vanschoren, J. (2019). An open source AutoML benchmark. Paper presented at 6th ICML Workshop on Automated Machine Learning, Long Beach, United States.
Gijsbers, Pieter ; LeDell, Erin ; Poirier, Sébastien ; Thomas, Janek ; Bischl, Bernd ; Vanschoren, Joaquin. / An open source AutoML benchmark. Paper presented at 6th ICML Workshop on Automated Machine Learning, Long Beach, United States.8 p.
@conference{4ae49c3aade448858065f3a0e1bd3cee,
title = "An open source AutoML benchmark",
abstract = "In recent years, an active field of research has developed around automated machine learning(AutoML). Unfortunately, comparing different AutoML systems is hard and often doneincorrectly. We introduce an open, ongoing, and extensible benchmark framework whichfollows best practices and avoids common mistakes. The framework is open-source, usespublic datasets and has a website with up-to-date results. We use the framework to conducta thorough comparison of 4 AutoML systems across 39 datasets and analyze the results.",
keywords = "AutoML, benchmark, open source",
author = "Pieter Gijsbers and Erin LeDell and S{\'e}bastien Poirier and Janek Thomas and Bernd Bischl and Joaquin Vanschoren",
year = "2019",
month = "6",
language = "English",
note = "6th ICML Workshop on Automated Machine Learning, AutoML@ICML2019 ; Conference date: 14-06-2019 Through 14-06-2019",
url = "https://sites.google.com/view/automl2019icml/home",

}

Gijsbers, P, LeDell, E, Poirier, S, Thomas, J, Bischl, B & Vanschoren, J 2019, 'An open source AutoML benchmark' Paper presented at 6th ICML Workshop on Automated Machine Learning, Long Beach, United States, 14/06/19 - 14/06/19, .

An open source AutoML benchmark. / Gijsbers, Pieter; LeDell, Erin; Poirier, Sébastien; Thomas, Janek; Bischl, Bernd; Vanschoren, Joaquin.

2019. Paper presented at 6th ICML Workshop on Automated Machine Learning, Long Beach, United States.

Research output: Contribution to conference › Paper › Academic

TY - CONF

T1 - An open source AutoML benchmark

AU - Gijsbers, Pieter

AU - LeDell, Erin

AU - Poirier, Sébastien

AU - Thomas, Janek

AU - Bischl, Bernd

AU - Vanschoren, Joaquin

PY - 2019/6

Y1 - 2019/6

N2 - In recent years, an active field of research has developed around automated machine learning(AutoML). Unfortunately, comparing different AutoML systems is hard and often doneincorrectly. We introduce an open, ongoing, and extensible benchmark framework whichfollows best practices and avoids common mistakes. The framework is open-source, usespublic datasets and has a website with up-to-date results. We use the framework to conducta thorough comparison of 4 AutoML systems across 39 datasets and analyze the results.

AB - In recent years, an active field of research has developed around automated machine learning(AutoML). Unfortunately, comparing different AutoML systems is hard and often doneincorrectly. We introduce an open, ongoing, and extensible benchmark framework whichfollows best practices and avoids common mistakes. The framework is open-source, usespublic datasets and has a website with up-to-date results. We use the framework to conducta thorough comparison of 4 AutoML systems across 39 datasets and analyze the results.

KW - AutoML

KW - benchmark

KW - open source

M3 - Paper

ER -

Gijsbers P, LeDell E, Poirier S, Thomas J, Bischl B, Vanschoren J. An open source AutoML benchmark. 2019. Paper presented at 6th ICML Workshop on Automated Machine Learning, Long Beach, United States.

Access to Document