Fast algorithm selection using learning curves

J.N. Rijn, van, S.M. Abdulrahman, P. Brazdil, J. Vanschoren

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

25 Citations (Scopus)
1 Downloads (Pure)

Abstract

One of the challenges in Machine Learning to find a classifier and parameter settings that work well on a given dataset. Evaluating all possible combinations typically takes too much time, hence many solutions have been proposed that attempt to predict which classifiers are most promising to try. As the first recommended classifier is not always the correct choice, multiple recommendations should be made, making this a ranking problem rather than a classification problem. Even though this is a well studied problem, there is currently no good way of evaluating such rankings. We advocate the use of Loss Time Curves, as used in the optimization literature. These visualize the amount of budget (time) needed to converge to a acceptable solution. We also investigate a method that utilizes the measured performances of classifiers on small samples of data to make such recommendation, and adapt it so that it works well in Loss Time space. Experimental results show that this method converges extremely fast to an acceptable solution. Keywords: Algorithm selection; Meta-learning; Subsampling
Original languageEnglish
Title of host publicationAdvances in Intelligent Data Analysis XIV (14th International Symposium, IDA 2015, Saint Etienne, France, October 22-24, 2015)
EditorsE. Fromont, T. De Bie, M. Leeuwen, van
Place of PublicationDordrecht
PublisherSpringer
Pages298-309
ISBN (Print)978-3-319-24464-8
DOIs
Publication statusPublished - 2015
Event14th International Symposium on Intelligent Data Analysis (IDA 2015), October 22-24, 2015, Saint-Etienne, France - Saint-Etienne, France
Duration: 22 Oct 201524 Oct 2015
https://ida2015.univ-st-etienne.fr/

Publication series

NameLecture Notes in Computer Science
Volume9385
ISSN (Print)0302-9743

Conference

Conference14th International Symposium on Intelligent Data Analysis (IDA 2015), October 22-24, 2015, Saint-Etienne, France
Abbreviated titleIDA 2015
CountryFrance
CitySaint-Etienne
Period22/10/1524/10/15
Internet address

Fingerprint Dive into the research topics of 'Fast algorithm selection using learning curves'. Together they form a unique fingerprint.

Cite this