Abstract
One of the challenges in Machine Learning to find a classifier and parameter settings that work well on a given dataset. Evaluating all possible combinations typically takes too much time, hence many solutions have been proposed that attempt to predict which classifiers are most promising to try. As the first recommended classifier is not always the correct choice, multiple recommendations should be made, making this a ranking problem rather than a classification problem. Even though this is a well studied problem, there is currently no good way of evaluating such rankings. We advocate the use of Loss Time Curves, as used in the optimization literature. These visualize the amount of budget (time) needed to converge to a acceptable solution. We also investigate a method that utilizes the measured performances of classifiers on small samples of data to make such recommendation, and adapt it so that it works well in Loss Time space. Experimental results show that this method converges extremely fast to an acceptable solution.
Keywords: Algorithm selection; Meta-learning; Subsampling
Original language | English |
---|---|
Title of host publication | Advances in Intelligent Data Analysis XIV (14th International Symposium, IDA 2015, Saint Etienne, France, October 22-24, 2015) |
Editors | E. Fromont, T. De Bie, M. Leeuwen, van |
Place of Publication | Dordrecht |
Publisher | Springer |
Pages | 298-309 |
ISBN (Print) | 978-3-319-24464-8 |
DOIs | |
Publication status | Published - 2015 |
Event | 14th International Symposium on Intelligent Data Analysis (IDA 2015), October 22-24, 2015, Saint-Etienne, France - Saint-Etienne, France Duration: 22 Oct 2015 → 24 Oct 2015 https://ida2015.univ-st-etienne.fr/ |
Publication series
Name | Lecture Notes in Computer Science |
---|---|
Volume | 9385 |
ISSN (Print) | 0302-9743 |
Conference
Conference | 14th International Symposium on Intelligent Data Analysis (IDA 2015), October 22-24, 2015, Saint-Etienne, France |
---|---|
Abbreviated title | IDA 2015 |
Country/Territory | France |
City | Saint-Etienne |
Period | 22/10/15 → 24/10/15 |
Internet address |