Identifying the best machine learning algorithm for a given problem continues to be an active area of research. In this paper we present a new method which exploits both meta-level information acquired in past experiments and active testing, an algorithm selection strategy. Active testing attempts to iteratively identify an algorithm whose performance will most likely exceed the performance of previously tried algorithms. The novel method described in this paper uses tests on smaller data sample to rank the most promising candidates, thus optimizing the schedule of experiments to be carried out. The experimental results show that this approach leads to considerably faster algorithm selection. Keywords: Algorithm selection, Meta-learning, Active testing, Algorithm Ranking
|Title of host publication||International Workshop on Meta-Learning and Algorithm Selection (MetaSel 2015, Porto, Portugal September 7, 2015; co-located with ECMLPKDD 2015)|
|Editors||J. Vanschoren, P. Brazdil, C. Giraud-Carrier, L. Kotthoff|
|Publication status||Published - 2015|
|Name||CEUR Workshop Proceedings|
Abdulrahman, S. M., Brazdil, P., van Rijn, J. N., & Vanschoren, J. (2015). Algorithm selection via meta-learning and sample-based active testing. In J. Vanschoren, P. Brazdil, C. Giraud-Carrier, & L. Kotthoff (Eds.), International Workshop on Meta-Learning and Algorithm Selection (MetaSel 2015, Porto, Portugal September 7, 2015; co-located with ECMLPKDD 2015) (pp. 55-66). (CEUR Workshop Proceedings; Vol. 1455). CEUR-WS.org.