Diversity in random subspacing ensembles

A. Tsymbal, M. Pechenizkiy, P. Cunningham

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    6 Citations (Scopus)


    Ensembles of learnt models constitute one of the main current directions in machine learning and data mining. It was shown experimentally and theoretically that in order for an ensemble to be effective, it should consist of classifiers having diversity in their predictions. A number of ways are known to quantify diversity in ensembles, but little research has been done about their appropriateness. In this paper, we compare eight measures of the ensemble diversity with regard to their correlation with the accuracy improvement due to ensembles. We conduct experiments on 21 data sets from the UCI machine learning repository, comparing the correlations for random subspacing ensembles with different ensemble sizes and with six different ensemble integration methods. Our experiments show that the greatest correlation of the accuracy improvement, on average, is with the disagreement, entropy, and ambiguity diversity measures, and the lowest correlation, surprisingly, is with the Q and double fault measures. Normally, the correlation decreases linearly as the ensemble size increases. Much higher correlation values can be seen with the dynamic integration methods, which are shown to better utilize the ensemble diversity than their static analogues.
    Original languageEnglish
    Title of host publicationData Warehousing and Knowledge Discovery (Proceedings 6th International Conference, DaWaK'04, Zaragoza, Spain, September 1-3, 2004)
    EditorsY. Kambayashi, M.K. Mohania, W. Wöß
    Place of PublicationBerlin
    ISBN (Print)3-540-22937-X
    Publication statusPublished - 2004

    Publication series

    NameLecture Notes in Computer Science
    ISSN (Print)0302-9743


    Dive into the research topics of 'Diversity in random subspacing ensembles'. Together they form a unique fingerprint.

    Cite this