How to measure uncertainty in uncertainty sampling for active learning

Vu-Linh Nguyen, Mohammad Hossein Shaker, Eyke Hüllermeier (Corresponding author)

Research output: Contribution to journalArticleAcademicpeer-review

38 Citations (Scopus)
444 Downloads (Pure)


Various strategies for active learning have been proposed in the machine learning literature. In uncertainty sampling, which is among the most popular approaches, the active learner sequentially queries the label of those instances for which its current prediction is maximally uncertain. The predictions as well as the measures used to quantify the degree of uncertainty, such as entropy, are traditionally of a probabilistic nature. Yet, alternative approaches to capturing uncertainty in machine learning, alongside with corresponding uncertainty measures, have been proposed in recent years. In particular, some of these measures seek to distinguish different sources and to separate different types of uncertainty, such as the reducible (epistemic) and the irreducible (aleatoric) part of the total uncertainty in a prediction. The goal of this paper is to elaborate on the usefulness of such measures for uncertainty sampling, and to compare their performance in active learning. To this end, we instantiate uncertainty sampling with different measures, analyze the properties of the sampling strategies thus obtained, and compare them in an experimental study.
Original languageEnglish
Pages (from-to)89-122
Number of pages34
JournalMachine Learning
Publication statusPublished - Jan 2022


  • Active learning
  • Aleatoric uncertainty
  • Credal uncertainty
  • Epistemic uncertainty
  • Uncertainty sampling


Dive into the research topics of 'How to measure uncertainty in uncertainty sampling for active learning'. Together they form a unique fingerprint.

Cite this