How to exploit weaknesses in biomedical challenge design and organization

Annika Reinke, Matthias Eisenmann, Sinan Onogur, Marko Stankovic, Patrick Scholz, Peter M. Full, Hrvoje Bogunovic, Bennett A. Landman, Oskar Maier, Bjoern H. Menze, Gregory C. Sharp, Korsuk Sirinukunwattana, Stefanie Speidel, Fons van der Sommen, Guoyan Zheng, Henning Müller, Michal Kozubek, Tal Arbel, Andrew P. Bradley, Pierre JanninAnnette Kopp-Schneider, Lena Maier-hein

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

10 Citations (Scopus)

Abstract

Since the first MICCAI grand challenge organized in 2007 in Brisbane, challenges have become an integral part of MICCAI conferences. In the meantime, challenge datasets have become widely recognized as international benchmarking datasets and thus have a great influence on the research community and individual careers. In this paper, we show several ways in which weaknesses related to current challenge design and organization can potentially be exploited. Our experimental analysis, based on MICCAI segmentation challenges organized in 2015, demonstrates that both challenge organizers and participants can potentially undertake measures to substantially tune rankings. To overcome these problems we present best practice recommendations for improving challenge design and organization.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings
EditorsAlejandro F. Frangi, Gabor Fichtinger, Julia A. Schnabel, Carlos Alberola-López, Christos Davatzikos
Place of PublicationBerlin
PublisherSpringer
Chapter45
Pages388-395
Number of pages8
ISBN (Electronic)978-3-030-00937-3
ISBN (Print)978-3-030-00936-6
DOIs
Publication statusPublished - 13 Sep 2018
Event21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018 - Granada, Spain
Duration: 16 Sep 201820 Sep 2018

Publication series

NameLecture Notes in Computer Science
Volume11073
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018
CountrySpain
CityGranada
Period16/09/1820/09/18

Fingerprint Dive into the research topics of 'How to exploit weaknesses in biomedical challenge design and organization'. Together they form a unique fingerprint.

Cite this