How to exploit weaknesses in biomedical challenge design and organization

Annika Reinke; Matthias Eisenmann; Sinan Onogur; Marko Stankovic; Patrick Scholz; Peter M. Full; Hrvoje Bogunovic; Bennett A. Landman; Oskar Maier; Bjoern H. Menze; Gregory C. Sharp; Korsuk Sirinukunwattana; Stefanie Speidel; Fons van der Sommen; Guoyan Zheng; Henning Müller; Michal Kozubek; Tal Arbel; Andrew P. Bradley; Pierre Jannin; Annette Kopp-Schneider; Lena Maier-hein

doi:10.1007/978-3-030-00937-3_45

How to exploit weaknesses in biomedical challenge design and organization

Annika Reinke, Matthias Eisenmann, Sinan Onogur, Marko Stankovic, Patrick Scholz, Peter M. Full, Hrvoje Bogunovic, Bennett A. Landman, Oskar Maier, Bjoern H. Menze, Gregory C. Sharp, Korsuk Sirinukunwattana, Stefanie Speidel, Fons van der Sommen, Guoyan Zheng, Henning Müller, Michal Kozubek, Tal Arbel, Andrew P. Bradley, Pierre JanninAnnette Kopp-Schneider, Lena Maier-hein

Video Coding & Architectures

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

26 Citaten (Scopus)

Samenvatting

Since the first MICCAI grand challenge organized in 2007 in Brisbane, challenges have become an integral part of MICCAI conferences. In the meantime, challenge datasets have become widely recognized as international benchmarking datasets and thus have a great influence on the research community and individual careers. In this paper, we show several ways in which weaknesses related to current challenge design and organization can potentially be exploited. Our experimental analysis, based on MICCAI segmentation challenges organized in 2015, demonstrates that both challenge organizers and participants can potentially undertake measures to substantially tune rankings. To overcome these problems we present best practice recommendations for improving challenge design and organization.

Originele taal-2	Engels
Titel	Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings
Redacteuren	Alejandro F. Frangi, Gabor Fichtinger, Julia A. Schnabel, Carlos Alberola-López, Christos Davatzikos
Plaats van productie	Berlin
Uitgeverij	Springer
Hoofdstuk	45
Pagina's	388-395
Aantal pagina's	8
ISBN van elektronische versie	978-3-030-00937-3
ISBN van geprinte versie	978-3-030-00936-6
DOI's	https://doi.org/10.1007/978-3-030-00937-3_45
Status	Gepubliceerd - 13 sep. 2018
Evenement	21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018 - Granada, Spanje Duur: 16 sep. 2018 → 20 sep. 2018

Publicatie series

Naam	Lecture Notes in Computer Science
Volume	11073
ISSN van geprinte versie	0302-9743
ISSN van elektronische versie	1611-3349

Congres

Congres	21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018
Land/Regio	Spanje
Stad	Granada
Periode	16/09/18 → 20/09/18

Toegang tot document

10.1007/978-3-030-00937-3_45

Andere bestanden en links

Link to publication in Scopus

Citeer dit

Reinke, A., Eisenmann, M., Onogur, S., Stankovic, M., Scholz, P., Full, P. M., Bogunovic, H., Landman, B. A., Maier, O., Menze, B. H., Sharp, G. C., Sirinukunwattana, K., Speidel, S., van der Sommen, F., Zheng, G., Müller, H., Kozubek, M., Arbel, T., Bradley, A. P., ... Maier-hein, L. (2018). How to exploit weaknesses in biomedical challenge design and organization. In A. F. Frangi, G. Fichtinger, J. A. Schnabel, C. Alberola-López, & C. Davatzikos (editors), Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings (blz. 388-395). (Lecture Notes in Computer Science; Vol. 11073). Springer. https://doi.org/10.1007/978-3-030-00937-3_45

Reinke, Annika ; Eisenmann, Matthias ; Onogur, Sinan et al. / How to exploit weaknesses in biomedical challenge design and organization. Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings. redacteur / Alejandro F. Frangi ; Gabor Fichtinger ; Julia A. Schnabel ; Carlos Alberola-López ; Christos Davatzikos. Berlin : Springer, 2018. blz. 388-395 (Lecture Notes in Computer Science).

@inproceedings{490e97900d194b85ba9c716ee405d2a7,

title = "How to exploit weaknesses in biomedical challenge design and organization",

abstract = "Since the first MICCAI grand challenge organized in 2007 in Brisbane, challenges have become an integral part of MICCAI conferences. In the meantime, challenge datasets have become widely recognized as international benchmarking datasets and thus have a great influence on the research community and individual careers. In this paper, we show several ways in which weaknesses related to current challenge design and organization can potentially be exploited. Our experimental analysis, based on MICCAI segmentation challenges organized in 2015, demonstrates that both challenge organizers and participants can potentially undertake measures to substantially tune rankings. To overcome these problems we present best practice recommendations for improving challenge design and organization.",

author = "Annika Reinke and Matthias Eisenmann and Sinan Onogur and Marko Stankovic and Patrick Scholz and Full, {Peter M.} and Hrvoje Bogunovic and Landman, {Bennett A.} and Oskar Maier and Menze, {Bjoern H.} and Sharp, {Gregory C.} and Korsuk Sirinukunwattana and Stefanie Speidel and {van der Sommen}, Fons and Guoyan Zheng and Henning M{\"u}ller and Michal Kozubek and Tal Arbel and Bradley, {Andrew P.} and Pierre Jannin and Annette Kopp-Schneider and Lena Maier-hein",

year = "2018",

month = sep,

day = "13",

doi = "10.1007/978-3-030-00937-3_45",

language = "English",

isbn = "978-3-030-00936-6",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "388--395",

editor = "Frangi, {Alejandro F.} and Gabor Fichtinger and Schnabel, {Julia A.} and Carlos Alberola-L{\'o}pez and Christos Davatzikos",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings",

note = "21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018 ; Conference date: 16-09-2018 Through 20-09-2018",

}

Reinke, A, Eisenmann, M, Onogur, S, Stankovic, M, Scholz, P, Full, PM, Bogunovic, H, Landman, BA, Maier, O, Menze, BH, Sharp, GC, Sirinukunwattana, K, Speidel, S, van der Sommen, F, Zheng, G, Müller, H, Kozubek, M, Arbel, T, Bradley, AP, Jannin, P, Kopp-Schneider, A & Maier-hein, L 2018, How to exploit weaknesses in biomedical challenge design and organization. in AF Frangi, G Fichtinger, JA Schnabel, C Alberola-López & C Davatzikos (uitgave), Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings. Lecture Notes in Computer Science, vol. 11073, Springer, Berlin, blz. 388-395, 21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018, Granada, Spanje, 16/09/18. https://doi.org/10.1007/978-3-030-00937-3_45

How to exploit weaknesses in biomedical challenge design and organization. / Reinke, Annika; Eisenmann, Matthias; Onogur, Sinan et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings. uitgave / Alejandro F. Frangi; Gabor Fichtinger; Julia A. Schnabel; Carlos Alberola-López; Christos Davatzikos. Berlin: Springer, 2018. blz. 388-395 (Lecture Notes in Computer Science; Vol. 11073).

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

TY - GEN

T1 - How to exploit weaknesses in biomedical challenge design and organization

AU - Reinke, Annika

AU - Eisenmann, Matthias

AU - Onogur, Sinan

AU - Stankovic, Marko

AU - Scholz, Patrick

AU - Full, Peter M.

AU - Bogunovic, Hrvoje

AU - Landman, Bennett A.

AU - Maier, Oskar

AU - Menze, Bjoern H.

AU - Sharp, Gregory C.

AU - Sirinukunwattana, Korsuk

AU - Speidel, Stefanie

AU - van der Sommen, Fons

AU - Zheng, Guoyan

AU - Müller, Henning

AU - Kozubek, Michal

AU - Arbel, Tal

AU - Bradley, Andrew P.

AU - Jannin, Pierre

AU - Kopp-Schneider, Annette

AU - Maier-hein, Lena

PY - 2018/9/13

Y1 - 2018/9/13

N2 - Since the first MICCAI grand challenge organized in 2007 in Brisbane, challenges have become an integral part of MICCAI conferences. In the meantime, challenge datasets have become widely recognized as international benchmarking datasets and thus have a great influence on the research community and individual careers. In this paper, we show several ways in which weaknesses related to current challenge design and organization can potentially be exploited. Our experimental analysis, based on MICCAI segmentation challenges organized in 2015, demonstrates that both challenge organizers and participants can potentially undertake measures to substantially tune rankings. To overcome these problems we present best practice recommendations for improving challenge design and organization.

AB - Since the first MICCAI grand challenge organized in 2007 in Brisbane, challenges have become an integral part of MICCAI conferences. In the meantime, challenge datasets have become widely recognized as international benchmarking datasets and thus have a great influence on the research community and individual careers. In this paper, we show several ways in which weaknesses related to current challenge design and organization can potentially be exploited. Our experimental analysis, based on MICCAI segmentation challenges organized in 2015, demonstrates that both challenge organizers and participants can potentially undertake measures to substantially tune rankings. To overcome these problems we present best practice recommendations for improving challenge design and organization.

UR - http://www.scopus.com/inward/record.url?scp=85053834551&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-00937-3_45

DO - 10.1007/978-3-030-00937-3_45

M3 - Conference contribution

SN - 978-3-030-00936-6

T3 - Lecture Notes in Computer Science

SP - 388

EP - 395

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings

A2 - Frangi, Alejandro F.

A2 - Fichtinger, Gabor

A2 - Schnabel, Julia A.

A2 - Alberola-López, Carlos

A2 - Davatzikos, Christos

PB - Springer

CY - Berlin

T2 - 21st International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2018

Y2 - 16 September 2018 through 20 September 2018

ER -

Reinke A, Eisenmann M, Onogur S, Stankovic M, Scholz P, Full PM et al. How to exploit weaknesses in biomedical challenge design and organization. In Frangi AF, Fichtinger G, Schnabel JA, Alberola-López C, Davatzikos C, redacteurs, Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 - 21st International Conference, 2018, Proceedings. Berlin: Springer. 2018. blz. 388-395. (Lecture Notes in Computer Science). doi: 10.1007/978-3-030-00937-3_45

How to exploit weaknesses in biomedical challenge design and organization

Samenvatting

Publicatie series

Congres

Toegang tot document

Andere bestanden en links

Vingerafdruk

Citeer dit