A survey of crowdsourcing in medical image analysis

Silas Ørting, Andrew Doyle, Matthias Hirth, Arno van Hilten, Oana Inel, Christopher R. Madan, Panagiotis Mavridis, Helen Spiers, Veronika Cheplygina

Research output: Contribution to journalArticleAcademic

Abstract

Rapid advances in image processing capabilities have been seen across many domains, fostered by the application of machine learning algorithms to "big-data". However, within the realm of medical image analysis, advances have been curtailed, in part, due to the limited availability of large-scale, well-annotated datasets. One of the main reasons for this is the high cost often associated with producing large amounts of high-quality meta-data. Recently, there has been growing interest in the application of crowdsourcing for this purpose; a technique that has proven effective for creating large-scale datasets across a range of disciplines, from computer vision to astrophysics. Despite the growing popularity of this approach, there has not yet been a comprehensive literature review to provide guidance to researchers considering using crowdsourcing methodologies in their own medical imaging analysis. In this survey, we review studies applying crowdsourcing to the analysis of medical images, published prior to July 2018. We identify common approaches, challenges and considerations, providing guidance of utility to researchers adopting this approach. Finally, we discuss future opportunities for development within this emerging domain.
LanguageEnglish
Article number1902.09159
Number of pages12
JournalarXiv
StatePublished - 25 Feb 2019

Fingerprint

Image analysis
Astrophysics
Medical imaging
Metadata
Learning algorithms
Computer vision
Learning systems
Image processing
Availability
Costs
Big data

Bibliographical note

While this paper is a preprint, we welcome feedback from other researchers, which we will aim to incorporate in the journal version. Interested researchers can submit comments via https://goo.gl/forms/Qzr2yAJQjOnRCAF23

Keywords

    Cite this

    Ørting, S., Doyle, A., Hirth, M., van Hilten, A., Inel, O., Madan, C. R., ... Cheplygina, V. (2019). A survey of crowdsourcing in medical image analysis. arXiv, [1902.09159].
    Ørting, Silas ; Doyle, Andrew ; Hirth, Matthias ; van Hilten, Arno ; Inel, Oana ; Madan, Christopher R. ; Mavridis, Panagiotis ; Spiers, Helen ; Cheplygina, Veronika. / A survey of crowdsourcing in medical image analysis. In: arXiv. 2019
    @article{201c35c837394da8acf9ab4df8b9a062,
    title = "A survey of crowdsourcing in medical image analysis",
    abstract = "Rapid advances in image processing capabilities have been seen across many domains, fostered by the application of machine learning algorithms to {"}big-data{"}. However, within the realm of medical image analysis, advances have been curtailed, in part, due to the limited availability of large-scale, well-annotated datasets. One of the main reasons for this is the high cost often associated with producing large amounts of high-quality meta-data. Recently, there has been growing interest in the application of crowdsourcing for this purpose; a technique that has proven effective for creating large-scale datasets across a range of disciplines, from computer vision to astrophysics. Despite the growing popularity of this approach, there has not yet been a comprehensive literature review to provide guidance to researchers considering using crowdsourcing methodologies in their own medical imaging analysis. In this survey, we review studies applying crowdsourcing to the analysis of medical images, published prior to July 2018. We identify common approaches, challenges and considerations, providing guidance of utility to researchers adopting this approach. Finally, we discuss future opportunities for development within this emerging domain.",
    keywords = "cs.CV, cs.HC",
    author = "Silas {\O}rting and Andrew Doyle and Matthias Hirth and {van Hilten}, Arno and Oana Inel and Madan, {Christopher R.} and Panagiotis Mavridis and Helen Spiers and Veronika Cheplygina",
    note = "While this paper is a preprint, we welcome feedback from other researchers, which we will aim to incorporate in the journal version. Interested researchers can submit comments via https://goo.gl/forms/Qzr2yAJQjOnRCAF23",
    year = "2019",
    month = "2",
    day = "25",
    language = "English",
    journal = "arXiv",
    publisher = "Cornell University Library",

    }

    Ørting, S, Doyle, A, Hirth, M, van Hilten, A, Inel, O, Madan, CR, Mavridis, P, Spiers, H & Cheplygina, V 2019, 'A survey of crowdsourcing in medical image analysis' arXiv.

    A survey of crowdsourcing in medical image analysis. / Ørting, Silas; Doyle, Andrew; Hirth, Matthias; van Hilten, Arno; Inel, Oana; Madan, Christopher R.; Mavridis, Panagiotis; Spiers, Helen; Cheplygina, Veronika.

    In: arXiv, 25.02.2019.

    Research output: Contribution to journalArticleAcademic

    TY - JOUR

    T1 - A survey of crowdsourcing in medical image analysis

    AU - Ørting,Silas

    AU - Doyle,Andrew

    AU - Hirth,Matthias

    AU - van Hilten,Arno

    AU - Inel,Oana

    AU - Madan,Christopher R.

    AU - Mavridis,Panagiotis

    AU - Spiers,Helen

    AU - Cheplygina,Veronika

    N1 - While this paper is a preprint, we welcome feedback from other researchers, which we will aim to incorporate in the journal version. Interested researchers can submit comments via https://goo.gl/forms/Qzr2yAJQjOnRCAF23

    PY - 2019/2/25

    Y1 - 2019/2/25

    N2 - Rapid advances in image processing capabilities have been seen across many domains, fostered by the application of machine learning algorithms to "big-data". However, within the realm of medical image analysis, advances have been curtailed, in part, due to the limited availability of large-scale, well-annotated datasets. One of the main reasons for this is the high cost often associated with producing large amounts of high-quality meta-data. Recently, there has been growing interest in the application of crowdsourcing for this purpose; a technique that has proven effective for creating large-scale datasets across a range of disciplines, from computer vision to astrophysics. Despite the growing popularity of this approach, there has not yet been a comprehensive literature review to provide guidance to researchers considering using crowdsourcing methodologies in their own medical imaging analysis. In this survey, we review studies applying crowdsourcing to the analysis of medical images, published prior to July 2018. We identify common approaches, challenges and considerations, providing guidance of utility to researchers adopting this approach. Finally, we discuss future opportunities for development within this emerging domain.

    AB - Rapid advances in image processing capabilities have been seen across many domains, fostered by the application of machine learning algorithms to "big-data". However, within the realm of medical image analysis, advances have been curtailed, in part, due to the limited availability of large-scale, well-annotated datasets. One of the main reasons for this is the high cost often associated with producing large amounts of high-quality meta-data. Recently, there has been growing interest in the application of crowdsourcing for this purpose; a technique that has proven effective for creating large-scale datasets across a range of disciplines, from computer vision to astrophysics. Despite the growing popularity of this approach, there has not yet been a comprehensive literature review to provide guidance to researchers considering using crowdsourcing methodologies in their own medical imaging analysis. In this survey, we review studies applying crowdsourcing to the analysis of medical images, published prior to July 2018. We identify common approaches, challenges and considerations, providing guidance of utility to researchers adopting this approach. Finally, we discuss future opportunities for development within this emerging domain.

    KW - cs.CV

    KW - cs.HC

    M3 - Article

    JO - arXiv

    T2 - arXiv

    JF - arXiv

    M1 - 1902.09159

    ER -

    Ørting S, Doyle A, Hirth M, van Hilten A, Inel O, Madan CR et al. A survey of crowdsourcing in medical image analysis. arXiv. 2019 Feb 25. 1902.09159.