Colorectal polyp classification using confidence-calibrated convolutional neural networks

Koen Kusters; Thom Scheeve; Nikoo Dehghani; Fons van der Sommen; Peter H.N. de With

doi:10.1117/12.2606801

Colorectal polyp classification using confidence-calibrated convolutional neural networks

Koen Kusters, Thom Scheeve (Supervisor), Nikoo Dehghani (Supervisor), Fons van der Sommen (Supervisor), Peter H.N. de With (Editor)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

8 Citations (Scopus)

253 Downloads (Pure)

Abstract

Computer-Aided Diagnosis (CADx) systems for in-vivo characterization of Colorectal Polyps (CRPs) which are precursor lesions of Colorectal Cancer (CRC), can assist clinicians with diagnosis and better informed decisionmaking during colonoscopy procedures. Current deep learning-based state-of-the-art solutions achieve a high classification performance, but lack measures to increase the reliability of such systems. In this paper, the reliability of a Convolutional Neural Network (CNN) for characterization of CRPs is specifically addressed by confidence calibration. Well-calibrated models produce classification-confidence scores that reflect the actual correctness likelihood of the model, thereby supporting reliable predictions by trustworthy and informative confidence scores. Two recently proposed trainable calibration methods are explored for CRP classification to calibrate the confidence of the proposed CNN. We show that the confidence-calibration error can be decreased by 33.86% (-0.01648 ± 0.01085), 48.33% (-0.04415 ± 0.01731), 50.57% (-0.11423 ± 0.00680), 61.68% (-0.01553 ± 0.00204) and 48.27% (-0.22074 ± 0.08652) for the Expected Calibration Error (ECE), Average Calibration Error (ACE), Maximum Calibration Error (MCE), Over-Confidence Error (OE) and Cumulative Calibration Error (CUMU), respectively. Moreover, the absolute difference between the average entropy and the expected entropy was considerably reduced by 32.00% (-0.04374 ± 0.01238) on average. Furthermore, even a slightly improved classification performance is observed, compared to the uncalibrated equivalent. The obtained results show that the proposed model for CRP classification with confidence calibration produces better calibrated predictions without sacrificing classification performance. This work shows promising points of engagement towards obtaining reliable and well-calibrated CADx systems for in-vivo polyp characterization, to assist clinicians during colonoscopy procedures.

Original language	English
Title of host publication	Medical Imaging 2022
Subtitle of host publication	Computer-Aided Diagnosis
Editors	Karen Drukker, Khan M. Iftekharuddin
Publisher	SPIE
Pages	442-454
Number of pages	13
ISBN (Electronic)	9781510649415
DOIs	https://doi.org/10.1117/12.2606801
Publication status	Published - 4 Apr 2022
Event	Medical Imaging 2022: Computer-Aided Diagnosis - Virtual, Online Duration: 21 Mar 2022 → 27 Mar 2022

Publication series

Name	Proceedings of SPIE
Volume	12033
ISSN (Print)	1605-7422

Conference

Conference	Medical Imaging 2022: Computer-Aided Diagnosis
City	Virtual, Online
Period	21/03/22 → 27/03/22

Bibliographical note

Funding Information:
We thankfully acknowledge the Dutch Cancer Society for funding the COMET-OPTICAL project (Project No. 12639), which guided the presented study. The Dutch Cancer Society did not contribute to the study protocol, data analysis or manuscript writing.

Publisher Copyright:
© 2022 SPIE.

Keywords

Classification
Colorectal polyps
Convolutional Neural Networks
Model Confidence Calibration

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1117/12.2606801

SPIEMed2022_Manuscript_KoenKustersFinal author version , 2.64 MB

Cite this

Kusters, K., Scheeve, T., Dehghani, N., van der Sommen, F., & de With, P. H. N. (Ed.) (2022). Colorectal polyp classification using confidence-calibrated convolutional neural networks. In K. Drukker, & K. M. Iftekharuddin (Eds.), Medical Imaging 2022: Computer-Aided Diagnosis (pp. 442-454). Article 120331P (Proceedings of SPIE; Vol. 12033). SPIE. https://doi.org/10.1117/12.2606801

@inproceedings{364511c74998420bbb8c3ef79973b119,

title = "Colorectal polyp classification using confidence-calibrated convolutional neural networks",

abstract = "Computer-Aided Diagnosis (CADx) systems for in-vivo characterization of Colorectal Polyps (CRPs) which are precursor lesions of Colorectal Cancer (CRC), can assist clinicians with diagnosis and better informed decisionmaking during colonoscopy procedures. Current deep learning-based state-of-the-art solutions achieve a high classification performance, but lack measures to increase the reliability of such systems. In this paper, the reliability of a Convolutional Neural Network (CNN) for characterization of CRPs is specifically addressed by confidence calibration. Well-calibrated models produce classification-confidence scores that reflect the actual correctness likelihood of the model, thereby supporting reliable predictions by trustworthy and informative confidence scores. Two recently proposed trainable calibration methods are explored for CRP classification to calibrate the confidence of the proposed CNN. We show that the confidence-calibration error can be decreased by 33.86% (-0.01648 ± 0.01085), 48.33% (-0.04415 ± 0.01731), 50.57% (-0.11423 ± 0.00680), 61.68% (-0.01553 ± 0.00204) and 48.27% (-0.22074 ± 0.08652) for the Expected Calibration Error (ECE), Average Calibration Error (ACE), Maximum Calibration Error (MCE), Over-Confidence Error (OE) and Cumulative Calibration Error (CUMU), respectively. Moreover, the absolute difference between the average entropy and the expected entropy was considerably reduced by 32.00% (-0.04374 ± 0.01238) on average. Furthermore, even a slightly improved classification performance is observed, compared to the uncalibrated equivalent. The obtained results show that the proposed model for CRP classification with confidence calibration produces better calibrated predictions without sacrificing classification performance. This work shows promising points of engagement towards obtaining reliable and well-calibrated CADx systems for in-vivo polyp characterization, to assist clinicians during colonoscopy procedures.",

keywords = "Classification, Colorectal polyps, Convolutional Neural Networks, Model Confidence Calibration",

author = "Koen Kusters and Thom Scheeve and Nikoo Dehghani and {van der Sommen}, Fons and {de With}, {Peter H.N.}",

note = "Funding Information: We thankfully acknowledge the Dutch Cancer Society for funding the COMET-OPTICAL project (Project No. 12639), which guided the presented study. The Dutch Cancer Society did not contribute to the study protocol, data analysis or manuscript writing. Publisher Copyright: {\textcopyright} 2022 SPIE.; Medical Imaging 2022: Computer-Aided Diagnosis ; Conference date: 21-03-2022 Through 27-03-2022",

year = "2022",

month = apr,

day = "4",

doi = "10.1117/12.2606801",

language = "English",

series = "Proceedings of SPIE",

publisher = "SPIE",

pages = "442--454",

editor = "Karen Drukker and Iftekharuddin, {Khan M.}",

booktitle = "Medical Imaging 2022",

address = "United States",

}

Kusters, K , Scheeve, T , Dehghani, N , van der Sommen, F & de With, PHN (ed.) 2022, Colorectal polyp classification using confidence-calibrated convolutional neural networks. in K Drukker & KM Iftekharuddin (eds), Medical Imaging 2022: Computer-Aided Diagnosis., 120331P, Proceedings of SPIE, vol. 12033, SPIE, pp. 442-454, Medical Imaging 2022: Computer-Aided Diagnosis, Virtual, Online, 21/03/22. https://doi.org/10.1117/12.2606801

Colorectal polyp classification using confidence-calibrated convolutional neural networks. / Kusters, Koen ; Scheeve, Thom (Supervisor); Dehghani, Nikoo (Supervisor) et al.
Medical Imaging 2022: Computer-Aided Diagnosis. ed. / Karen Drukker; Khan M. Iftekharuddin. SPIE, 2022. p. 442-454 120331P (Proceedings of SPIE; Vol. 12033).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Colorectal polyp classification using confidence-calibrated convolutional neural networks

AU - Kusters, Koen

A2 - Scheeve, Thom

A2 - Dehghani, Nikoo

A2 - van der Sommen, Fons

A2 - de With, Peter H.N.

A2 - Drukker, Karen

A2 - Iftekharuddin, Khan M.

N1 - Funding Information: We thankfully acknowledge the Dutch Cancer Society for funding the COMET-OPTICAL project (Project No. 12639), which guided the presented study. The Dutch Cancer Society did not contribute to the study protocol, data analysis or manuscript writing. Publisher Copyright: © 2022 SPIE.

PY - 2022/4/4

Y1 - 2022/4/4

N2 - Computer-Aided Diagnosis (CADx) systems for in-vivo characterization of Colorectal Polyps (CRPs) which are precursor lesions of Colorectal Cancer (CRC), can assist clinicians with diagnosis and better informed decisionmaking during colonoscopy procedures. Current deep learning-based state-of-the-art solutions achieve a high classification performance, but lack measures to increase the reliability of such systems. In this paper, the reliability of a Convolutional Neural Network (CNN) for characterization of CRPs is specifically addressed by confidence calibration. Well-calibrated models produce classification-confidence scores that reflect the actual correctness likelihood of the model, thereby supporting reliable predictions by trustworthy and informative confidence scores. Two recently proposed trainable calibration methods are explored for CRP classification to calibrate the confidence of the proposed CNN. We show that the confidence-calibration error can be decreased by 33.86% (-0.01648 ± 0.01085), 48.33% (-0.04415 ± 0.01731), 50.57% (-0.11423 ± 0.00680), 61.68% (-0.01553 ± 0.00204) and 48.27% (-0.22074 ± 0.08652) for the Expected Calibration Error (ECE), Average Calibration Error (ACE), Maximum Calibration Error (MCE), Over-Confidence Error (OE) and Cumulative Calibration Error (CUMU), respectively. Moreover, the absolute difference between the average entropy and the expected entropy was considerably reduced by 32.00% (-0.04374 ± 0.01238) on average. Furthermore, even a slightly improved classification performance is observed, compared to the uncalibrated equivalent. The obtained results show that the proposed model for CRP classification with confidence calibration produces better calibrated predictions without sacrificing classification performance. This work shows promising points of engagement towards obtaining reliable and well-calibrated CADx systems for in-vivo polyp characterization, to assist clinicians during colonoscopy procedures.

AB - Computer-Aided Diagnosis (CADx) systems for in-vivo characterization of Colorectal Polyps (CRPs) which are precursor lesions of Colorectal Cancer (CRC), can assist clinicians with diagnosis and better informed decisionmaking during colonoscopy procedures. Current deep learning-based state-of-the-art solutions achieve a high classification performance, but lack measures to increase the reliability of such systems. In this paper, the reliability of a Convolutional Neural Network (CNN) for characterization of CRPs is specifically addressed by confidence calibration. Well-calibrated models produce classification-confidence scores that reflect the actual correctness likelihood of the model, thereby supporting reliable predictions by trustworthy and informative confidence scores. Two recently proposed trainable calibration methods are explored for CRP classification to calibrate the confidence of the proposed CNN. We show that the confidence-calibration error can be decreased by 33.86% (-0.01648 ± 0.01085), 48.33% (-0.04415 ± 0.01731), 50.57% (-0.11423 ± 0.00680), 61.68% (-0.01553 ± 0.00204) and 48.27% (-0.22074 ± 0.08652) for the Expected Calibration Error (ECE), Average Calibration Error (ACE), Maximum Calibration Error (MCE), Over-Confidence Error (OE) and Cumulative Calibration Error (CUMU), respectively. Moreover, the absolute difference between the average entropy and the expected entropy was considerably reduced by 32.00% (-0.04374 ± 0.01238) on average. Furthermore, even a slightly improved classification performance is observed, compared to the uncalibrated equivalent. The obtained results show that the proposed model for CRP classification with confidence calibration produces better calibrated predictions without sacrificing classification performance. This work shows promising points of engagement towards obtaining reliable and well-calibrated CADx systems for in-vivo polyp characterization, to assist clinicians during colonoscopy procedures.

KW - Classification

KW - Colorectal polyps

KW - Convolutional Neural Networks

KW - Model Confidence Calibration

UR - http://www.scopus.com/inward/record.url?scp=85132843042&partnerID=8YFLogxK

U2 - 10.1117/12.2606801

DO - 10.1117/12.2606801

M3 - Conference contribution

AN - SCOPUS:85132843042

T3 - Proceedings of SPIE

SP - 442

EP - 454

BT - Medical Imaging 2022

PB - SPIE

T2 - Medical Imaging 2022: Computer-Aided Diagnosis

Y2 - 21 March 2022 through 27 March 2022

ER -

Colorectal polyp classification using confidence-calibrated convolutional neural networks

Abstract

Publication series

Conference

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this