Abstract
Human-computer interaction, particularly the graphical user interface (GUI) for displaying detection results, is an important but underexplored aspect of CADe systems in endoscopy. The aim of this study was to study if the use of a bounding box GUI or a heatmap GUI results in different performance of endoscopists when using CADe for Barrett’s neoplasia.
Methods
Thirty-seven endoscopists from six countries with varying expertise assessed 70 Barrett’s esophagus videos. All videos were evaluated by our previously developed CADe system and comprised, at some point, a CADe detection, irrespective if the video contained neoplasia or not. The study had two phases: initially, videos were shown with either a bounding box or heatmap; after a two-week wash-out, the same videos were reordered and displayed with the alternate GUI. Endoscopists marked perceived neoplastic lesions and biopsy sites, also noting their subjective GUI preference. Primary endpoints were objective classification and localization performance. Secondary endpoint was the subjective preference.
Results
There was no statistically significant difference in classification performance when endoscopists were provided with the bounding box or the heatmap visualization (sensitivity 83% vs. 83%, p = 0.29; specificity 86% vs 86%, p =0.09). Also, the comparison of localization accuracy between the bounding box and heatmap methods showed no significant differences, with both methods yielding a median score of 97%. Subjectively, 23 endoscopists preferred the heatmap and 14 the bounding box (p=0.04).
Conclusions
Although endoscopists expressed a preference for the heatmap GUI, this was not associated with a statistical difference in performance outcomes.
| Original language | English |
|---|---|
| Journal | Gastrointestinal Endoscopy |
| Volume | XX |
| Issue number | X |
| DOIs | |
| Publication status | E-pub ahead of print - 28 Feb 2025 |
Funding
We thank the members and collaborators of the BONS-AI Consortium: Alaa Alkhalaf, Isala Hospital, Zwolle, the Netherlands; Lorenza Alvarez Herrero, St. Antonius Hospital, Utrecht, the Netherlands; Francisco Baldaque-Silva, Karolinska University Hospital, Stockholm, Sweden; Maximilien Barret, Cochin Hospital, Paris, France; Jacques J. Bergman, Amsterdam UMC, Amsterdam, the Netherlands; Torsten Beyna, Evangelisches Krankenhaus D\u00FCsseldorf, D\u00FCsseldorf, Germany; Raf Bisschops, University Hospitals Leuven, Leuven, Belgium; Tim G. Boers, TU Eindhoven, Eindhoven, the Netherlands; Wouter Curvers, Catharina Hospital, Eindhoven, the Netherlands; Pierre H. Deprez, University Hospital Saint-Luc, Brussels, Belgium; Lucas C. Duits, Amsterdam UMC, Amsterdam, the Netherlands; Peter Elbe, Karolinska University Hospital and Karolinska Institute, Stockholm, Sweden; Jose Miguel Esteban, Cl\u00EDnico San Carlos, Madrid, Spain; Gary W. Falk, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania, USA; Kiki N. Fockens, Amsterdam UMC, Amsterdam, the Netherlands; Gregory G. Ginsberg, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA; Albert J. de Groof, Amsterdam UMC, Amsterdam, the Netherlands; Rehan Haidry, Cleveland Clinic London and University College Hospitals NHS, London, UK; Martin H. Houben, HagaZiekenhuis, The Hague, the Netherlands; Anthony Infantolino, Thomas Jefferson University Hospital, Philadelphia, Pennsylvania, USA; Prasad G. Iyer, Mayo Clinic, Rochester, Minnesota, USA; Martijn Jong, Amsterdam UMC, Amsterdam, the Netherlands; Pieter-Jan de Jonge, Erasmus MC, Rotterdam, the Netherlands; Jelmer B. Jukema, Amsterdam UMC, Amsterdam, the Netherlands; Arjun Koch, Erasmus MC, Rotterdam, the Netherlands; Srinadh Komanduri, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA; Vani Konda, Baylor University Medical Center, Dallas, Texas, USA; Koen C. Kusters, TU Eindhoven, Eindhoven, the Netherlands; Philippe Leclerq, University Hospitals Leuven, Leuven, Belgium; Cadman L. Leggett, Mayo Clinic, Rochester, Minnesota, USA; Charles J. Lightdale, Columbia University Medical Center, New York, New York, USA; Rosalie C. Mallant-Hent, Flevoziekenhuis, Almere, the Netherlands; Guiomar Moral Villarejo, Nottingham University Hospitals NHS Trust, Nottingham, UK; V. Raman Muthusamy, UCLA Health, Los Angeles, California, USA; Jacobo Ortiz Fern\u00E1ndez-Sordo, Nottingham University Hospitals NHS Trust, Nottingham, UK Oliver Pech, St John of God Hospital, Regensburg, Germany; Ian Penman, Royal Infirmary Edinburgh, Edinburgh, UK; Roos E. Pouw, Amsterdam UMC, Amsterdam, the Netherlands; Joost A. van der Putten, TU Eindhoven, Eindhoven, the Netherlands; Krish Ragunath, Royal Perth Hospital, Perth, Australia; Pieter Scholten, Onze Lieve Vrouwe Gasthuis, Amsterdam, the Netherlands; Stefan Seewald, Klinik Hirslanden, Zurich, Switzerland; Amritha Sethi, Columbia University Medical Center, New York, New York, USA; Michael S. Smith, Mount Sinai West and Mount Sinai Morningside Hospitals, New York, New York, USA; Fons van der Sommen, TU Eindhoven, Eindhoven, the Netherlands; Arvind Trindade, Zucker School of Medicine at Hofstra/Northwell, Long Island Jewish Medical Center, New Hyde Park, New York, New York, USA; Sachin Wani, University of Colorado Anschutz Medical Center, Aurora, Colorado, USA; Irving Waxman, Rush University Medical Center, Chicago, Illinois, USA; Jessie Westerhof, UMC Groningen, Groningen, the Netherlands; Bas L. Weusten, UMC Utrecht and St Antonius Hospital, Utrecht, the Netherlands; Peter H. de With, TU Eindhoven, Eindhoven, the Netherlands; Herbert C. Wolfsen, Mayo Clinic, Jacksonville, Florida, USA.