A historical perspective of biomedical explainable AI research

Luca Malinverno, Vesna Barros (Corresponding author), Francesco Ghisoni, Giovanni Visonà, Roman Kern, Philip J. Nickel, Barbara Elvira Ventura, Ilija Šimić, Sarah Stryeck, Francesca Manni, Cesar Ferri, Claire Jean-Quartier, Laura Genga, Gabriele Schweikert, Mario Lovrić, Michal Rosen-Zvi

Research output: Contribution to journalReview articlepeer-review

13 Citations (Scopus)
60 Downloads (Pure)

Abstract

The black-box nature of most artificial intelligence (AI) models encourages the development of explainability methods to engender trust into the AI decision-making process. Such methods can be broadly categorized into two main types: post hoc explanations and inherently interpretable algorithms. We aimed at analyzing the possible associations between COVID-19 and the push of explainable AI (XAI) to the forefront of biomedical research. We automatically extracted from the PubMed database biomedical XAI studies related to concepts of causality or explainability and manually labeled 1,603 papers with respect to XAI categories. To compare the trends pre- and post-COVID-19, we fit a change point detection model and evaluated significant changes in publication rates. We show that the advent of COVID-19 in the beginning of 2020 could be the driving factor behind an increased focus concerning XAI, playing a crucial role in accelerating an already evolving trend. Finally, we present a discussion with future societal use and impact of XAI technologies and potential future directions for those who pursue fostering clinical trust with interpretable machine learning models.

Original languageEnglish
Article number100830
Number of pages9
JournalPatterns
Volume4
Issue number9
DOIs
Publication statusPublished - 8 Sept 2023

Bibliographical note

Funding Information:
Dr.sc. Mario Lovrić leads the laboratory for chemical and biomedical informatics at the Institute for Anthropological Research in Zagreb and teaches as an assistant professor at the Faculty of Electrical Engineering in Osijek, both in Croatia. He is also scientific director of the Horizon EDIAQI project funded by the European Commission. Mario holds a PhD in computational chemistry with a focus on toxicology and has conducted his post-doc research in Know-Center, Austria, and RegionH, Denmark. Mario has published more than 30 peer-reviewed papers.

Funding Information:
Philip J. Nickel (PhD UCLA, 2002) is an associate professor in the philosophy and e thics group at Eindhoven University of Technology. He specializes in the philosophy of trust and the ethics of disruptive technologies. Some of his research is in the domain of biomedical ethics, focusing on issues of consent and the mediation of care through technology and data. He leads the project “Mobile Support Systems for Behavior Change” funded by the Dutch Research Council’s Responsible Innovation program.

Funding Information:
M.R.-Z. and V.B. are employees of IBM Research, Haifa, Israel. F.M. is an employee of Philips Research, Eindhoven, the Netherlands. I.S. has received funding from multiple funding agencies through a collaborative funding program and declares no support from any organization for the submitted work. P.J.N. receives funding from the Dutch Research Council (DWO) for the grant “Mobile Support Systems for Behavior Change,” of which he is the principal investigator (P.I.). M.L. is funded by the EU-Commission grant no. 101057497-EDIAQI.

Funding Information:
We are extremely grateful for Prof. Chris Holmes for his critical reading and valuable comments. We acknowledge the funding received from the European Union's Framework Programme for Research and Innovation Horizon 2020 (2014–2020) under the Marie Skłodowska-Curie Grant agreement no. 813533-MSCA-ITN-2018. I.S. was funded by the “DDAI” COMET Module within the COMET – Competence Centers for Excellent Technologies Programme, funded by the Austrian Federal Ministry for Transport, Innovation and Technology (BMVIT), the Austrian Federal Ministry for Digital and Economic Affairs (BMDW), the Austrian Research Promotion Agency (FFG), the Province of Styria (SFG), and partners from industry and academia. The COMET Program is managed by FFG. Finally, we acknowledge the Big Data Value Association (BDVA), Brussels, Belgium. Conceptualization of the study was conducted by M.R.-Z. Methodology was led by G.S. G.V. L.M. M.L. M.R.-Z. and V.B. and was conducted by all authors. Data curation was done by B.E.V. C.F. C.J.-Q. F.G. F.M. G.V. I.S. L.G. L.M. M.L. M.R.-Z. P.J.N. R.K. S.S. and V.B. Formal analysis was conducted by G.V. M.R.-Z. and V.B. Investigation was equally conducted by all authors. Project administration was done by M.L. and V.B. Resources and software was done by B.E.V. F.G. G.V. L.M. and V.B. Supervision was done by M.R.-Z. Validation, visualization, and writing of original draft were done by G.S. G.V. M.R.-Z. L.G. L.M. P.J.N. R.K. and V.B. Writing (review & editing) was equally done by all authors. M.R.-Z. and V.B. are employees of IBM Research, Haifa, Israel. F.M. is an employee of Philips Research, Eindhoven, the Netherlands. I.S. has received funding from multiple funding agencies through a collaborative funding program and declares no support from any organization for the submitted work. P.J.N. receives funding from the Dutch Research Council (DWO) for the grant “Mobile Support Systems for Behavior Change,” of which he is the principal investigator (P.I.). M.L. is funded by the EU-Commission grant no. 101057497-EDIAQI.

Funding Information:
We are extremely grateful for Prof. Chris Holmes for his critical reading and valuable comments. We acknowledge the funding received from the European Union’s Framework Programme for Research and Innovation Horizon 2020 (2014–2020) under the Marie Skłodowska-Curie Grant agreement no. 813533-MSCA-ITN-2018 . I.S. was funded by the “ DDAI ” COMET Module within the COMET – Competence Centers for Excellent Technologies Programme , funded by the Austrian Federal Ministry for Transport , Innovation and Technology ( BMVIT ), the Austrian Federal Ministry for Digital and Economic Affairs ( BMDW ), the Austrian Research Promotion Agency (FFG), the Province of Styria (SFG), and partners from industry and academia. The COMET Program is managed by FFG. Finally, we acknowledge the Big Data Value Association (BDVA), Brussels, Belgium.

Publisher Copyright:
© 2023 The Authors

Funding

Dr.sc. Mario Lovrić leads the laboratory for chemical and biomedical informatics at the Institute for Anthropological Research in Zagreb and teaches as an assistant professor at the Faculty of Electrical Engineering in Osijek, both in Croatia. He is also scientific director of the Horizon EDIAQI project funded by the European Commission. Mario holds a PhD in computational chemistry with a focus on toxicology and has conducted his post-doc research in Know-Center, Austria, and RegionH, Denmark. Mario has published more than 30 peer-reviewed papers. Philip J. Nickel (PhD UCLA, 2002) is an associate professor in the philosophy and e thics group at Eindhoven University of Technology. He specializes in the philosophy of trust and the ethics of disruptive technologies. Some of his research is in the domain of biomedical ethics, focusing on issues of consent and the mediation of care through technology and data. He leads the project “Mobile Support Systems for Behavior Change” funded by the Dutch Research Council’s Responsible Innovation program. M.R.-Z. and V.B. are employees of IBM Research, Haifa, Israel. F.M. is an employee of Philips Research, Eindhoven, the Netherlands. I.S. has received funding from multiple funding agencies through a collaborative funding program and declares no support from any organization for the submitted work. P.J.N. receives funding from the Dutch Research Council (DWO) for the grant “Mobile Support Systems for Behavior Change,” of which he is the principal investigator (P.I.). M.L. is funded by the EU-Commission grant no. 101057497-EDIAQI. We are extremely grateful for Prof. Chris Holmes for his critical reading and valuable comments. We acknowledge the funding received from the European Union's Framework Programme for Research and Innovation Horizon 2020 (2014–2020) under the Marie Skłodowska-Curie Grant agreement no. 813533-MSCA-ITN-2018. I.S. was funded by the “DDAI” COMET Module within the COMET – Competence Centers for Excellent Technologies Programme, funded by the Austrian Federal Ministry for Transport, Innovation and Technology (BMVIT), the Austrian Federal Ministry for Digital and Economic Affairs (BMDW), the Austrian Research Promotion Agency (FFG), the Province of Styria (SFG), and partners from industry and academia. The COMET Program is managed by FFG. Finally, we acknowledge the Big Data Value Association (BDVA), Brussels, Belgium. Conceptualization of the study was conducted by M.R.-Z. Methodology was led by G.S. G.V. L.M. M.L. M.R.-Z. and V.B. and was conducted by all authors. Data curation was done by B.E.V. C.F. C.J.-Q. F.G. F.M. G.V. I.S. L.G. L.M. M.L. M.R.-Z. P.J.N. R.K. S.S. and V.B. Formal analysis was conducted by G.V. M.R.-Z. and V.B. Investigation was equally conducted by all authors. Project administration was done by M.L. and V.B. Resources and software was done by B.E.V. F.G. G.V. L.M. and V.B. Supervision was done by M.R.-Z. Validation, visualization, and writing of original draft were done by G.S. G.V. M.R.-Z. L.G. L.M. P.J.N. R.K. and V.B. Writing (review & editing) was equally done by all authors. M.R.-Z. and V.B. are employees of IBM Research, Haifa, Israel. F.M. is an employee of Philips Research, Eindhoven, the Netherlands. I.S. has received funding from multiple funding agencies through a collaborative funding program and declares no support from any organization for the submitted work. P.J.N. receives funding from the Dutch Research Council (DWO) for the grant “Mobile Support Systems for Behavior Change,” of which he is the principal investigator (P.I.). M.L. is funded by the EU-Commission grant no. 101057497-EDIAQI. We are extremely grateful for Prof. Chris Holmes for his critical reading and valuable comments. We acknowledge the funding received from the European Union’s Framework Programme for Research and Innovation Horizon 2020 (2014–2020) under the Marie Skłodowska-Curie Grant agreement no. 813533-MSCA-ITN-2018 . I.S. was funded by the “ DDAI ” COMET Module within the COMET – Competence Centers for Excellent Technologies Programme , funded by the Austrian Federal Ministry for Transport , Innovation and Technology ( BMVIT ), the Austrian Federal Ministry for Digital and Economic Affairs ( BMDW ), the Austrian Research Promotion Agency (FFG), the Province of Styria (SFG), and partners from industry and academia. The COMET Program is managed by FFG. Finally, we acknowledge the Big Data Value Association (BDVA), Brussels, Belgium.

Keywords

  • artificial intelligence
  • coronavirus
  • COVID-19
  • decision-making
  • DSML 2: Proof-of-concept: Data science output has been formulated, implemented, and tested for one domain/problem
  • explainability
  • foundation models
  • machine learning
  • meta-review
  • PRISMA
  • trustworthiness

Fingerprint

Dive into the research topics of 'A historical perspective of biomedical explainable AI research'. Together they form a unique fingerprint.

Cite this