A community challenge to predict clinical outcomes after immune checkpoint blockade in non-small cell lung cancer

Mike Mason; Óscar Lapuente-Santana; Anni S. Halkola; Wenyu Wang; Raghvendra Mall; Xu Xiao; Jacob Kaufman; Jingxin Fu; Jacob Pfeil; Jineta Banerjee; Verena Chung; Han Chang; Scott D. Chasalow; Hung Ying Lin; Rongrong Chai; Thomas Yu; Francesca Finotello; Tuomas Mirtti; Mikko I. Mäyränpää; Jie Bao; Emmy W. Verschuren; Eiman I. Ahmed; Michele Ceccarelli; Lance D. Miller; Gianni Monaco; Wouter R.L. Hendrickx; Shimaa Sherif; Lin Yang; Ming Tang; Shengqing Stan Gu; Wubing Zhang; Yi Zhang; Zexian Zeng; Avinash Das Sahu; Yang Liu; Wenxian Yang; Davide Bedognetti; Jing Tang; Federica Eduati; Teemu D. Laajala; William J. Geese; Justin Guinney; Joseph D. Szustakowski; Benjamin G. Vincent; David P. Carbone

doi:10.1186/s12967-023-04705-3

A community challenge to predict clinical outcomes after immune checkpoint blockade in non-small cell lung cancer

Mike Mason, Óscar Lapuente-Santana, Anni S. Halkola, Wenyu Wang, Raghvendra Mall, Xu Xiao, Jacob Kaufman, Jingxin Fu, Jacob Pfeil, Jineta Banerjee, Verena Chung, Han Chang, Scott D. Chasalow, Hung Ying Lin, Rongrong Chai, Thomas Yu, Francesca Finotello, Tuomas Mirtti, Mikko I. Mäyränpää, Jie BaoEmmy W. Verschuren, Eiman I. Ahmed, Michele Ceccarelli, Lance D. Miller, Gianni Monaco, Wouter R.L. Hendrickx, Shimaa Sherif, Lin Yang, Ming Tang, Shengqing Stan Gu, Wubing Zhang, Yi Zhang, Zexian Zeng, Avinash Das Sahu, Yang Liu, Wenxian Yang, Davide Bedognetti, Jing Tang, Federica Eduati, Teemu D. Laajala, William J. Geese, Justin Guinney, Joseph D. Szustakowski, Benjamin G. Vincent, David P. Carbone (Corresponding author)

Research output: Contribution to journal › Article › Academic › peer-review

1 Citation (Scopus)

25 Downloads (Pure)

Abstract

BACKGROUND: Predictive biomarkers of immune checkpoint inhibitor (ICI) efficacy are currently lacking for non-small cell lung cancer (NSCLC). Here, we describe the results from the Anti-PD-1 Response Prediction DREAM Challenge, a crowdsourced initiative that enabled the assessment of predictive models by using data from two randomized controlled clinical trials (RCTs) of ICIs in first-line metastatic NSCLC.

METHODS: Participants developed and trained models using public resources. These were evaluated with data from the CheckMate 026 trial (NCT02041533), according to the model-to-data paradigm to maintain patient confidentiality. The generalizability of the models with the best predictive performance was assessed using data from the CheckMate 227 trial (NCT02477826). Both trials were phase III RCTs with a chemotherapy control arm, which supported the differentiation between predictive and prognostic models. Isolated model containers were evaluated using a bespoke strategy that considered the challenges of handling transcriptome data from clinical trials.

RESULTS: A total of 59 teams participated, with 417 models submitted. Multiple predictive models, as opposed to a prognostic model, were generated for predicting overall survival, progression-free survival, and progressive disease status with ICIs. Variables within the models submitted by participants included tumor mutational burden (TMB), programmed death ligand 1 (PD-L1) expression, and gene-expression-based signatures. The best-performing models showed improved predictive power over reference variables, including TMB or PD-L1.

CONCLUSIONS: This DREAM Challenge is the first successful attempt to use protected phase III clinical data for a crowdsourced effort towards generating predictive models for ICI clinical outcomes and could serve as a blueprint for similar efforts in other tumor types and disease states, setting a benchmark for future studies aiming to identify biomarkers predictive of ICI efficacy.

TRIAL REGISTRATION: CheckMate 026; NCT02041533, registered January 22, 2014. CheckMate 227; NCT02477826, registered June 23, 2015.

Original language	English
Article number	190
Number of pages	16
Journal	Journal of Translational Medicine
Volume	22
Issue number	1
DOIs	https://doi.org/10.1186/s12967-023-04705-3
Publication status	Published - 21 Feb 2024

Bibliographical note

Keywords

Humans
Carcinoma, Non-Small-Cell Lung/drug therapy
Immune Checkpoint Inhibitors/therapeutic use
Lung Neoplasms/pathology
B7-H1 Antigen
Biomarkers, Tumor
Programmed death ligand 1
Immune checkpoint inhibitor
Predictive model
Crowdsource
Non-small cell lung cancer
Biomarkers
Programmed death-1

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1186/s12967-023-04705-3

pdfFinal published version, 1.75 MBLicence: CC BY

Cite this

Mason, M., Lapuente-Santana, Ó., Halkola, A. S., Wang, W., Mall, R., Xiao, X., Kaufman, J., Fu, J., Pfeil, J., Banerjee, J., Chung, V., Chang, H., Chasalow, S. D., Lin, H. Y., Chai, R., Yu, T., Finotello, F., Mirtti, T., Mäyränpää, M. I., ... Carbone, D. P. (2024). A community challenge to predict clinical outcomes after immune checkpoint blockade in non-small cell lung cancer. Journal of Translational Medicine, 22(1), Article 190. https://doi.org/10.1186/s12967-023-04705-3

@article{e13e80a9f9664149a95ed4a38ce406c5,

title = "A community challenge to predict clinical outcomes after immune checkpoint blockade in non-small cell lung cancer",

abstract = "BACKGROUND: Predictive biomarkers of immune checkpoint inhibitor (ICI) efficacy are currently lacking for non-small cell lung cancer (NSCLC). Here, we describe the results from the Anti-PD-1 Response Prediction DREAM Challenge, a crowdsourced initiative that enabled the assessment of predictive models by using data from two randomized controlled clinical trials (RCTs) of ICIs in first-line metastatic NSCLC.METHODS: Participants developed and trained models using public resources. These were evaluated with data from the CheckMate 026 trial (NCT02041533), according to the model-to-data paradigm to maintain patient confidentiality. The generalizability of the models with the best predictive performance was assessed using data from the CheckMate 227 trial (NCT02477826). Both trials were phase III RCTs with a chemotherapy control arm, which supported the differentiation between predictive and prognostic models. Isolated model containers were evaluated using a bespoke strategy that considered the challenges of handling transcriptome data from clinical trials.RESULTS: A total of 59 teams participated, with 417 models submitted. Multiple predictive models, as opposed to a prognostic model, were generated for predicting overall survival, progression-free survival, and progressive disease status with ICIs. Variables within the models submitted by participants included tumor mutational burden (TMB), programmed death ligand 1 (PD-L1) expression, and gene-expression-based signatures. The best-performing models showed improved predictive power over reference variables, including TMB or PD-L1.CONCLUSIONS: This DREAM Challenge is the first successful attempt to use protected phase III clinical data for a crowdsourced effort towards generating predictive models for ICI clinical outcomes and could serve as a blueprint for similar efforts in other tumor types and disease states, setting a benchmark for future studies aiming to identify biomarkers predictive of ICI efficacy.TRIAL REGISTRATION: CheckMate 026; NCT02041533, registered January 22, 2014. CheckMate 227; NCT02477826, registered June 23, 2015.",

keywords = "Humans, Carcinoma, Non-Small-Cell Lung/drug therapy, Immune Checkpoint Inhibitors/therapeutic use, Lung Neoplasms/pathology, B7-H1 Antigen, Biomarkers, Tumor, Programmed death ligand 1, Immune checkpoint inhibitor, Predictive model, Crowdsource, Non-small cell lung cancer, Biomarkers, Programmed death-1",

author = "Mike Mason and {\'O}scar Lapuente-Santana and Halkola, {Anni S.} and Wenyu Wang and Raghvendra Mall and Xu Xiao and Jacob Kaufman and Jingxin Fu and Jacob Pfeil and Jineta Banerjee and Verena Chung and Han Chang and Chasalow, {Scott D.} and Lin, {Hung Ying} and Rongrong Chai and Thomas Yu and Francesca Finotello and Tuomas Mirtti and M{\"a}yr{\"a}np{\"a}{\"a}, {Mikko I.} and Jie Bao and Verschuren, {Emmy W.} and Ahmed, {Eiman I.} and Michele Ceccarelli and Miller, {Lance D.} and Gianni Monaco and Hendrickx, {Wouter R.L.} and Shimaa Sherif and Lin Yang and Ming Tang and Gu, {Shengqing Stan} and Wubing Zhang and Yi Zhang and Zexian Zeng and {Das Sahu}, Avinash and Yang Liu and Wenxian Yang and Davide Bedognetti and Jing Tang and Federica Eduati and Laajala, {Teemu D.} and Geese, {William J.} and Justin Guinney and Szustakowski, {Joseph D.} and Vincent, {Benjamin G.} and Carbone, {David P.}",

note = "{\textcopyright} 2024. The Author(s).",

year = "2024",

month = feb,

day = "21",

doi = "10.1186/s12967-023-04705-3",

language = "English",

volume = "22",

journal = "Journal of Translational Medicine",

issn = "1479-5876",

publisher = "BioMed Central",

number = "1",

}

Mason, M, Lapuente-Santana, Ó, Halkola, AS, Wang, W, Mall, R, Xiao, X, Kaufman, J, Fu, J, Pfeil, J, Banerjee, J, Chung, V, Chang, H, Chasalow, SD, Lin, HY, Chai, R, Yu, T, Finotello, F, Mirtti, T, Mäyränpää, MI, Bao, J, Verschuren, EW, Ahmed, EI, Ceccarelli, M, Miller, LD, Monaco, G, Hendrickx, WRL, Sherif, S, Yang, L, Tang, M, Gu, SS, Zhang, W, Zhang, Y, Zeng, Z, Das Sahu, A, Liu, Y, Yang, W, Bedognetti, D, Tang, J, Eduati, F, Laajala, TD, Geese, WJ, Guinney, J, Szustakowski, JD, Vincent, BG & Carbone, DP 2024, 'A community challenge to predict clinical outcomes after immune checkpoint blockade in non-small cell lung cancer', Journal of Translational Medicine, vol. 22, no. 1, 190. https://doi.org/10.1186/s12967-023-04705-3

TY - JOUR

T1 - A community challenge to predict clinical outcomes after immune checkpoint blockade in non-small cell lung cancer

AU - Mason, Mike

AU - Lapuente-Santana, Óscar

AU - Halkola, Anni S.

AU - Wang, Wenyu

AU - Mall, Raghvendra

AU - Xiao, Xu

AU - Kaufman, Jacob

AU - Fu, Jingxin

AU - Pfeil, Jacob

AU - Banerjee, Jineta

AU - Chung, Verena

AU - Chang, Han

AU - Chasalow, Scott D.

AU - Lin, Hung Ying

AU - Chai, Rongrong

AU - Yu, Thomas

AU - Finotello, Francesca

AU - Mirtti, Tuomas

AU - Mäyränpää, Mikko I.

AU - Bao, Jie

AU - Verschuren, Emmy W.

AU - Ahmed, Eiman I.

AU - Ceccarelli, Michele

AU - Miller, Lance D.

AU - Monaco, Gianni

AU - Hendrickx, Wouter R.L.

AU - Sherif, Shimaa

AU - Yang, Lin

AU - Tang, Ming

AU - Gu, Shengqing Stan

AU - Zhang, Wubing

AU - Zhang, Yi

AU - Zeng, Zexian

AU - Das Sahu, Avinash

AU - Liu, Yang

AU - Yang, Wenxian

AU - Bedognetti, Davide

AU - Tang, Jing

AU - Eduati, Federica

AU - Laajala, Teemu D.

AU - Geese, William J.

AU - Guinney, Justin

AU - Szustakowski, Joseph D.

AU - Vincent, Benjamin G.

AU - Carbone, David P.

PY - 2024/2/21

Y1 - 2024/2/21

N2 - BACKGROUND: Predictive biomarkers of immune checkpoint inhibitor (ICI) efficacy are currently lacking for non-small cell lung cancer (NSCLC). Here, we describe the results from the Anti-PD-1 Response Prediction DREAM Challenge, a crowdsourced initiative that enabled the assessment of predictive models by using data from two randomized controlled clinical trials (RCTs) of ICIs in first-line metastatic NSCLC.METHODS: Participants developed and trained models using public resources. These were evaluated with data from the CheckMate 026 trial (NCT02041533), according to the model-to-data paradigm to maintain patient confidentiality. The generalizability of the models with the best predictive performance was assessed using data from the CheckMate 227 trial (NCT02477826). Both trials were phase III RCTs with a chemotherapy control arm, which supported the differentiation between predictive and prognostic models. Isolated model containers were evaluated using a bespoke strategy that considered the challenges of handling transcriptome data from clinical trials.RESULTS: A total of 59 teams participated, with 417 models submitted. Multiple predictive models, as opposed to a prognostic model, were generated for predicting overall survival, progression-free survival, and progressive disease status with ICIs. Variables within the models submitted by participants included tumor mutational burden (TMB), programmed death ligand 1 (PD-L1) expression, and gene-expression-based signatures. The best-performing models showed improved predictive power over reference variables, including TMB or PD-L1.CONCLUSIONS: This DREAM Challenge is the first successful attempt to use protected phase III clinical data for a crowdsourced effort towards generating predictive models for ICI clinical outcomes and could serve as a blueprint for similar efforts in other tumor types and disease states, setting a benchmark for future studies aiming to identify biomarkers predictive of ICI efficacy.TRIAL REGISTRATION: CheckMate 026; NCT02041533, registered January 22, 2014. CheckMate 227; NCT02477826, registered June 23, 2015.

AB - BACKGROUND: Predictive biomarkers of immune checkpoint inhibitor (ICI) efficacy are currently lacking for non-small cell lung cancer (NSCLC). Here, we describe the results from the Anti-PD-1 Response Prediction DREAM Challenge, a crowdsourced initiative that enabled the assessment of predictive models by using data from two randomized controlled clinical trials (RCTs) of ICIs in first-line metastatic NSCLC.METHODS: Participants developed and trained models using public resources. These were evaluated with data from the CheckMate 026 trial (NCT02041533), according to the model-to-data paradigm to maintain patient confidentiality. The generalizability of the models with the best predictive performance was assessed using data from the CheckMate 227 trial (NCT02477826). Both trials were phase III RCTs with a chemotherapy control arm, which supported the differentiation between predictive and prognostic models. Isolated model containers were evaluated using a bespoke strategy that considered the challenges of handling transcriptome data from clinical trials.RESULTS: A total of 59 teams participated, with 417 models submitted. Multiple predictive models, as opposed to a prognostic model, were generated for predicting overall survival, progression-free survival, and progressive disease status with ICIs. Variables within the models submitted by participants included tumor mutational burden (TMB), programmed death ligand 1 (PD-L1) expression, and gene-expression-based signatures. The best-performing models showed improved predictive power over reference variables, including TMB or PD-L1.CONCLUSIONS: This DREAM Challenge is the first successful attempt to use protected phase III clinical data for a crowdsourced effort towards generating predictive models for ICI clinical outcomes and could serve as a blueprint for similar efforts in other tumor types and disease states, setting a benchmark for future studies aiming to identify biomarkers predictive of ICI efficacy.TRIAL REGISTRATION: CheckMate 026; NCT02041533, registered January 22, 2014. CheckMate 227; NCT02477826, registered June 23, 2015.

KW - Humans

KW - Carcinoma, Non-Small-Cell Lung/drug therapy

KW - Immune Checkpoint Inhibitors/therapeutic use

KW - Lung Neoplasms/pathology

KW - B7-H1 Antigen

KW - Biomarkers, Tumor

KW - Programmed death ligand 1

KW - Immune checkpoint inhibitor

KW - Predictive model

KW - Crowdsource

KW - Non-small cell lung cancer

KW - Biomarkers

KW - Programmed death-1

UR - http://www.scopus.com/inward/record.url?scp=85185665029&partnerID=8YFLogxK

U2 - 10.1186/s12967-023-04705-3

DO - 10.1186/s12967-023-04705-3

M3 - Article

C2 - 38383458

SN - 1479-5876

VL - 22

JO - Journal of Translational Medicine

JF - Journal of Translational Medicine

IS - 1

M1 - 190

ER -

A community challenge to predict clinical outcomes after immune checkpoint blockade in non-small cell lung cancer

Abstract

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this