Improving Machine Learning Models of Late-onset Sepsis Prediction Using Optimized Control Group Selection: A Comparative Study

Late-onset sepsis (LOS) remains a major cause of morbidity and mortality in preterm infants due to their immature immune systems and the complex clinical environment of neonatal intensive care units (NICUs). Predictive models leveraging non-invasive vital sign data, such as heart rate variability (HRV), respiration and motion, have shown potential for early detection of LOS. However, the use of different control group selection strategies across LOS prediction studies, without any systematic comparison, has limited the potential to further improve the model performance. In this study, we evaluated three LOS prediction models trained using different control group selection strategies: 1) independent control, 2) intra-LOS control, and 3) a combined strategy. The training dataset included 128 preterm infants (60 LOS, 68 controls), while a independent test dataset of 49 patients (12 LOS, 37 non-LOS) was used for evaluation. The prediction performance of the models was assessed using the area under the receiver operating characteristic curve (AUC), along with sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) across varying prediction time windows. Our results showed that the combined strategy achieved the highest AUC (86.2%) within the 3-hour prediction window, outperforming the independent (70.9%) and intra-LOS (78.2%) strategies. While the independent strategy displayed the highest sensitivity and NPV, the intra-LOS approach offered better specificity and PPV. The combined strategy effectively integrated these strengths, offering superior predictive performance across key metrics. This study highlights the importance of control group selection in LOS model development and recommends the combined strategy as a promising approach for improving model performance.Clinical Relevance- This study highlights the impact of control group selection strategies on the predictive performance of LOS models, enhancing model generalizability in clinical LOS management.