• Mehmet Ulvi Saygi Ayvaci, Varghese S Jacobi, Young Ryu, Saikrishna Pannaga Srikar Gundreddy, and Bekir Tanriover.
• Information Systems, Naveen Jindal School of Management, The University of Texas at Dallas, Dallas, Texas.
• Am. J. Med. 2024 Oct 16.

BackgroundMachine learning algorithms are essential for predicting severe outcomes during public health crises like COVID-19. However, the dynamic nature of diseases requires continual evaluation and updating of these algorithms. This study aims to compare three update strategies for predicting severe COVID-19 outcomes post-diagnosis: 'naive' (a single initial model), 'frequent' (periodic retraining), and 'context-driven' (retraining informed by clinical insights). The goal is to determine the most effective timing and approach for adapting algorithms to evolving disease dynamics and emerging data.MethodsA dataset of 1.11 million COVID-19 patients from diverse U.S. regions was used to develop and validate an XGBoost algorithm for predicting severe outcomes upon diagnosis. Data included patient demographics, vital signs, comorbidities, and immunity-related factors (prior infection and vaccination status) from January 2007 to November 2021. The study analyzed the performance of the three update strategies from March 2020 to November 2021.ResultsPredictive features changed over the pandemic, with comorbidities and vitals being significant initially, and geography, demographics, and immunity-related variables gaining importance later. The 'naive' strategy had an average AUC of 0.77, the 'frequent' strategy-maintained stability with an average AUC of 0.81, and the 'context-driven' strategy averaged an AUC of 0.80, outperforming the 'naive' strategy and aligning closely with the 'frequent' strategy.ConclusionA context-driven approach, guided by clinical insights, can enhance predictive performance and offer cost-effective solutions for dynamic public health challenges. These findings have significant implications for efficiently managing healthcare resources during evolving disease outbreaks.Copyright © 2024. Published by Elsevier Inc.

3 keywords...

hide…