• James Feghali, Shahab Aldin Sattari, Elizabeth E Wicks, Abhishek Gami, Sarah Rapaport, Tej D Azad, Wuyang Yang, Risheng Xu, Rafael J Tamargo, and Judy Huang.
• Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.
• Neurosurgery. 2022 May 1; 90 (5): 552-561.

BackgroundInterest in machine learning (ML)-based predictive modeling has led to the development of models predicting outcomes after aneurysmal subarachnoid hemorrhage (aSAH), including the Nijmegen acute subarachnoid hemorrhage calculator (Nutshell). Generalizability of such models to external data remains unclear.ObjectiveTo externally validate the performance of the Nutshell tool while comparing it with the conventional Subarachnoid Hemorrhage International Trialists (SAHIT) models and to review the ML literature on outcome prediction after aSAH and aneurysm treatment.MethodsA prospectively maintained database of patients with aSAH presenting consecutively to our institution in the 2013 to 2018 period was used. The web-based Nutshell and SAHIT calculators were used to derive the risks of poor long-term (12-18 months) outcomes and 30-day mortality. Discrimination was evaluated using the area under the curve (AUC), and calibration was investigated using calibration plots. The literature on relevant ML models was surveyed for a synopsis.ResultsIn 269 patients with aSAH, the SAHIT models outperformed the Nutshell tool (AUC: 0.786 vs 0.689, P = .025) in predicting long-term functional outcomes. A logistic regression model of the Nutshell variables derived from our data achieved adequate discrimination (AUC = 0.759) of poor outcomes. The SAHIT models outperformed the Nutshell tool in predicting 30-day mortality (AUC: 0.810 vs 0.636, P < .001). Calibration properties were more favorable for the SAHIT models. Most published aneurysm-related ML-based outcome models lack external validation and usable testing platforms.ConclusionThe Nutshell tool demonstrated limited performance on external validation in comparison with the SAHIT models. External validation and the dissemination of testing platforms for ML models must be emphasized.Copyright © Congress of Neurological Surgeons 2022. All rights reserved.

16 keywords...

hide…