• Jonathan P Wanderer, Edward A Bittner, and Matthew J G Sigakis.
• Department of Anesthesia, Critical Care, and Pain Medicine, Massachusetts General Hospital, Gray-Bigelow 444, 55 Fruit Street, Boston, Massachusetts 02114, USA. msigakis@partners.org
• Anesthesiology. 2013 Sep 1;119(3):525-40.

BackgroundExternal validation of published risk stratification models is essential to determine their generalizability. This study evaluates the performance of the Risk Stratification Indices (RSIs) and 30-day mortality Risk Quantification Index (RQI).Methods108,423 adult hospital admissions with anesthetics were identified (2006–2011). RSIs for mortality and length-of-stay endpoints were calculated using published methodology. 91,128 adult, noncardiac inpatient surgeries were identified with administrative data required for RQI calculation.ResultsRSI in-hospital mortality and RQI 30-day mortality Brier scores were 0.308 and 0.017, respectively. RSI discrimination, by area under the receiver operating curves, was excellent at 0.966 (95% CI, 0.963–0.970) for in-hospital mortality, 0.903 (0.896–0.909) for 30-day mortality, 0.866 (0.861–0.870) for 1-yr mortality, and 0.884 (0.882–0.886) for length-of-stay. RSI calibration, however, was poor overall (17% predicted in-hospital mortality vs. 1.5% observed after inclusion of the regression constant) as demonstrated by calibration plots. Removal of self-fulfilling diagnosis and procedure codes (20,001 of 108,423; 20%) yielded similar results. RQIs were calculated for only 62,640 of 91,128 patients (68.7%) due to unmatched procedure codes. Patients with unmatched codes were younger, had higher American Society of Anesthesiologists physical status and 30-day mortality. The area under the receiver operating curve for 30-day mortality RQI was 0.888 (0.879–0.897). The model also demonstrated good calibration. Performance of a restricted index, Procedure Severity Score + American Society of Anesthesiologists physical status, performed as well as the original RQI model (age + American Society of Anesthesiologists + Procedure Severity Score).ConclusionAlthough the RSIs demonstrated excellent discrimination, poor calibration limits their generalizability. The 30-day mortality RQI performed well with age providing a limited contribution.

22 keywords...

hide…