When adding new markers to existing prediction models, it is necessary to evaluate the models to determine whether the additional markers are useful. The net reclassification improvement (NRI) has gained popularity in this role because of its simplicity, ease of estimation, and understandability. Although the NRI provides a single-number summary describing the improvement new markers bring to a model, it also has several potential disadvantages. ⋯ Also, overfitting, or otherwise misspecified training models, produce overly positive NRI results. Because of the unaccounted for uncertainty in the model coefficient estimation, investigators should rely on bootstrapped confidence intervals rather than on tests of significance. Keeping in mind the limitations and drawbacks, the NRI can be helpful when used correctly.
Elizabeth S Jewell, Michael D Maile, Milo Engoren, and Michael Elliott.
From the Department of Anesthesiology, University of Michigan, Ann Arbor, Michigan; and Department of Biostatistics, University of Michigan, Ann Arbor, Michigan.
Anesth. Analg. 2016 Mar 1; 122 (3): 818-824.
AbstractWhen adding new markers to existing prediction models, it is necessary to evaluate the models to determine whether the additional markers are useful. The net reclassification improvement (NRI) has gained popularity in this role because of its simplicity, ease of estimation, and understandability. Although the NRI provides a single-number summary describing the improvement new markers bring to a model, it also has several potential disadvantages. Any improved classification by the new model is weighted equally, regardless of the direction of reclassification. In prediction models that already identify the high- and low-risk groups well, a positive NRI may not mean better classification of those with medium risk, where it could make the most difference. Also, overfitting, or otherwise misspecified training models, produce overly positive NRI results. Because of the unaccounted for uncertainty in the model coefficient estimation, investigators should rely on bootstrapped confidence intervals rather than on tests of significance. Keeping in mind the limitations and drawbacks, the NRI can be helpful when used correctly.