• Mark E Cohen, Yaoming Liu, Kristopher M Huffman, Clifford Y Ko, and Bruce L Hall.
• *Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL†Department of Surgery, University of California Los Angeles David Geffen School of Medicine and the VA Greater Los Angeles Healthcare System, Los Angeles, CA‡Department of Surgery, Washington University in St. Louis; Center for Health Policy and the Olin Business School at Washington University in St Louis; John Cochran Veterans Affairs Medical Center; and BJC Healthcare, St. Louis, MO.
• Ann. Surg. 2016 Dec 1; 264 (6): 966-972.

BackgroundSurgical quality improvement depends on hospitals having accurate and timely information about comparative performance. Profiling accuracy is improved by risk adjustment and shrinkage adjustment to stabilize estimates. These adjustments are included in ACS NSQIP reports, where hospital odds ratios (OR) are estimated using hierarchical models built on contemporaneous data. However, the timeliness of feedback remains an issue.Study DesignWe describe an alternative, nonhierarchical approach, which yields risk- and shrinkage-adjusted rates. In contrast to our "Traditional" NSQIP method, this approach uses preexisting equations, built on historical data, which permits hospitals to have near immediate access to profiling results. We compared our traditional method to this new "on-demand" approach with respect to outlier determinations, kappa statistics, and correlations between logged OR and standardized rates, for 12 models (4 surgical groups by 3 outcomes).ResultsWhen both methods used the same contemporaneous data, there were similar numbers of hospital outliers and correlations between logged OR and standardized rates were high. However, larger differences were observed when the effect of contemporaneous versus historical data was added to differences in statistical methodology.ConclusionsThe on-demand, nonhierarchical approach provides results similar to the traditional hierarchical method and offers immediacy, an "over-time" perspective, application to a broader range of models and data subsets, and reporting of more easily understood rates. Although the nonhierarchical method results are now available "on-demand" in a web-based application, the hierarchical approach has advantages, which support its continued periodic publication as the gold standard for hospital profiling in the program.

14 keywords...

hide…