• Elise H Lawson, Clifford Y Ko, John L Adams, Warren B Chow, and Bruce Lee Hall.
• *Department of Surgery, David Geffen School of Medicine, University of California, Los Angeles †Division of Research and Optimal Patient Care, American College of Surgeons, Chicago, IL ‡VA Greater Los Angeles Healthcare System, Los Angeles, California §RAND Corporation, Santa Monica, CA; and ¶Department of Surgery, Washington University School of Medicine in St Louis, Barnes Jewish Hospital, Saint Louis Veterans Affairs Medical Center, and Washington University Center for Health Policy and Olin Business School, St Louis, MO.
• Ann. Surg. 2013 Dec 1; 258 (6): 994-1000.

ObjectiveTo determine whether risk-adjusted colorectal SSI rates are statistically reliable as hospital quality measures.BackgroundPolicymakers use surgical site infections (SSI) for public reporting of hospital quality and pay-for-performance because they are a relatively common and costly cause of patient morbidity.MethodsPatients who underwent a colorectal procedure in 2009 were identified from the American College of Surgeons National Surgical Quality Improvement Program. We developed hierarchical multivariate logistic models for (1) superficial SSI, (2) deep/organ-space SSI, and (3) "any SSI" and compared how each model ranked hospital-level risk-adjusted performance. Statistical reliability of hospital quality measurements was estimated on a scale from 0 to 1; with 0 indicating that apparent variation between a hospital's quality measurement and the average hospital is statistically unreliable, and 1 indicating that any observed variation is due to a real difference in performance.ResultsMean reliability of hospital-level quality measurements was 0.650 for superficial, 0.404 for deep/organ-space, and 0.586 for "any SSI." Lower reliability was accounted for by relatively little variation in risk-adjusted SSI rates between hospitals and insufficient numbers of colorectal cases submitted by individual hospitals. In 2009, we estimate that 22.1% of all US hospitals performed a sufficient number of colorectal cases to report superficial SSI rates at a high standard of statistical reliability and 1.0% did for deep/organ-space SSI.ConclusionsAs currently constructed, colorectal SSI quality measures might not meet a high standard of statistical reliability for most hospitals, limiting their ability to confidently differentiate high and low performance. Despite an expectation of improving statistical power, combining superficial and deep/organ-space SSI into an "any SSI" measure worsens reliability.

15 keywords...

hide…