• Ryan Howard, Jyothi Thumma, and Michael Englesbe.
• Department of Surgery, University of Michigan, Ann Arbor, MI.
• Ann. Surg. 2023 May 1; 277 (5): 775780775-780.

ObjectiveTo evaluate the reliability of surgeon outcomes.BackgroundSurgeons' outcomes are now widely used in public reporting and value-based reimbursement, but the reliability of these measures continues to raise concerns.MethodsWe performed a retrospective study of surgeons performing cholecystectomy, colectomy, and hernia repair on adult patients between January 1, 2017, and December 31, 2020. Outcomes were risk-adjusted rates of complications and high patient satisfaction. We estimated the reliability of each outcome, its relationship with case volume, and the number of surgeons who reached an acceptable level of reliability (≥0.70).ResultsA total of 23,533 patients with a mean age of 56.8 (16.2) years and 10,191 (43.3%) females underwent operations by 333 surgeons. Risk-adjusted complication rate was 2.5% [95% confidence interval (CI): 2.2%-2.8%] and risk-adjusted high satisfaction rate was 79.9% (95% CI: 78.7%-81.0%). The reliability of the complication rate was 0.27 (95% CI: 0.25-0.29) and the reliability of the high satisfaction rate was 0.53 (95% CI: 0.50-0.55). Reliability increased with case volume; however, only 5 (1.5%) surgeons performed enough cases to reach acceptable reliability for their complication rate, while 86 (25.8%) surgeons reached acceptable reliability for their patient satisfaction rate. After adjustment for reliability, the range of complication rates decreased 29-fold from 0% to 14.3% to 2.4% to 2.9%, and the range of patient satisfaction decreased 2.6-fold from 25.3% to 100.0% to 64.9% to 92.4%.ConclusionsAmong surgeons performing common operations, complications and patient satisfaction had relatively low reliability. Although reliability increased with volume, most surgeons had insufficient case volume to achieve acceptable reliability of their outcomes. As such, these measures likely offer little to no meaningful information to inform decision-making.Copyright © 2022 Wolters Kluwer Health, Inc. All rights reserved.

19 keywords...

hide…