• Jacob Wallace, J Michael McWilliams, Anthony Lollo, Janet Eaton, and Chima D Ndumele.
• Yale School of Public Health, New Haven, Connecticut (J.W., A.L., C.D.N.).
• Ann. Intern. Med. 2022 Mar 1; 175 (3): 314-324.

BackgroundRisk adjustment is used widely in payment systems and performance assessments, but the extent to which it distinguishes plan or provider effects from confounding due to patient differences is typically unknown.ObjectiveTo assess the degree to which risk-adjusted measures of health plan performance adequately adjust for the variation across plans that arises because of differences in patient characteristics (residual confounding).DesignComparison between plan performance estimates based on enrollees who made plan choices (observational population) and estimates based on enrollees assigned to plans (randomized population).SettingNatural experiment in which more than two thirds of a state's Medicaid population in 1 region was randomly assigned to 1 of 5 plans.Participants137 933 enrollees in 2013 to 2014, of whom 31.1% selected a plan and 68.9% were randomly assigned to 1 of the same 5 plans.MeasurementsAnnual total spending (that is, payments to providers), primary care use, dental care use, and avoidable emergency department visits, all scored as plan-specific deviations from the "average" plan performance within each population.ResultsEnrollee characteristics were appreciably imbalanced across plans in the observational population, as expected, but were not in the randomized population. Annual total spending varied across plans more in the observational population (SD, $147 per enrollee) than in the randomized population (SD, $70 per enrollee) after accounting for baseline differences in the observational and randomized populations and for differences across plans. On average, a plan's spending score (its deviation from the "average" performance) in the observational population differed from its score in the randomized population by $67 per enrollee in absolute value (95% CI, $38 to $123), or 4.2% of mean spending per enrollee (P = 0.009, rejecting the null hypothesis that this difference would be expected from sampling error). The difference was reduced modestly by risk adjustment to $62 per enrollee (P = 0.012). Residual confounding was similarly substantial for most other performance measures. Further adjustment for social factors did not materially change estimates.LimitationPotential heterogeneity in plan effects between the 2 populations.ConclusionResidual confounding in risk-adjusted performance assessments can be substantial and should caution policymakers against assuming that risk adjustment isolates real differences in plan performance.Primary Funding SourceArnold Ventures.

9 keywords...

hide…