• Eric S Holmboe, Kenji Yamazaki, Thomas J Nasca, and Stanley J Hamstra.
• E.S. Holmboe is chief research, milestones development and evaluation officer, Accreditation Council for Graduate Medical Education, Chicago, Illinois; ORCID: https://orcid.org/0000-0003-0108-6021. K. Yamazaki is senior analyst, Milestones, Accreditation Council for Graduate Medical Education, Chicago, Illinois. T.J. Nasca is president and chief executive officer, Accreditation Council for Graduate Medical Education, Chicago, Illinois, professor of medicine and molecular physiology, Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania, and senior scholar, Department of Education, University of Illinois at Chicago School of Medicine, Chicago, Illinois; ORCID: https://orcid.org/0000-0003-0811-5462. S.J. Hamstra is vice president, Milestones Research and Evaluation, Accreditation Council for Graduate Medical Education, Chicago, Illinois, adjunct professor, Faculty of Education, University of Ottawa, Ottawa, Ontario, Canada, and adjunct professor, Department of Medical Education, Feinberg School of Medicine, Northwestern University, Chicago, Illinois; ORCID: https://orcid.org/0000-0002-0680-366X.
• Acad Med. 2020 Jan 1; 95 (1): 97-103.

PurposeTo investigate the effectiveness of using national, longitudinal milestones data to provide formative assessments to identify residents at risk of not achieving recommended competency milestone goals by residency completion. The investigators hypothesized that specific, lower milestone ratings at earlier time points in residency would be predictive of not achieving recommended Level (L) 4 milestones by graduation.MethodIn 2018, the investigators conducted a longitudinal cohort study of emergency medicine (EM), family medicine (FM), and internal medicine (IM) residents who completed their residency programs from 2015 to 2018. They calculated predictive values and odds ratios, adjusting for nesting within programs, for specific milestone rating thresholds at 6-month intervals for all subcompetencies within each specialty. They used final milestones ratings (May-June 2018) as the outcome variables, setting L4 as the ideal educational outcome.ResultsThe investigators included 1,386 (98.9%) EM residents, 3,276 (98.0%) FM residents, and 7,399 (98.0%) IM residents in their analysis. The percentage of residents not reaching L4 by graduation ranged from 11% to 31% in EM, 16% to 53% in FM, and 5% to 15% in IM. Using a milestone rating of L2.5 or lower at the end of post-graduate year 2, the predictive probability of not attaining the L4 milestone graduation goal ranged from 32% to 56% in EM, 32% to 67% in FM, and 15% to 36% in IM.ConclusionsLongitudinal milestones ratings may provide educationally useful, predictive information to help individual residents address potential competency gaps, but the predictive power of the milestones ratings varies by specialty and subcompetency within these 3 adult care specialties.

21 keywords...

hide…