• Ting Sun, Yoon Soo Park, Fei Chen, Sean O Hogan, and Pedro Tanaka.
• Office of Surgical Education, Department of Surgery, University of Utah School of Medicine, Salt Lake City, UT.
• Anesthesiology. 2025 Jan 27.

BackgroundLongitudinal Milestones data reported to the Accreditation Council for Graduate Medical Education (ACGME) provide a structured framework for assessing the developmental progression of residents in key competencies and subcompetencies. This study aims to investigate the previously underexplored longitudinal reliability of Milestones data, with the goal of identifying patterns in learning trajectories that can inform targeted interventions for residents and programs.MethodsWe conducted a retrospective cohort study with national anesthesiology Milestones data collected from 2014 to 2020. Mixed-effects growth curve models were fit to model residents' growth trajectories. Longitudinal reliability was assessed using the indices of growth rate reliability (GRR) and growth curve reliability (GCR). We also examined variance components attributable to the factors at both the learner and program levels. Latent class growth analyses (LCGA) were performed to identify latent groups of learners with different learning trajectories.ResultsThe study included a total of 682,475 ratings for 4976 learners in 140 programs. Growth curve model results indicated that the mean baseline Milestone rating across the 25 subcompetencies was 2.05 (95% CI: 1.96-2.14), with an average increase of 0.49 (95% CI: 0.48-0.51) units per reporting period. The GRR (mean ± SD, 0.58 ± 0.03) suggested a moderate capability of anesthesiology Milestones to detect individual differences in the growth of latent competency. GCR estimates (mean ± SD, 0.76 ± 0.02) suggested acceptable overall reliability of Milestones across all the six assessment points. Significant variability was observed at both the program and learner levels (P < 0.001). LCGA identified 3-4 latent groups of learners with distinct learning trajectories across the 25 subcompetencies.ConclusionsOur study indicated that the anesthesiology Milestones provides moderately reliable information for tracking individual progress over time. The findings underscore the importance of using a multi-faceted approach to assessment and providing individualized learning plans to support resident development.Copyright © 2025 The Author(s). Published by Wolters Kluwer Health, Inc., on behalf of the American Society of Anesthesiologists.

5 keywords...

hide…