• Shelley Ross, Natalia M Binczyk, Deena M Hamza, Shirley Schipper, Paul Humphries, Darren Nichols, and Michel G Donoff.
• Department of Family Medicine, University of Alberta, Edmonton, Alberta, Canada.
• JAMA Netw Open. 2018 Nov 2; 1 (7): e184581.

ImportanceCompetency-based medical education is now established in health professions training. However, critics stress that there is a lack of published outcomes for competency-based medical education or competency-based assessment tools.ObjectiveTo determine whether competency-based assessment is associated with better identification of and support for residents in difficulty.Design, Setting, And ParticipantsThis cohort study of secondary data from archived files on 458 family medicine residents (2006-2008 and 2010-2016) was conducted between July 5, 2016, and March 2, 2018, using a large, urban family medicine residency program in Canada.ExposuresIntroduction of the Competency-Based Achievement System (CBAS).Main Outcomes And MeasuresProportion of residents (1) with at least 1 performance or professionalism flag, (2) receiving flags on multiple distinct rotations, (3) classified as in difficulty, and (4) with flags addressed by the residency program.ResultsFiles from 458 residents were reviewed (pre-CBAS: n = 163; 81 [49.7%] women; 90 [55.2%] aged >30 years; 105 [64.4%] Canadian medical graduates; post-CBAS: n = 295; 144 [48.8%] women; 128 [43.4%] aged >30 years; 243 [82.4%] Canadian medical graduates). A significant reduction in the proportion of residents receiving at least 1 flag during training after CBAS implementation was observed (0.38; 95% CI, 0.377-0.383), as well as a significant decrease in the numbers of distinct rotations during which residents received flags on summative assessments (0.24; 95% CI, 0.237-0.243). There was a decrease in the number of residents in difficulty after CBAS (from 0.13 [95% CI, 0.128-0.132] to 0.17 [95% CI, 0.168-0.172]) depending on the strictness of criteria defining a resident in difficulty. Furthermore, there was a significant increase in narrative documentation that a flag was discussed with the resident between the pre-CBAS and post-CBAS conditions (0.18; 95% CI, 0.178-0.183).Conclusions And RelevanceThe CBAS approach to assessment appeared to be associated with better identification of residents in difficulty, facilitating the program's ability to address learners' deficiencies in competence. After implementation of CBAS, residents experiencing challenges were better supported and their deficiencies did not recur on later rotations. A key argument for shifting to competency-based medical education is to change assessment approaches; these findings suggest that competency-based assessment may be useful.

17 keywords...

hide…