• David R Taylor, Yoon Soo Park, Rylan Egan, Ming-Ka Chan, Jolanta Karpinski, Claire Touchie, Linda S Snell, and Ara Tekian.
• D.R. Taylor is associate professor, Department of Medicine, Queen's University School of Medicine, Kingston, Ontario, Canada. Y.S. Park is assistant professor, Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois. R. Egan is assistant professor and director, Office of Health Science Education, Queen's University School of Medicine, Kingston, Ontario, Canada. M.-K. Chan is associate professor, Department of Pediatrics and Child Health, University of Manitoba, Winnipeg, Manitoba, Canada, and clinician educator, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada. J. Karpinski is associate professor, Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada, and associate director, Specialties Unit, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada. C. Touchie is associate professor, Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada, and chief medical education advisor, Medical Council of Canada, Ottawa, Ontario, Canada. L.S. Snell is professor, Department of Medicine, McGill University, Montreal, Quebec, Canada, and senior clinician educator, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada. A. Tekian is professor, Department of Medical Education, University of Illinois at Chicago, Chicago, Illinois.
• Acad Med. 2017 Nov 1; 92 (11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions): S110-S117.

PurposeEntrustable professional activities (EPAs) have become a cornerstone of assessment in competency-based medical education (CBME). Increasingly, EPAs are being adopted that do not conform to EPA standards. This study aimed to develop and validate a scoring rubric to evaluate EPAs for alignment with their purpose, and to identify substandard EPAs.MethodThe EQual rubric was developed and revised by a team of education scholars with expertise in EPAs. It was then applied by four residency program directors/CBME leads (PDs) and four nonclinician support staff to 31 stage-specific EPAs developed for internal medicine in the Royal College of Physicians and Surgeons of Canada's Competency by Design framework. Results were analyzed using a generalizability study to evaluate overall reliability, with the EPAs as the object of measurement. Item-level analysis was performed to determine reliability and discrimination value for each item. Scores from the PDs were also compared with decisions about revisions made independently by the education scholars group.ResultsThe EQual rubric demonstrated high reliability in the G-study with a phi-coefficient of 0.84 when applied by the PDs, and moderate reliability when applied by the support staff at 0.67. Item-level analysis identified three items that performed poorly with low item discrimination and low interrater reliability indices. Scores from support staff only moderately correlated with PDs. Using the preestablished cut score, PDs identified 9 of 10 EPAs deemed to require major revision.ConclusionsEQual rubric scores reliably measured alignment of EPAs with literature-described standards. Further, its application accurately identified EPAs requiring major revisions.

11 keywords...

hide…