• Tim Dwyer, Jesse Slade Shantz, Kulamakan Mahan Kulasegaram, Jaskarndip Chahal, David Wasserstein, Rachel Schachar, Brian Devitt, John Theodoropoulos, Brian Hodges, and Darrell Ogilvie-Harris.
• Women's College Hospital, Toronto, Ontario, Canada; Mt. Sinai Hospital, Toronto, Ontario, Canada. Electronic address: tim.dwyer@wchospital.ca.
• Arthroscopy. 2016 Dec 1; 32 (12): 2572-2581.e3.

PurposeThe purpose of this study was to determine if the use of an Objective Structured Assessment of Technical skill (OSATS), using dry models, would be a valid method of assessing residents' ability to perform sports medicine procedures after training in a competency-based model.MethodsOver 18 months, 27 residents (19 junior [postgraduate year (PGY) 1-3] and 8 senior [PGY 4-5]) sat the OSATS after their rotation, in addition to 14 sports medicine staff and fellows. Each resident was provided a list of 10 procedures in which they were expected to show competence. At the end of the rotation, each resident undertook an OSATS composed of 6 stations sampled from the 10 procedures using dry models-faculty used the Arthroscopic Surgical Skill Evaluation Tool (ASSET), task-specific checklists, as well as an overall 5-point global rating scale (GRS) to score each resident. Each procedure was videotaped for blinded review.ResultsThe overall reliability of the OSATS (0.9) and the inter-rater reliability (0.9) were both high. A significant difference by year in training was seen for the overall GRS, the total ASSET score, and the total checklist score, as well as for each technical procedure (P < .001). Further analysis revealed a significant difference in the total ASSET score between junior (mean 18.4, 95% confidence interval [CI] 16.8 to 19.9) and senior residents (24.2, 95% CI 22.7 to 25.6), senior residents and fellows (30.1, 95% CI 28.2 to 31.9), as well as between fellows and faculty (37, 95% CI 36.1 to 27.8) (P < .05).ConclusionsThe results of this study show that an OSATS using dry models shows evidence of validity when used to assess performance of technical procedures after a sports medicine rotation. However, junior residents were not able to perform as well as senior residents, suggesting that overall surgical experience is as important as intensive teaching.Clinical RelevanceAs postgraduate medical training shifts to a competency-based model, methods of assessing performance of technical procedures become necessary.Copyright © 2016 Arthroscopy Association of North America. Published by Elsevier Inc. All rights reserved.

15 keywords...

hide…