• Deborah M Rooney, Eric S Hungness, Debra A Darosa, and Carla M Pugh.
• Northwestern Center for Advanced Surgical Education, Simulation Technology and Immersive Learning, Feinberg School of Medicine, Chicago, IL 60611, USA. d-rooney@northwestern.edu
• Surgery. 2012 Jun 1; 151 (6): 796-802.

IntroductionThe purpose of this study was to compare faculty ratings between live versus video-recorded resident performances and faculty versus skills coaches' ratings of video-recorded resident performances.MethodsPGY1 residents were observed, video-recorded, and rated during a Verification of Proficiency examination on 4 stations (ie, suturing, laparotomy, central line, and cricothyroidotomy). One surgeon and 2 trained skills coaches independently rated each video-recorded performance (N = 25). The chi-square test was used to compare checklist ratings. Analysis of variance was used to compare global ratings. Intraclass correlations were used to evaluate inter-rater agreement.ResultsThere were no statistical differences in faculty checklist ratings for live versus video-recorded performances (P > .05), and we found a nearly perfect interrater agreement, intraclass correlation coefficient (ICC) = 0.99 (P < .001). When comparing faculty versus skills coaches' ratings on video-recorded performances, we found no differences for the global or checklist ratings. Inter-rater agreement was moderately high for the global ratings, ICC = 0.71 (P <. 0.01, 95% confidence interval 0.23-0.96), and nearly perfect for the checklist ratings, ICC = 0.99 (P < .001, 95% confidence interval 0.94-1.00).ConclusionWhen assessing residents' performances, use of video-recorded performance ratings and skills coaches may be viable alternatives to live ratings performed by surgical faculty.Copyright © 2012 Mosby, Inc. All rights reserved.

23 keywords...

hide…