• Matthew R Thomas, Thomas J Beckman, Karen F Mauck, Stephen S Cha, and Kris G Thomas.
• Division of Primary Care Internal Medicine, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, USA. thomas.matthew@mayo.edu
• J Gen Intern Med. 2011 Jul 1; 26 (7): 759-64.

BackgroundIndividual faculty assessments of resident competency are complicated by inconsistent application of standards, lack of reliability, and the "halo" effect.ObjectiveWe determined whether the addition of faculty group assessments of residents in an ambulatory clinic, compared with individual faculty-of-resident assessments alone, have better reliability and reduced halo effects.DesignThis prospective, longitudinal study was performed in the outpatient continuity clinics of a large internal medicine residency program.Main MeasuresFaculty-on-resident and group faculty-on-resident assessment scores were used for comparison.Key ResultsOverall mean scores were significantly higher for group than individual assessments (3.92 ± 0.51 vs. 3.83 ± 0.38, p = 0.0001). Overall inter-rater reliability increased when combining group and individual assessments compared to individual assessments alone (intraclass correlation coefficient, 95% CI = 0.828, 0.785-0.866 vs. 0.749, 0.686-0.804). Inter-item correlations were less for group (0.49) than individual (0.68) assessments.ConclusionsThis study demonstrates improved inter-rater reliability and reduced range restriction (halo effect) of resident assessment across multiple performance domains by adding the group assessment method to traditional individual faculty-on-resident assessment. This feasible model could help graduate medical education programs achieve more reliable and discriminating resident assessments.

17 keywords...

hide…