-
Randomized Controlled Trial Comparative Study
Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial.
- David A Cook, Denise M Dupras, Thomas J Beckman, Kris G Thomas, and V Shane Pankratz.
- Office of Education Research, College of Medicine, Mayo Clinic, Rochester, MN 55905, USA. cook.david33@mayo.edu
- J Gen Intern Med. 2009 Jan 1; 24 (1): 74-9.
BackgroundMini-CEX scores assess resident competence. Rater training might improve mini-CEX score interrater reliability, but evidence is lacking.ObjectiveEvaluate a rater training workshop using interrater reliability and accuracy.DesignRandomized trial (immediate versus delayed workshop) and single-group pre/post study (randomized groups combined).SettingAcademic medical center.ParticipantsFifty-two internal medicine clinic preceptors (31 randomized and 21 additional workshop attendees).InterventionThe workshop included rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion. Delayed group received no intervention until after posttest.MeasurementsMini-CEX ratings at baseline (just before workshop for workshop group), and four weeks later using videotaped resident-patient encounters; mini-CEX ratings of live resident-patient encounters one year preceding and one year following the workshop; rater confidence using mini-CEX.ResultsAmong 31 randomized participants, interrater reliabilities in the delayed group (baseline intraclass correlation coefficient [ICC] 0.43, follow-up 0.53) and workshop group (baseline 0.40, follow-up 0.43) were not significantly different (p = 0.19). Mean ratings were similar at baseline (delayed 4.9 [95% confidence interval 4.6-5.2], workshop 4.8 [4.5-5.1]) and follow-up (delayed 5.4 [5.0-5.7], workshop 5.3 [5.0-5.6]; p = 0.88 for interaction). For the entire cohort, rater confidence (1 = not confident, 6 = very confident) improved from mean (SD) 3.8 (1.4) to 4.4 (1.0), p = 0.018. Interrater reliability for ratings of live encounters (entire cohort) was higher after the workshop (ICC 0.34) than before (ICC 0.18) but the standard error of measurement was similar for both periods.ConclusionsRater training did not improve interrater reliability or accuracy of mini-CEX scores.Clinical Trials Registrationclinicaltrials.gov identifier NCT00667940
Notes
Knowledge, pearl, summary or comment to share?You can also include formatting, links, images and footnotes in your notes
- Simple formatting can be added to notes, such as
*italics*
,_underline_
or**bold**
. - Superscript can be denoted by
<sup>text</sup>
and subscript<sub>text</sub>
. - Numbered or bulleted lists can be created using either numbered lines
1. 2. 3.
, hyphens-
or asterisks*
. - Links can be included with:
[my link to pubmed](http://pubmed.com)
- Images can be included with:
![alt text](https://bestmedicaljournal.com/study_graph.jpg "Image Title Text")
- For footnotes use
[^1](This is a footnote.)
inline. - Or use an inline reference
[^1]
to refer to a longer footnote elseweher in the document[^1]: This is a long footnote.
.