Academic medicine : journal of the Association of American Medical Colleges
-
Comparative Study
Does feedback on examination performance help directors of internal medicine residencies evaluate the medical knowledge of their residents against national norms?
As part of the admission process to the American Board of Internal Medicine's certifying examination in internal medicine, training program directors evaluate residents in several components of clinical competence, including medical knowledge. Research suggested that these ratings had different meanings across programs. A report comparing certifying examination performance and ratings of medical knowledge at the program and national levels was developed and sent to program directors after the 1988 through 1992 examinations. The present study investigated whether feedback helped program directors identify where their residents ranked nationally. ⋯ Since equated scores are directly comparable, declining mean scores but unchanged mean ratings suggest that the standards applied by program directors drifted downward. The increasing correlations suggest that program directors improved in their abilities to evaluate residents relative to a common standard. It is not clear what effect the feedback had on program directors' evaluations. It is encouraging, however, to see a higher level of agreement among program directors on the meaning of the ratings.