• Kiki M J M H Lombarts, Andrew Ferguson, Markus W Hollmann, Bente Malling, SMART Collaborators, and Onyebuchi A Arah.
• From the Professional Performance Research Group, Center for Evidence-Based Education, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands (K.M.J.M.H.L.); Department of Anaesthesia and Intensive Care Medicine, Craigavon Area Hospital, Portadown, United Kingdom (A.F.); Department of Anesthesiology, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands (M.W.H.); Centre of Health Sciences Education, Faculty of Health, Aarhus University, Aarhus, Denmark (B.M.); Department of Epidemiology, Fielding School of Public Health, University of California, Los Angeles (UCLA), Los Angeles, California (O.A.A.); and UCLA Center for Health Policy Research, Los Angeles, California (O.A.A.). Paracelsus Medical University, Salzburg, Austria European Union of Medical Specialists, Brussels, Switzerland University Medical Center, Utrecht, The Netherlands Diakonessen Hospital, Utrecht, The Netherlands Free University, Amsterdam, The Netherlands Universitätsklinikum Köln, Köln, Germany Universitätsklinikum Köln, Köln, Germany Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands University Hospital of Basel, Basel, Switzerland Erasmus Medical Center, Rotterdam, The Netherlands.
• Anesthesiology. 2016 Nov 1; 125 (5): 1056-1065.

BackgroundGiven the increasing international recognition of clinical teaching as a competency and regulation of residency training, evaluation of anesthesiology faculty teaching is needed. The System for Evaluating Teaching Qualities (SETQ) Smart questionnaires were developed for assessing teaching performance of faculty in residency training programs in different countries. This study investigated (1) the structure, (2) the psychometric qualities of the new tools, and (3) the number of residents' evaluations needed per anesthesiology faculty to use the instruments reliably.MethodsTwo SETQ Smart questionnaires-for faculty self-evaluation and for resident evaluation of faculty-were developed. A multicenter survey was conducted among 399 anesthesiology faculty and 430 residents in six countries. Statistical analyses included exploratory factor analysis, reliability analysis using Cronbach α, item-total scale correlations, interscale correlations, comparison of composite scales to global ratings, and generalizability analysis to assess residents' evaluations needed per faculty.ResultsIn total, 240 residents completed 1,622 evaluations of 247 faculty. The SETQ Smart questionnaires revealed six teaching qualities consisting of 25 items. Cronbach α's were very high (greater than 0.95) for the overall SETQ Smart questionnaires and high (greater than 0.80) for the separate teaching qualities. Interscale correlations were all within the acceptable range of moderate correlation. Overall, questionnaire and scale scores correlated moderately to highly with the global ratings. For reliable feedback to individual faculty, three to five resident evaluations are needed.ConclusionsThe first internationally piloted questionnaires for evaluating individual anesthesiology faculty teaching performance can be reliably, validly, and feasibly used for formative purposes in residency training.

21 keywords...

hide…