• James H Wykowski and Helene Starks.
• Department of Medicine, Division of Pulmonary Sciences and Critical Care Medicine, School of Medicine, University of Colorado, Aurora, CO, USA. james.wykowski@cuanschutz.edu.
• J Gen Intern Med. 2024 Nov 4.

AbstractIn medical education, learner self-assessments are standard methods used to evaluate the impact of curricula and workshops. Due to the subjective nature of self-assessment, these measures are prone to known biases including framing, recall, social desirability, and response-shift bias. These biases can contribute to floor and ceiling effects of measurement, which can lead to false conclusions about whether the intended learning objectives were achieved. Ideal assessments of skills-based educational activities would include standardized tests and structured observations of learners demonstrating skill use before and after the educational intervention. However, educators often lack the necessary resources, time, and expertise to routinely conduct these appraisals and rely on self-assessment as a pragmatic approach to obtaining curriculum feedback and evaluation data. In this review, we describe three common designs for self-assessments: the pre-post, now-then, and post-only designs. We then give recommendations for choosing between each design to minimize bias. The choice of the best design is based on alignments with four considerations: (1) the educational objectives (e.g., demonstrate skill competency and/or change in skill level); (2) participants' prior experience and shared understanding of levels of skill performance; (3) the nature of the educational activity; and (4) available resources. For each design, we review strengths, weaknesses, and known biases and discuss examples to highlight trade-offs between options. We also discuss the use of control groups and follow-up surveys to measure retention over time as additional methods to address bias and related confounding. The guidance presented here is intended to raise educators' awareness of common pitfalls in self-assessment; minimize the impact of known biases when possible; provide evidence, examples, and rationales for optimal design choices; and increase the rigor of self-assessment evaluations.© 2024. The Author(s), under exclusive licence to Society of General Internal Medicine.

15 keywords...

hide…