Multisource assessment (MSA) uses multiple assessors to provide feedback. Little is known about the validity of using MSA feedback for improving students' ability to self-assess in a preclinical environment. Therefore, the aim of this study was to measure the validity of using a defined reflective process involving an MSA tool for building skill in dental students' self-evaluation of caries excavation on extracted teeth. ⋯ The results showed that interrater agreement between the self- and expert assessors increased significantly: from a 28% agreement in formative assessment to a 60% agreement in summative assessment. Significance in percentage shifts between assessments was demonstrated with a McNemar score of 0.26 (p<0.001). These results suggest that the described MSA tool and reflective process in an SGSP may be valid methods for improving skill in student self-evaluation of competence in caries excavation on extracted teeth.
Tracy L de Peralta, Vidya Ramaswamy, Elisabeta Karl, Elizabeth Van Tubergen, Mary Ellen McLean, and Mark Fitzgerald.
Dr. de Peralta is Clinical Associate Professor, Department of Cariology, Restorative Sciences, Endodontics, and Director of Curriculum and Assessment Integration, School of Dentistry, University of Michiga... more n; Dr. Ramaswamy is Associate Director of Curriculum and Program Evaluation, Office of Academic Affairs, School of Dentistry, University of Michigan; Dr. Karl is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. Van Tubergen is Clinical Assistant Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; Dr. McLean is Clinical Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan; and Dr. Fitzgerald is Associate Chair and Associate Professor, Department of Cariology, Restorative Sciences, and Endodontics, School of Dentistry, University of Michigan. tdeperal@umich.edu. less
J Dent Educ. 2017 Jan 1; 81 (1): 87-95.
AbstractMultisource assessment (MSA) uses multiple assessors to provide feedback. Little is known about the validity of using MSA feedback for improving students' ability to self-assess in a preclinical environment. Therefore, the aim of this study was to measure the validity of using a defined reflective process involving an MSA tool for building skill in dental students' self-evaluation of caries excavation on extracted teeth. As part of this process, 104 first-year students at one U.S. dental school used a self-generated study plan (SGSP) for structured reflection on MSA feedback during the 2013-14 academic year. Interrater agreement, determined through calculation of percentage-agreements in scoring, was measured among three assessor groups (self-, peer, and expert assessors) in formative assessment and between two assessor groups (self- and expert assessors) in summative assessment two weeks apart, allowing for reflective practice and completion of an SGSP between assessments. Validity for improving self-assessment was determined by measuring significance in positive shifts of agreement between self- and expert assessors. The results showed that interrater agreement between the self- and expert assessors increased significantly: from a 28% agreement in formative assessment to a 60% agreement in summative assessment. Significance in percentage shifts between assessments was demonstrated with a McNemar score of 0.26 (p<0.001). These results suggest that the described MSA tool and reflective process in an SGSP may be valid methods for improving skill in student self-evaluation of competence in caries excavation on extracted teeth.