Journal of general internal medicine
-
Comparative Study
Comparing Scoring Consistency of Large Language Models with Faculty for Formative Assessments in Medical Education.
The Liaison Committee on Medical Education requires that medical students receive individualized feedback on their self-directed learning skills. Pre-clinical students are asked to complete multiple spaced critical appraisal assignments. However, the individual feedback requires significant faculty time. As large language models (LLMs) can score and generate feedback, we explored their use in grading formative assessments through validity and feasibility lenses. ⋯ This study of psychometric characteristics of ChatGPT demonstrates the potential role for LLMs to assist faculty in assessing and providing feedback for formative assignments.
-
While 26% of US adults are disabled, only 3.1 to 9.3% of practicing physicians report having a disability. Ableism within medical training and practice diminishes physician diversity and wellbeing and contributes to healthcare disparities. ⋯ PDs held contradictory views of RWDs. PD insights revealed opportunities to alleviate PD-RWD information asymmetry in recruitment/accommodation processes, which could help align needs and improve representation and inclusion.
-
Indirect supervision is essential for granting autonomy to learners. Sometimes referred to as leaving the learner "unsupervised," there is growing recognition that learners and supervisors engage in clinical support through ongoing interactions, albeit at a distance. ⋯ Indirect supervision creates clinical support through ongoing communication between learners and supervisors at a distance. It is a collaborative process for mutual reassurance that safe patient care is being provided and that support is available when needed.
-
In the present assessment environment in undergraduate medical education at U.S. medical schools, the prevalence and implementation of Entrustable Professional Activities (EPAs) in internal medicine (IM) clerkships are not well understood. ⋯ Although EPAs have experienced substantial uptake in the IM clerkship and contribute to formative and summative assessment of learners, their use does not appear to be associated with enhanced efforts to obtain validity information.
-
Institutions rely on student evaluations of teaching (SET) to ascertain teaching quality. Manual review of narrative comments can identify faculty with teaching concerns but can be resource and time-intensive. ⋯ NLP methods can identify teaching quality concerns with good accuracy and reasonable recall, but relatively low precision. An existing, free, NLP sentiment analysis dictionary can perform nearly as well as dictionaries requiring expert coding or manual creation.