Journal of evaluation in clinical practice
-
Programmatic assessment has been identified as a system-oriented approach to achieving the multiple purposes for assessment within Competency-Based Medical Education (CBME, i.e., formative, summative, and program improvement). While there are well-established principles for designing and evaluating programs of assessment, few studies illustrate and critically interpret, what a system of programmatic assessment looks like in practice. This study aims to use systems thinking and the 'two communities' metaphor to interpret a model of programmatic assessment and to identify challenges and opportunities with operationalization. ⋯ The findings of this research suggest that program stakeholders can benefit from a systems perspective regarding how their assessment practices contribute to the efficacy of the system as a whole. Academic Advisors are well positioned to support educational development efforts focused on overcoming challenges with operationalizing programmatic assessment.
-
In 2015, Queen's University embarked on an institution-wide transition to a competency-based medical education (CBME) curriculum for all 29 postgraduate medical education programmes. On 1 July 2017, this goal was accomplished. With this mass transition came the requirement to assess the efficacy of implementation through a programme evaluation process, which included the use of outcome harvesting (Wilson-Grau). Outcome harvesting identified the intended and unintended outcomes of CBME implementation, which helped us understand how the intervention was achieved and how the relationship between behaviours and stakeholders contributed to the successful transition. ⋯ Using outcome harvesting to assess the efficacy of CBME implementation produced a robust set of themes and resultant outcomes that can be categorized as requirements for success of implementation of any curricular innovation. Emergent themes included collaboration, community of practice, and stakeholder commitment. More unique observations noted through the harvest process included new policy development, creation of learner ownership, and an increase in the output of scholarly activity involving CBME.
-
Effective treatment of depression involves collaboration with informed patients and families and appropriate knowledge sharing. We describe here our experience, as a case example, of a collaboration to "translate" a clinical guideline designed for practitioners into an accessible, plainlanguage version that patients and families can use during the care process, both to provide basic educational information and to foster informed discussions with their treatment providers. Content experts in knowledge translation, patient advocacy, patient-oriented research, and psychiatry guided overall project design. ⋯ Both during and at the end of the project, writers provided additional recommendations for improving the process, including more in-person meetings, distribution of step-by-step instructions on the writing task, and a key terms glossary of technical terms to support their role. In conclusion, we describe a process with practical tips and reflective feedback on important considerations for engaging persons with lived experience as leaders in the codesign and writing process of lay treatment guidelines. These methods may serve as a model for similar projects in other areas of healthcare.
-
Within competency-based medical education, self-regulated learning (SRL) requires residents to leverage self-assessment and faculty feedback. We sought to investigate the potential for competency-based assessments to foster SRL by quantifying the relationship between faculty feedback and entrustment ratings as well as the congruence between faculty assessment and resident self-assessment. ⋯ Minimal and inconsistent associations were found between narrative and numerical feedback. Performance monitoring accuracy and feedback should be included in assessment validation.