Paul Caroline R, Ryan Michael S, Dallaghan Gary L Beck, Jirasevijinda Thanakorn, Quigley Patricia D, Hanson Janice L, Khidir Amal M, Petershack Jean, Jackson Joseph, Tewksbury Linda, Rocha Mary Esther M
Assistant Professor, Department of Pediatrics, University of Wisconsin School of Medicine and Public Health.
Associate Professor, Department of Pediatrics, Virginia Commonwealth University School of Medicine.
MedEdPORTAL. 2019 Apr 12;15:10817. doi: 10.15766/mep_2374-8265.10817.
There is an increasing call for developing validity evidence in medical education assessment. The literature lacks a practical resource regarding an actual development process. Our workshop teaches how to apply principles of validity evidence to existing assessment instruments and how to develop new instruments that will yield valid data.
The literature, consensus findings of curricula and content experts, and principles of adult learning guided the content and methodology of the workshop. The workshop underwent stringent peer review prior to presentation at one international and three national academic conferences. In the interactive workshop, selected domains of validity evidence were taught with sequential cycles of didactics, demonstration, and deliberate practice with facilitated feedback. An exercise guide steered participants through a stepwise approach. Using Likert-scale items and open-response questions, an evaluation form rated the workshop's effectiveness, captured details of how learners reached the objectives, and determined participants' plans for future work.
The workshop demonstrated generalizability with successful implementation in diverse settings. Sixty-five learners, the majority being clinician-educators, completed evaluations. Learners rated the workshop favorably for each prompt. Qualitative comments corroborated the workshop's effectiveness. The active application and facilitated feedback components allowed learners to reflect in real time as to how they were meeting a particular objective.
This feasible and practical educational intervention fills a literature gap by showing the medical educator how to apply validity evidence to both existing and in-development assessment instruments. Thus, it holds the potential to significantly impact learner and, subsequently, patient outcomes.
在医学教育评估中，对制定效度证据的呼声日益高涨。文献中缺乏关于实际开发过程的实用资源。我们的工作坊教授如何将效度证据原则应用于现有的评估工具，以及如何开发能够产生有效数据的新工具。
文献、课程和内容专家的共识结果以及成人学习原则指导了工作坊的内容和方法。在一次国际学术会议和三次全国学术会议上展示之前，该工作坊经过了严格的同行评审。在互动式工作坊中，通过教学、示范和有指导反馈的刻意练习的连续循环，教授选定的效度证据领域。一份练习指南引导参与者采用逐步推进的方法。使用李克特量表项目和开放式问题，一份评估表对工作坊的有效性进行评分，记录学习者实现目标的详细情况，并确定参与者未来的工作计划。
该工作坊在不同环境中成功实施，证明了其可推广性。65名学习者完成了评估，其中大多数是临床教育工作者。学习者对每个提示都给予了好评。定性评论证实了工作坊的有效性。积极应用和有指导反馈环节让学习者能够实时反思他们如何实现特定目标。
这种可行且实用的教育干预通过向医学教育工作者展示如何将效度证据应用于现有和正在开发的评估工具，填补了文献空白。因此，它有可能对学习者以及随后的患者结果产生重大影响。
