Stephenson Christopher R, Bonnes Sara L, Sawatsky Adam P, Richards Lukas W, Schleck Cathy D, Mandrekar Jayawant N, Beckman Thomas J, Wittich Christopher M
Division of General Internal Medicine, Mayo Clinic, 200 First St SW, Rochester, MN, 55905, USA.
Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, MN, USA.
BMC Med Educ. 2020 Nov 4;20(1):403. doi: 10.1186/s12909-020-02331-x.
Continuing medical education (CME) often uses passive educational models including lectures. However, numerous studies have questioned the effectiveness of these less engaging educational strategies. Studies outside of CME suggest that engaged learning is associated with improved educational outcomes. However, measuring participants' engagement can be challenging. We developed and determined the validity evidence for a novel instrument to assess learner engagement in CME.
We conducted a cross-sectional validation study at a large, didactic-style CME conference. Content validity evidence was established through review of literature and previously published engagement scales and conceptual frameworks on engagement, along with an iterative process involving experts in the field, to develop an eight-item Learner Engagement Instrument (LEI). Response process validity was established by vetting LEI items on item clarity and perceived meaning prior to implementation, as well as using a well-developed online platform with clear instructions. Internal structure validity evidence was based on factor analysis and calculating internal consistency reliability. Relations to other variables validity evidence was determined by examining associations between LEI and previously validated CME Teaching Effectiveness (CMETE) instrument scores. Following each presentation, all participants were invited to complete the LEI and the CMETE.
51 out of 206 participants completed the LEI and CMETE (response rate 25%) Correlations between the LEI and the CMETE overall scores were strong (r = 0.80). Internal consistency reliability for the LEI was excellent (Cronbach's alpha = 0.96). To support validity to internal structure, a factor analysis was performed and revealed a two dimensional instrument consisting of internal and external engagement domains. The internal consistency reliabilities were 0.96 for the internal engagement domain and 0.95 for the external engagement domain.
Engagement, as measured by the LEI, is strongly related to teaching effectiveness. The LEI is supported by robust validity evidence including content, response process, internal structure, and relations to other variables. Given the relationship between learner engagement and teaching effectiveness, identifying more engaging and interactive methods for teaching in CME is recommended.
继续医学教育（CME）通常采用包括讲座在内的被动教育模式。然而，众多研究对这些缺乏吸引力的教育策略的有效性提出了质疑。CME以外的研究表明，参与式学习与更好的教育成果相关。然而，衡量参与者的参与度可能具有挑战性。我们开发并确定了一种用于评估CME中学习者参与度的新型工具的效度证据。
我们在一个大型的讲授式CME会议上进行了一项横断面验证研究。通过回顾文献、先前发表的关于参与度的参与度量表和概念框架，以及一个涉及该领域专家的迭代过程，建立内容效度证据，以开发一个包含八个条目的学习者参与度工具（LEI）。通过在实施前审查LEI项目的清晰度和感知意义，以及使用一个具有清晰说明的完善在线平台，建立反应过程效度。内部结构效度证据基于因子分析和计算内部一致性信度。通过检查LEI与先前验证的CME教学效果（CMETE）工具分数之间的关联，确定与其他变量的关系效度证据。在每次演讲后，邀请所有参与者完成LEI和CMETE。
206名参与者中有51名完成了LEI和CMETE（回复率25%）。LEI和CMETE总体分数之间的相关性很强（r = 0.80）。LEI的内部一致性信度非常好（Cronbach's alpha = 0.96）。为了支持内部结构的效度，进行了因子分析，结果显示该工具是一个二维工具，由内部和外部参与领域组成。内部参与领域的内部一致性信度为0.96，外部参与领域的内部一致性信度为0.95。
通过LEI衡量的参与度与教学效果密切相关。LEI得到了包括内容、反应过程、内部结构以及与其他变量的关系等强有力的效度证据的支持。鉴于学习者参与度与教学效果之间的关系，建议在CME中确定更具吸引力和互动性的教学方法。
