K.J. Ferguson is professor of general internal medicine, Department of Internal Medicine, and director, Office of Consultation and Research in Medical Education, University of Iowa Carver College of Medicine, Iowa City, Iowa; ORCID: http://orcid.org/0000-0003-1605-0611. C.D. Kreiter is professor of family medicine, Department of Family Medicine, and consultant, Office of Consultation and Research in Medical Education, University of Iowa Carver College of Medicine, Iowa City, Iowa; ORCID: http://orcid.org/0000-0002-8303-2387. T.H. Haugen is director, Pathology and Laboratory Service, Veterans Administration Medical Center, and associate professor of pathology-clinical pathology, University of Iowa Carver College of Medicine, Iowa City, Iowa. F.R. Dee is professor (emeritus), Department of Pathology, University of Iowa Carver College of Medicine, Iowa City, Iowa.
Acad Med. 2018 Aug;93(8):1146-1149. doi: 10.1097/ACM.0000000000002184.
As medical schools move from discipline-based courses to more integrated approaches, identifying assessment tools that parallel this change is an important goal.
The authors describe the use of test item statistics to assess the reliability and validity of web-enabled mechanistic case diagrams (MCDs) as a potential tool to assess students' ability to integrate basic science and clinical information. Students review a narrative clinical case and construct an MCD using items provided by the case author. Students identify the relationships among underlying risk factors, etiology, pathogenesis and pathophysiology, and the patients' signs and symptoms. They receive one point for each correctly identified link.
In 2014-2015 and 2015-2016, case diagrams were implemented in consecutive classes of 150 medical students. The alpha reliability coefficient for the overall score, constructed using each student's mean proportion correct across all cases, was 0.82. Discrimination indices for each of the case scores with the overall score ranged from 0.23 to 0.51. In a G study using those students with complete data (n = 251) on all 16 cases, 10% of the variance was true score variance, and systematic case variance was large. Using 16 cases generated a G coefficient (relative score reliability) equal to 0.72 and a Phi equal to 0.65.
The next phase of the project will involve deploying MCDs in higher-stakes settings to determine whether similar results can be achieved. Further analyses will determine whether these assessments correlate with other measures of higher-order thinking skills.
随着医学院校从基于学科的课程转向更加综合的方法，确定与之相匹配的评估工具是一个重要目标。
作者描述了使用测试项目统计数据来评估基于网络的机械病例图（MCD）的可靠性和有效性，作为评估学生整合基础科学和临床信息能力的潜在工具。学生审查一个临床案例叙述，并使用案例作者提供的项目构建一个 MCD。学生识别潜在风险因素、病因、发病机制和病理生理学以及患者的症状和体征之间的关系。他们为每个正确识别的链接获得一分。
在 2014-2015 年和 2015-2016 年，病例图在连续两个 150 名医学生班级中实施。使用每个学生在所有案例中的平均正确比例构建的总分的 alpha 可靠性系数为 0.82。每个案例得分与总分的鉴别指数范围为 0.23 至 0.51。在一项使用所有 16 个案例的完整数据（n = 251）的 G 研究中，10%的方差是真实分数方差，系统案例方差很大。使用 16 个案例生成的 G 系数（相对分数可靠性）等于 0.72，Phi 等于 0.65。
该项目的下一阶段将涉及在高风险环境中部署 MCD，以确定是否可以取得类似的结果。进一步的分析将确定这些评估是否与其他高阶思维技能的衡量标准相关。
