Division of Medical Education, Göttingen University Medical Centre, Göttingen, Germany.
Department of Cognitive and Decision Psychology, Georg-August University Göttingen, Göttingen, Germany.
Med Teach. 2022 Nov;44(11):1253-1259. doi: 10.1080/0142159X.2022.2077716. Epub 2022 Jun 2.
Validation of examinations is usually based on classical test theory. In this study, we analysed a key feature examination according to item response theory and compared the results with those of a classical test theory approach.
Over the course of five years, 805 fourth-year undergraduate students took a key feature examination on general medicine consisting of 30 items. Analyses were run according to a classical test theory approach as well as using item response theory. Classical test theory analyses are reported as item difficulty, discriminatory power, and Cronbach's alpha while item response theory analyses are presented as item characteristics curves, item information curves and a test information function.
According to classical test theory findings, the examination was labelled as easy. Analyses according to item response theory more specifically indicated that the examination was most suited to identify struggling students. Furthermore, the analysis allowed for adapting the examination to specific ability ranges by removing items, as well as comparing multiple samples with varying ability ranges.
Item response theory analyses revealed results not yielded by classical test theory. Thus, both approaches should be routinely combined to increase the information yield of examination data.
考试的验证通常基于经典测试理论。在这项研究中，我们根据项目反应理论分析了一项关键特征考试，并将结果与经典测试理论方法进行了比较。
在五年的时间里，805 名四年级本科生参加了一项包含 30 个项目的普通医学关键特征考试。根据经典测试理论方法和项目反应理论进行了分析。经典测试理论分析报告了项目难度、区分度和克朗巴赫的阿尔法，而项目反应理论分析则呈现了项目特征曲线、项目信息曲线和测试信息函数。
根据经典测试理论的发现，考试被标记为简单。根据项目反应理论的分析更具体地表明，该考试最适合识别学习困难的学生。此外，该分析允许通过删除项目来适应特定的能力范围，以及比较具有不同能力范围的多个样本。
项目反应理论分析揭示了经典测试理论无法得出的结果。因此，应该常规地结合这两种方法，以提高考试数据的信息量。
