Testa Silvia, Toscano Anna, Rosato Rosalba
Department of Psychology, University of Turin, Turin, Italy.
UMR INSERM 1246, SPHERE, Methods in Patient-Centered Outcomes and Health Research, University of Nantes, University of Tours, Nantes, France.
Front Psychol. 2018 Aug 28;9:1585. doi: 10.3389/fpsyg.2018.01585. eCollection 2018.
Multiple-choice items are one of the most commonly used tools for evaluating students' knowledge and skills. A key aspect of this type of assessment is the presence of functioning distractors, i.e., incorrect alternatives intended to be plausible for students with lower achievement. To our knowledge, no work has investigated the relationship between distractor performance and the complexity of the cognitive task required to give the correct answer. The aim of this study was to investigate this relation, employing the first three levels of Bloom's taxonomy (Knowledge, Comprehension, and Application). Specifically, it was hypothesized that items classified into a higher level of Bloom's classification would show a greater number of functioning distractors. The study involved 174 items administered to a sample of 848 undergraduate psychology students during their statistics exam. Each student received 30 items randomly selected from the 174-item pool. The bivariate results mainly supported the authors' hypothesis: the highest percentage of functioning distractors was observed among the items classified into the Application category (η = 0.024 and Phi = 0.25 for the dichotomized measure). When the analysis controlled for other item features, it lost statistical significance, partly because of the confounding effect of item difficulty.
多项选择题是评估学生知识和技能最常用的工具之一。这类评估的一个关键方面是要有有效的干扰项，即那些对于成绩较低的学生来说看似合理的错误选项。据我们所知，尚无研究探讨干扰项的表现与给出正确答案所需认知任务的复杂性之间的关系。本研究的目的是利用布鲁姆分类法的前三个层次（知识、理解和应用）来探究这种关系。具体而言，研究假设是，被归类到布鲁姆分类法较高层次的题目会有更多有效的干扰项。该研究涉及在统计学考试期间向848名本科心理学专业学生的样本发放174道题目。每个学生随机收到从174道题目的题库中选出的30道题目。双变量结果主要支持了作者的假设：在归类到应用类别的题目中观察到的有效干扰项比例最高（二分变量测量的η = 0.024，Phi = 0.25）。当分析控制了其他题目特征时，该结果失去了统计学意义，部分原因是题目难度的混杂效应。
