Pharmacy Team-Based Learning Programme, Skaggs School of Pharmacy and Pharmaceutical Sciences, School of Medicine, University of California San Diego (UCSD), La Jolla, California, USA.
Med Educ. 2014 Oct;48(10):1020-7. doi: 10.1111/medu.12525.
Despite significant evidence supporting the use of three-option multiple-choice questions (MCQs), these are rarely used in written examinations for health professions students. The purpose of this study was to examine the effects of reducing four- and five-option MCQs to three-option MCQs on response times, psychometric characteristics, and absolute standard setting judgements in a pharmacology examination administered to health professions students.
We administered two versions of a computerised examination containing 98 MCQs to 38 Year 2 medical students and 39 Year 3 pharmacy students. Four- and five-option MCQs were converted into three-option MCQs to create two versions of the examination. Differences in response time, item difficulty and discrimination, and reliability were evaluated. Medical and pharmacy faculty judges provided three-level Angoff (TLA) ratings for all MCQs for both versions of the examination to allow the assessment of differences in cut scores.
Students answered three-option MCQs an average of 5 seconds faster than they answered four- and five-option MCQs (36 seconds versus 41 seconds; p = 0.008). There were no significant differences in item difficulty and discrimination, or test reliability. Overall, the cut scores generated for three-option MCQs using the TLA ratings were 8 percentage points higher (p = 0.04).
The use of three-option MCQs in a health professions examination resulted in a time saving equivalent to the completion of 16% more MCQs per 1-hour testing period, which may increase content validity and test score reliability, and minimise construct under-representation. The higher cut scores may result in higher failure rates if an absolute standard setting method, such as the TLA method, is used. The results from this study provide a cautious indication to health professions educators that using three-option MCQs does not threaten validity and may strengthen it by allowing additional MCQs to be tested in a fixed amount of testing time with no deleterious effect on the reliability of the test scores.
尽管有大量证据支持使用三选项多项选择题（MCQ），但这些在健康专业学生的书面考试中很少使用。本研究的目的是考察将四选项和五选项 MCQ 减少为三选项 MCQ 对健康专业学生药理学考试的反应时间、心理测量学特征和绝对标准设定判断的影响。
我们向 38 名二年级医学生和 39 名三年级药学学生施测了两个版本的计算机化考试，每个版本都包含 98 个 MCQ。为了创建两个考试版本，我们将四选项和五选项 MCQ 转换为三选项 MCQ。评估了反应时间、项目难度和区分度以及可靠性的差异。医学和药学教师评委对两个版本考试的所有 MCQ 进行了三级安戈夫（TLA）评分，以评估考试分数的差异。
学生回答三选项 MCQ 的平均时间比回答四选项和五选项 MCQ 快 5 秒（36 秒对 41 秒；p=0.008）。项目难度和区分度或测试可靠性没有显著差异。总体而言，使用 TLA 评分生成的三选项 MCQ 的分数高 8 个百分点（p=0.04）。
在健康专业考试中使用三选项 MCQ 可节省 16%的时间，相当于在 1 小时的测试期间完成了 16%的更多 MCQ，这可能会增加内容有效性和测试分数的可靠性，并最大程度减少结构代表性不足。如果使用绝对标准设定方法（如 TLA 方法），则较高的分数可能会导致更高的失败率。本研究的结果为健康专业教育者提供了一个谨慎的提示，即使用三选项 MCQ 不会威胁到有效性，并且通过允许在固定的测试时间内测试更多的 MCQ，可能会增强其有效性，而不会对测试分数的可靠性产生不利影响。
