多项选择题格式的比较评估。引入知识评分。

Comparative evaluation of multiple choice question formats. Introducing a knowledge score.

作者信息

Rahim Sheikh I, Abumadini Mahdi S

机构信息

Department of Psychiatry, King Fahd Hospital of the University, College of Medicine, King Faisal University, PO Box 10401, Al-Khobar 31952, Kingdom of Saudi Arabia. Tel. +966 (3) 8957911. Fax. +966 (3) 8993996. E-mail:

出版信息

Neurosciences (Riyadh). 2003 Jul;8(3):156-60.

PMID:23649110

Abstract

OBJECTIVE

Over the last 6 years different multiple choice question (MCQ) formats have been used in postgraduate examinations for trainees in psychiatry. In phase 1 - K-type True/False (T/F) items with negative marking; in phase 2 combined T/F and type-A one-best answer (OBA) questions without negative marking; in phase 3 exclusively OBA without negative marking. The study compares the gross scores (GS) obtained with different MCQ formats, and introduces knowledge score (KS).

METHODS

The study was conducted in the Saudi Council for Health Specialties, Riyadh, Kingdom of Saudi Arabia from 1996 to 2002. The mean percentile scores obtained by all postgraduate trainees sitting any Part I or Part II Saudi Board Examination in Psychiatry were subjected to a comparative analysis.

RESULTS

A total of 110 candidates sat 18 examinations returning 143 papers containing a total of 32,375 MCQ options. Phase 1 generated lowest overall mean GS (47.8%), phase 3 occupied an intermediate position (53.1%) and phase 2 produced the highest score (68.3%). The KS, to the contrary, generated strikingly similar results for all the 3 phases (47.8, 50.5 and 49.5%) indicating that the marked differences in the GS were probably related to benefits obtained from guessing in the absence of negative marking. In this respect, the OBA produced considerably higher KS scores than the T/F, presumably due to its facilitating extra benefits from cueing, partial knowledge and judgement.

CONCLUSION

Different MCQ formats generate dissimilar quantitative results. The OBA format seems superior to the T/F format in crediting judgement and application of knowledge. In non-negatively marked MCQ tests, the suggested KS provides results comparable to those of negatively marked tests. Pass marks in MCQ tests should be calibrated according to the used format.

摘要

目的

在过去6年里，精神病学研究生考试采用了不同的多项选择题（MCQ）形式。第一阶段采用K型是非题（T/F）并实行倒扣分制；第二阶段采用是非题与A型最佳选择题（OBA）相结合且无倒扣分制；第三阶段只采用无倒扣分制的最佳选择题。本研究比较了不同MCQ形式获得的总分（GS），并引入了知识分数（KS）。

方法

本研究于1996年至2002年在沙特阿拉伯王国利雅得的沙特卫生专业委员会进行。对所有参加沙特精神病学委员会第一部分或第二部分考试的研究生学员获得的平均百分制分数进行比较分析。

结果

共有110名考生参加了18场考试，提交了143份试卷，其中共有32375个MCQ选项。第一阶段产生的总体平均GS最低（47.8%），第三阶段处于中间位置（53.1%），第二阶段得分最高（68.3%）。相反，KS在所有三个阶段产生的结果惊人地相似（47.8%、50.5%和49.5%），这表明GS的显著差异可能与在无倒扣分制情况下猜测获得的好处有关。在这方面，最佳选择题产生的KS分数比是非题高得多，大概是因为它从线索提示、部分知识和判断中获得了额外的好处。