Jhpiego, Bahir Dar, Ethiopia.
Jhpiego, Addis Ababa, Ethiopia.
BMC Med Educ. 2022 Aug 22;22(1):635. doi: 10.1186/s12909-022-03687-y.
Assessment of cognitive competence is a major element of the internship qualification exam in undergraduate medical education in Ethiopia. Assessing the quality of exam items can help to improve the validity of assessments and assure stakeholders about the accuracy of the go/no decision to the internship. However, we know little about the quality of exam items utilized to ascertain fitness to join the medical internship. Therefore, this study aimed to analyze the quality of multiple-choice questions (MCQs) of the qualification exam administered to final-year medical students at Debre Tabor University (DTU), Ethiopia.
A psychometric study was conducted to assess the qualities of 120 randomly selected MCQs and 407 distractors. Item characteristics were estimated using the item response theory (IRT) model. T-test, one-way ANOVA, and chi-square tests were run to analyze the univariate association between factors. Pearson's correlation test was done to determine the predictive validity of the qualification examination.
Overall, 16, 51, and 33% of the items had high, moderate, and low distractor efficiency, respectively. About two-thirds (65.8%) of the items had two or more functioning distractors and 42.5% exhibited a desirable difficulty index. However, 77.8% of items administered in the qualification examination had a negative or poor discrimination index. Four and five option items didn't show significant differences in psychometric qualities. The qualification exam showed a positive predictive value of success in the national licensing examination (Pearson's correlation coefficient = 0.5).
The psychometric properties of the medical qualification exam were inadequate for making valid decisions. Five option MCQs were not better than four options in terms of psychometric qualities. The qualification examination had a positive predictive validity of future performance. High-stakes examination items must be properly created and reviewed before being administered.
在埃塞俄比亚的本科医学教育中，评估认知能力是实习资格考试的主要内容。评估考试项目的质量有助于提高评估的有效性，并向利益相关者保证实习的去/留决策的准确性。然而，我们对用于确定是否适合参加医学实习的考试项目的质量知之甚少。因此，本研究旨在分析德布雷塔博尔大学（DTU）最后一年医学生资格考试中的多项选择题（MCQ）的质量。
进行了一项心理测量学研究，以评估 120 道随机选择的 MCQ 和 407 道干扰项的质量。使用项目反应理论（IRT）模型评估项目特征。进行 t 检验、单因素方差分析和卡方检验，以分析因素之间的单变量关联。进行皮尔逊相关检验，以确定资格考试的预测有效性。
总体而言，16%、51%和 33%的项目具有高、中、低干扰效率，分别。大约三分之二（65.8%）的项目有两个或更多功能干扰项，42.5%的项目表现出理想的难度指数。然而，在资格考试中，77.8%的项目的辨别指数为负或较差。四选项和五选项项目在心理测量质量方面没有显著差异。资格考试与国家执照考试的成功具有正预测值（皮尔逊相关系数=0.5）。
医学资格考试的心理测量性质不足以做出有效决策。在心理测量质量方面，五选项 MCQ 并不优于四选项。资格考试对未来表现具有正预测效度。高风险考试项目必须在实施前进行适当的创建和审查。
