Evaluating the multiple-choice questions quality at the College of Medicine, University of Bisha, Saudi Arabia: a three-year experience.

BACKGROUND

Assessment is a central tool that drives and shapes students learning. Multiple choice questions (MCQs) are central in medical education assessment because they evaluate knowledge across large cohorts. Good quality items will help to achieve the learning objectives and provide trustful results. This study aims to evaluate the quality of MCQs utilized in the final exams of the Principal of Diseases (PRD) course over three academic years at the College of Medicine at The University of Bisha, Saudi Arabia.

METHOD

This cross-sectional institutional-based study used the final exams from the PRD course for the academic years 2016-2019. It was conducted at the College of Medicine, University of Bisha (UBCOM), Saudi Arabia (SA). The analysis process used item analysis (IA) of the PRD final theoretical examinations of the 2016-2017, 2017-2018, and 2018-2019 academic years. 80, 70, and 60 MCQ items were used per test in the above-mentioned years, respectively (210 total). The IA targets the reliability (KR20), difficulty index (DIF), discrimination index (DI), and distractor effectiveness (DE). The generated data were analyzed using SPSS (version 25.0), and statistical significance was set at P < 0.05.

RESULTS

The exams included 210 items. The reliability (KR20) ranged from 0.804 to 0.906. The DI indicated that 56.7% of items were excellent, 20.9% were good, 13.8% were poor, and 8.6% were defective. The DIF showed that 50.5% of items had acceptable difficulty, 37.6% were easy, and 11.9% were difficult. DE analysis revealed that 70.2% of distractors were functional, with a significant correlation between DI, DIF, and DE (P < 0.05).

CONCLUSION

Most of the examined items exhibited excellent discrimination and acceptable difficulty, with 70.2% having functional distractors, categorizing them as high-quality and well-constructed items. The study accentuates the importance of continuous item analysis to maintain and improve the quality of assessment tools used in medical education.