Niehaus D J, Jordaan E, Koen L, Mashile M, Mall S
SELSUS Centre, Department of Psychiatry, University of Stellenbosch, South Africa.
Afr J Psychiatry (Johannesbg). 2012 Mar;15(2):119-23. doi: 10.4314/ajpsy.v15i2.16.
There are several methods of evaluating medical students' performance, such as written examination, oral examination and objective structured clinical examination (OSCE). Many studies have focused on the reliability and validity of these methods but few studies have explored comparison between these methods. Psychiatry is the only subject at the University of Stellenbosch where the final assessment consists of solely an oral component. The aim of the study was to compare students' final overall and discipline specific examination marks (i.e. in the other subjects) with the examination marks in psychiatry, and to determine if content or structure (e,g. oral, written or OSCE format) of examination impacts more on the student performance in the examination.
343 final year medical students were included. All undertook their psychiatry rotation at the University of Stellenbosch, South Africa during 2008 and 2009. Data of marks obtained in all the disciplines during 2008 and 2009 were collected and class marks were compared with their final examination marks across all disciplines. Bland-Altman plots were used to assess the level of agreement between the class and examination marks. Cases below the lower threshold were compared to all other cases across all disciplines. The odds ratio for group status was calculated for gender distribution of examiners.
The psychiatry class mark and final oral examination mark provided similar measures within a width of 31.5. Cases below the threshold had poorer performance in two other disciplines. The gender distribution of the examiners (female-female) significantly increased the odds ratio for poorer performance in the oral examination.
The results suggest that a group of students underperform in their final examination independent of method of evaluation and that the gender of examiners impacts on examination marks. Therefore future research should focus on identifying and modifying factors (including choice of examiner combinations) that contribute to the poor performance of medical students in their final examination, in order to help students perform better. Gender distribution of examiners should also be considered when examinations are structured and designed.
评估医学生成绩有多种方法，如笔试、口试和客观结构化临床考试（OSCE）。许多研究聚焦于这些方法的可靠性和有效性，但很少有研究探讨这些方法之间的比较。精神病学是斯泰伦博斯大学唯一最终评估仅由口试组成的学科。本研究的目的是比较学生的最终总成绩和特定学科考试成绩（即其他学科的成绩）与精神病学考试成绩，并确定考试的内容或结构（如口试、笔试或OSCE形式）对学生考试成绩的影响是否更大。
纳入343名医学专业最后一年的学生。他们于2008年和2009年在南非斯泰伦博斯大学进行精神病学轮转。收集了2008年和2009年所有学科获得的成绩数据，并将班级成绩与所有学科的期末考试成绩进行比较。采用布兰德-奥特曼图评估班级成绩与考试成绩之间的一致性水平。将低于较低阈值的情况与所有学科的所有其他情况进行比较。计算考官性别分布的组状态优势比。
精神病学班级成绩和最终口试成绩在31.5的范围内提供了相似的衡量标准。低于阈值的学生在其他两个学科中的表现较差。考官的性别分布（女性-女性）显著增加了口试成绩较差的优势比。
结果表明，一组学生在期末考试中的表现不佳，与评估方法无关，并且考官的性别会影响考试成绩。因此，未来的研究应专注于识别和改变导致医学生期末考试成绩不佳的因素（包括考官组合的选择），以帮助学生表现得更好。在组织和设计考试时，也应考虑考官的性别分布。
