Auewarakul Chirayu, Downing Steven M, Praditsuwan Rungnirand, Jaturatamrong Uapong
Department of Medicine, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand.
Adv Health Sci Educ Theory Pract. 2005;10(2):105-13. doi: 10.1007/s10459-005-2315-3.
Utilization of objective structured clinical examinations (OSCEs) for final assessment of medical students in Internal Medicine requires a representative sample of OSCE stations. The reliability and generalizability of OSCE scores provides validity evidence for OSCE scores and supports its contribution to the final clinical grade of medical students. The objective of this study was to perform item analysis using OSCE stations as the unit of analysis and evaluate the extent to which OSCE score reliability can be improved using item analysis data. OSCE scores from eight cohorts of fourth-year medical students (n = 435) in a 6-year undergraduate program were analyzed. Generalizability (G) coefficients of OSCE scores were computed for each cohort. Item analysis was performed by considering each OSCE station as an item and computing the corrected item-total correlation. OSCE stations which negatively impacted the reliability were deleted and the G-coefficient was recalculated. The G-coefficients of OSCE scores from the eight cohorts ranged from 0.48 to 0.80 (median 0.62). The median number of OSCE stations that negatively impacted the G-coefficient was 3.5 (out of a median of 25 total stations). When the ''problem stations'' were deleted, the median G-coefficient across eight cohorts increased to 0.62--0.72. In conclusion, item analysis of OSCE stations is useful and should be performed to improve the reliability of total OSCE scores. Problem stations can then be identified and improved.
使用客观结构化临床考试（OSCE）对医学生内科实习进行最终评估需要有代表性的OSCE考站样本。OSCE分数的可靠性和可推广性为OSCE分数提供了效度证据，并支持其对医学生最终临床成绩的贡献。本研究的目的是以OSCE考站为分析单位进行项目分析，并评估利用项目分析数据可在多大程度上提高OSCE分数的可靠性。对一个6年制本科项目中八届四年级医学生（n = 435）的OSCE分数进行了分析。计算了每个队列的OSCE分数的概化（G）系数。通过将每个OSCE考站视为一个项目并计算校正后的项目与总分相关性来进行项目分析。删除对可靠性有负面影响的OSCE考站，并重新计算G系数。八届学生的OSCE分数的G系数范围为0.48至0.80（中位数为0.62）。对G系数有负面影响的OSCE考站的中位数为3.5个（总考站中位数为25个）。当删除“问题考站”后，八届学生的G系数中位数提高到0.62 - 0.72。总之，对OSCE考站进行项目分析是有用的，应该进行以提高OSCE总分的可靠性。然后可以识别并改进问题考站。
