Anastakis D J, Cohen R, Reznick R K
Toronto Hospital, Department of Surgery, University of Toronto, Ontario, Canada.
Am J Surg. 1991 Jul;162(1):67-70. doi: 10.1016/0002-9610(91)90205-r.
The purpose of this study was to determine the overall reliability, inter-rater reliability, and criterion validity of the structured oral examination (SOE) for assessing surgical residents. An SOE consisting of four predetermined clinically oriented scenarios was administered to 23 second postgraduate year surgical residents. Each scenario had five to six questions, each with a specific marking scheme. Candidates were assessed by two examiners and scores were derived independently. Overall reliability (Cronbach's alpha) was 0.75. Inter-rater reliability was significant for each pair of examiners and each question (r = 0.78 to 0.91: p less than 0.0001). Criterion validity was measured by correlating SOE scores with multiple-choice examination (MCQ) and objective structured clinical examination (OSCE) scores. Correlations between the SOE and MCQ and OSCE were significant and fell into the moderate range (0.48 to 0.51). The results of this study show that the SOE is useful in the assessment of clinical knowledge and problem-solving abilities of the surgical resident. Overall and inter-rater reliabilities achieved exceed those of traditional oral examination formats.
本研究的目的是确定用于评估外科住院医师的结构化口试（SOE）的整体信度、评分者间信度和效标效度。对23名外科二年级住院医师进行了由四个预先确定的临床导向场景组成的SOE。每个场景有五到六个问题，每个问题都有特定的评分方案。由两名考官对考生进行评估，分数独立得出。整体信度（克朗巴哈系数）为0.75。评分者间信度对于每对考官和每个问题都具有显著性（r = 0.78至0.91：p小于0.0001）。通过将SOE分数与多项选择题考试（MCQ）分数和客观结构化临床考试（OSCE）分数进行关联来测量效标效度。SOE与MCQ以及OSCE之间的相关性具有显著性，且处于中等范围（0.48至0.51）。本研究结果表明，SOE在评估外科住院医师的临床知识和解决问题能力方面是有用的。所达到的整体信度和评分者间信度超过了传统口试形式的信度。
