Turnbull J, MacFadyen J, Van Barneveld C, Norman G
Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
J Gen Intern Med. 2000 Aug;15(8):556-61. doi: 10.1046/j.1525-1497.2000.06099.x.
Existing systems of in-training evaluation (ITE) have been criticized as being unreliable and invalid methods for assessing student performance during clinical education. The purpose of this study was to assess the feasibility, reliability, and validity of a clinical work sampling (CWS) approach to ITE. This approach focused on the following: (1) basing performance data on observed behaviors, (2) using multiple observers and occasions, (3) recording data at the time of performance, and (4) allowing for a feasible system to receive feedback.
Sixty-two third-year University of Ottawa students were assessed during their 8-week internal medicine inpatient experience.
Four performance rating forms (Admission Rating Form, Ward Rating Form, Multidisciplinary Team Rating Form, and Patient's Rating Form) were introduced to document student performance. Voluntary participation rates were variable (12%-64%) with patients excluded from the analysis because of low response rate (12%). The mean number of evaluations per student per rotation (19) exceeded the number of evaluations needed to achieve sufficient reliability. Reliability coefficients were high for the Ward Form (.86) and the Admission Form (.73) but not for the Multidisciplinary Team (.22) Form. There was an examiner effect (rater leniency), but this was small relative to real differences between students. Correlations between the Ward Form and the Admission Form were high (.47), while those with the Multidisciplinary Team Form were lower (.37 and .26, respectively). The CWS approach ITE was considered to be content valid by expert judges.
The collection of ongoing performance data was reasonably feasible, reliable, and valid.
现行的住院医师培训评估（ITE）系统被批评为评估临床教育阶段学生表现的不可靠和无效方法。本研究旨在评估临床工作抽样（CWS）方法用于ITE的可行性、可靠性和有效性。该方法着重于以下几点：（1）基于观察到的行为获取表现数据；（2）使用多名观察者并在多个场合进行观察；（3）在表现发生时记录数据；（4）建立一个可行的系统以接收反馈。
62名渥太华大学三年级学生在其8周的内科住院医师实习期间接受了评估。
引入了四种表现评分表（入院评分表、病房评分表、多学科团队评分表和患者评分表）来记录学生表现。自愿参与率各不相同（12% - 64%），由于患者回应率较低（12%），故将其排除在分析之外。每名学生每次轮转的平均评估次数（19次）超过了达到足够可靠性所需的评估次数。病房评分表（.86）和入院评分表（.73）的可靠性系数较高，但多学科团队评分表（.22）的可靠性系数不高。存在考官效应（评分宽松），但相对于学生之间的实际差异而言较小。病房评分表与入院评分表之间的相关性较高（.47），而与多学科团队评分表的相关性较低（分别为.37和.26）。专家评委认为CWS方法用于ITE具有内容效度。
持续收集表现数据具有合理的可行性、可靠性和有效性。
