Kroboth F J, Hanusa B H, Parker S, Coulehan J L, Kapoor W N, Brown F H, Karpf M, Levey G S
Division of General Internal Medicine, University of Pittsburgh, Pennsylvania 15261.
J Gen Intern Med. 1992 Mar-Apr;7(2):174-9. doi: 10.1007/BF02598008.
To assess the internal consistency and inter-rater reliability of a clinical evaluation exercise (CEX) format that was designed to be easily utilized, but sufficiently detailed, to achieve uniform recording of the observed examination.
A comparison of 128 CEXs conducted for 32 internal medicine interns by full-time faculty. This paper reports alpha coefficients as measures of internal consistency and several measures of inter-rater reliability.
A university internal medicine program. Observations were conducted at the end of the internship year.
Participants were 32 interns and observers were 12 full-time faculty in the department of medicine. The entire intern group was chosen in order to optimize the spectrum of abilities represented. Patients used for the study were recruited by the chief resident from the inpatient medical service based on their ability and willingness to participate.
Each intern was observed twice and there were two examiners during each CEX. The examiners were given a standardized preparation and used a format developed over five years of previous pilot studies.
The format appeared to have excellent internal consistency; alpha coefficients ranged from 0.79 to 0.99. However, multiple methods of determining inter-rater reliability yielded similar results; intraclass correlations ranged from 0.23 to 0.50 and generalizability coefficients from a low of 0.00 for the overall rating of the CEX to a high of 0.61 for the physical examination section. Transforming scores to eliminate rater effects and dichotomizing results into pass-fail did not appear to enhance the reliability results.
Although the CEX is a valuable didactic tool, its psychometric properties preclude reliable assessment of clinical skills as a one-time observation.
评估一种临床评估练习（CEX）形式的内部一致性和评分者间信度，该形式旨在易于使用，但细节充分，以实现对观察到的检查进行统一记录。
对全职教员为32名内科实习医生进行的128次CEX进行比较。本文报告了作为内部一致性度量的阿尔法系数和几种评分者间信度度量。
一所大学的内科项目。观察在实习年结束时进行。
参与者为32名实习医生，观察者为医学系的12名全职教员。选择整个实习医生群体是为了优化所代表的能力范围。用于该研究的患者由总住院医师根据其参与的能力和意愿从住院医疗服务中招募。
每位实习医生被观察两次，每次CEX有两名考官。考官接受了标准化培训，并使用了经过五年前期试点研究开发的形式。
该形式似乎具有出色的内部一致性；阿尔法系数范围为0.79至0.99。然而，多种确定评分者间信度的方法得出了相似的结果；组内相关性范围为0.23至0.50，可推广性系数从CEX总体评分的低0.00到体格检查部分的高0.61。转换分数以消除评分者效应并将结果二分化为通过/失败似乎并未提高信度结果。
尽管CEX是一种有价值的教学工具，但其心理测量特性排除了将其作为一次性观察对临床技能进行可靠评估的可能性。
