Office of Educational Affairs, University of Massachusetts Medical School, Worcester, Massachusetts 01655-0002, USA.
Teach Learn Med. 2010 Oct;22(4):274-9. doi: 10.1080/10401334.2010.512542.
Assessment of professionalism in undergraduate medical education is challenging. One approach that has not been well studied in this context is performance-based examinations.
This study sought to investigate the reliability of standardized patients' scores of students' professionalism in performance-based examinations.
Twenty students were observed on 4 simulated cases involving professional challenges; 9 raters evaluated each encounter on 21 professionalism items. Correlational and multivariate generalizability (G) analyses were conducted.
G coefficients were .75, .53, and .68 for physicians, standardized patients (SPs), and lay raters, respectively. Composite G coefficient for all raters reached acceptable level of .86. Results indicated SP raters were more variable than other rater types in severity with which they rated students, although rank ordering of students was consistent among SPs.
SPs' ratings were less reliable and consistent than physician or lay ratings, although the SPs rank ordered students more consistently than the other rater types.
评估本科医学教育中的专业精神具有挑战性。在这种情况下，尚未充分研究的一种方法是基于表现的考试。
本研究旨在调查标准化患者在基于表现的考试中学生专业精神评分的可靠性。
对 20 名学生进行了 4 个涉及专业挑战的模拟案例观察；9 名评估者对每个案例的 21 项专业项目进行评估。进行了相关性和多变量可推广性（G）分析。
医生、标准化患者（SP）和非专业评估者的 G 系数分别为.75、.53 和.68。所有评估者的综合 G 系数达到可接受的.86 水平。结果表明，SP 评估者在评估学生的严重程度方面比其他评估者类型更具变异性，尽管 SP 对学生的排序比其他评估者类型更一致。
尽管 SP 比医生或非专业评估者更一致地对学生进行排名，但 SP 的评分不如医生或非专业评估者可靠和一致。
