Daelmans H E M, van der Hem-Stokroos H H, Hoogenboom R J I, Scherpbier A J J A, Stehouwer C D A, van der Vleuten C P M
Department of Skills Training, Vrije Universiteit Medical Centre, Amsterdam, Netherlands.
Neth J Med. 2005 Jul-Aug;63(7):279-84.
Global performance rating is frequently used in clinical training despite its known psychometric drawbacks. Inter-rater reliability is low in undergraduate training but better in residency training, possibly because residency offers more opportunities for supervision. The low or moderate predictive validity of global performance ratings in undergraduate and residency training may be due to low or unknown reliability of both global performance ratings and criterion measures. In an undergraduate clerkship, we investigated whether reliability improves when raters are more familiar with students' work and whether validity improves with increased reliability of the predictor and criterion instrument.
Inter-rater reliability was determined in a clerkship with more student-rater contacts than usual. The in-training assessment programme of the clerkship that immediately followed was used as the criterion measure to determine predictive validity.
With four ratings, inter-rater reliability was 0.41 and predictive validity was 0.32. Reliability was lower and validity slightly higher than similar results published for residency training.
Even with increased student-rater interaction, the reliability and validity of global performance ratings were too low to warrant the usage of global performance ratings as individual assessment format. However, combined with other assessment measures, global performance ratings may lead to improved integral assessment.
尽管全球绩效评级存在已知的心理测量缺陷，但在临床培训中仍经常使用。本科培训中的评分者间信度较低，但住院医师培训中的信度较好，这可能是因为住院医师培训提供了更多的监督机会。本科和住院医师培训中全球绩效评级的预测效度较低或中等，可能是由于全球绩效评级和标准测量的信度较低或未知。在一次本科临床实习中，我们调查了评分者对学生工作更熟悉时信度是否提高，以及预测指标和标准工具的信度提高时效度是否提高。
在一次学生与评分者接触比平时更多的临床实习中确定评分者间信度。随后立即进行的临床实习培训评估计划用作确定预测效度的标准测量。
有四次评级时，评分者间信度为0.41，预测效度为0.32。信度低于为住院医师培训发表的类似结果，效度略高于类似结果。
即使增加了学生与评分者的互动，全球绩效评级的信度和效度仍过低，无法保证将全球绩效评级用作个人评估形式。然而，与其他评估措施相结合，全球绩效评级可能会改善整体评估。
