评估者培训对迷你临床评估练习（mini-CEX）评分可靠性和准确性的影响：一项随机对照试验。
Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial.
作者信息
Cook David A, Dupras Denise M, Beckman Thomas J, Thomas Kris G, Pankratz V Shane
机构信息
Office of Education Research, College of Medicine, Mayo Clinic, Rochester, MN 55905, USA.
出版信息
J Gen Intern Med. 2009 Jan;24(1):74-9. doi: 10.1007/s11606-008-0842-3. Epub 2008 Nov 11.
BACKGROUND
Mini-CEX scores assess resident competence. Rater training might improve mini-CEX score interrater reliability, but evidence is lacking.
OBJECTIVE
Evaluate a rater training workshop using interrater reliability and accuracy.
DESIGN
Randomized trial (immediate versus delayed workshop) and single-group pre/post study (randomized groups combined).
SETTING
Academic medical center.
PARTICIPANTS
Fifty-two internal medicine clinic preceptors (31 randomized and 21 additional workshop attendees).
INTERVENTION
The workshop included rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion. Delayed group received no intervention until after posttest.
MEASUREMENTS
Mini-CEX ratings at baseline (just before workshop for workshop group), and four weeks later using videotaped resident-patient encounters; mini-CEX ratings of live resident-patient encounters one year preceding and one year following the workshop; rater confidence using mini-CEX.
RESULTS
Among 31 randomized participants, interrater reliabilities in the delayed group (baseline intraclass correlation coefficient [ICC] 0.43, follow-up 0.53) and workshop group (baseline 0.40, follow-up 0.43) were not significantly different (p = 0.19). Mean ratings were similar at baseline (delayed 4.9 [95% confidence interval 4.6-5.2], workshop 4.8 [4.5-5.1]) and follow-up (delayed 5.4 [5.0-5.7], workshop 5.3 [5.0-5.6]; p = 0.88 for interaction). For the entire cohort, rater confidence (1 = not confident, 6 = very confident) improved from mean (SD) 3.8 (1.4) to 4.4 (1.0), p = 0.018. Interrater reliability for ratings of live encounters (entire cohort) was higher after the workshop (ICC 0.34) than before (ICC 0.18) but the standard error of measurement was similar for both periods.
CONCLUSIONS
Rater training did not improve interrater reliability or accuracy of mini-CEX scores.
CLINICAL TRIALS REGISTRATION
clinicaltrials.gov identifier NCT00667940
背景
迷你临床演练评估（Mini-CEX）分数用于评估住院医师的能力。评估者培训可能会提高Mini-CEX分数的评分者间信度，但目前缺乏相关证据。
目的
使用评分者间信度和准确性评估一个评估者培训工作坊。
设计
随机试验（即时工作坊与延迟工作坊）和单组前后对照研究（随机分组合并）。
地点
学术医疗中心。
参与者
52名内科门诊带教老师（31名随机分组，另外21名参加工作坊）。
干预措施
该工作坊包括评估者误差培训、绩效维度培训、行为观察培训以及使用讲座、视频和小组讨论的参照框架培训。延迟组在测试后才接受干预。
测量指标
基线时（工作坊组在工作坊开始前）以及四周后使用住院医师与患者的录像互动进行Mini-CEX评分；工作坊前一年和后一年对住院医师与患者现场互动的Mini-CEX评分；使用Mini-CEX评估评估者的信心。
结果
在31名随机分组的参与者中，延迟组（基线组内相关系数[ICC]为0.43，随访时为0.53）和工作坊组（基线时为0.40，随访时为0.43）的评分者间信度无显著差异（p = 0.19）。基线时平均评分相似（延迟组4.9[95%置信区间4.6 - 5.2]，工作坊组4.8[4.5 - 5.1]），随访时也相似（延迟组5.4[5.0 - 5.7]，工作坊组5.3[5.0 - 5.6]；交互作用p = 0.88）。对于整个队列，评估者信心（1 = 不自信，6 = 非常自信）从平均（标准差）3.8（1.4）提高到4.4（1.0），p = 0.018。工作坊后对现场互动评分的评分者间信度（整个队列）高于工作坊前（ICC 0.34对ICC 0.18），但两个时期的测量标准误相似。
结论
评估者培训并未提高Mini-CEX分数的评分者间信度或准确性。
临床试验注册
clinicaltrials.gov标识符NCT00667940
Zotero 插件