Calhoun Aaron W, Rider Elizabeth A, Meyer Elaine C, Lamiani Giulia, Truog Robert D
Division of Pediatric Critical Care Medicine (A.W.C.), University of Louisville, Louisville, KY, USA.
Simul Healthc. 2009 Spring;4(1):22-9. doi: 10.1097/SIH.0b013e318184377a.
Multirater assessment is a powerful means of measuring communication skills. The use of gap analysis to assess self-appraisal is a strength of this technique. On the basis of Kalamazoo Consensus Statement framework and 360-degree assessment models, we developed a multirater instrument with gap analysis, with the goals of examining both communication skills and situational self-appraisal, and assessing the feasibility of the combined approach.
The multirater communication skills instrument was used to assess Pediatric and Neonatal Intensive Care fellows after participation in seven simulated family meetings. Instrument reliability was determined using Cronbach's Alpha and Factorial Analysis. Correlations between rater groups were examined with Spearman's Rank Coefficient. Gap analyses and rater perceptions of the instruments were analyzed using descriptive statistics.
Seven pediatric intensive care unit and neonatal intensive care fellows were each assessed by 11 to 18 raters (108 total assessments). Correlations were identified between disciplinary groups within each encounter. Among the 7 fellows, 30 communication strengths or areas needing improvement and 24 significant gaps were identified, indicating self under-appraisals, 9 (38%) of which overlapped. The instrument was logistically feasible and well received.
Our multirater communication skills instrument with gap analysis proved useful in identifying areas of strength and areas needing improvement, and in highlighting areas of self over- and under-appraisal that require focused feedback. The use of multirater assessment with gap analysis, in a simulated and "safe" environment, may assist in the delivery of feedback to trainees.
多评估者评估是衡量沟通技巧的有力手段。运用差距分析来评估自我评估是该技术的一项优势。基于卡拉马祖共识声明框架和360度评估模型，我们开发了一种带有差距分析的多评估者工具，旨在考察沟通技巧和情境自我评估，并评估这种综合方法的可行性。
在参与七次模拟家庭会议后，使用多评估者沟通技巧工具对儿科和新生儿重症监护研究员进行评估。使用克朗巴哈系数和因子分析来确定工具的可靠性。用斯皮尔曼等级系数检验评估者组之间的相关性。使用描述性统计分析差距分析和评估者对工具的看法。
七名儿科重症监护病房和新生儿重症监护研究员每人由11至18名评估者进行评估（共108次评估）。在每次会诊中确定了不同学科组之间的相关性。在这7名研究员中，确定了30项沟通优势或需要改进的方面以及24个显著差距，表明存在自我评估不足的情况，其中9个（38%）存在重叠。该工具在操作上是可行的，并且受到好评。
我们带有差距分析的多评估者沟通技巧工具被证明有助于确定优势领域和需要改进的领域，并突出需要重点反馈的自我评估过高和过低的领域。在模拟的“安全”环境中使用带有差距分析的多评估者评估，可能有助于向学员提供反馈。
