Department of Psychology, The Ohio State University, Columbus, OH, USA.
Psychother Res. 2021 Apr;31(4):548-556. doi: 10.1080/10503307.2020.1780510. Epub 2020 Jun 18.
Psychotherapy process research relies heavily upon trained raters to identify and code therapist and client behaviors. Raters are often selected out of convenience or availability with little research to inform what qualities are desirable for this role. In this study, we evaluated several rater characteristics as potential predictors of raters' performance. We provided training to 39 undergraduate students to serve as raters in a psychotherapy process study. We assessed baseline characteristics, personality characteristics, executive functioning, and interviewers' assessment of students' rating potential as predictors of rater agreement with gold standard ratings (i.e., consensus judgements from a panel of CBT-trained clinicians). We also assessed these variables as predictors of the risk of raters dropping out prior to completing assigned ratings. Higher conscientiousness, higher neuroticism, and a lower year in college predicted greater agreement with the gold standard ratings. Lower year in college also predicted lower risk of dropout. These findings provide empirical support for key characteristics as predictors of rater performance. Such evidence raises the possibility of using these characteristics to select raters and thereby enhance the psychometric properties of psychotherapy ratings.
心理治疗过程研究严重依赖经过培训的评分者来识别和编码治疗师和客户的行为。评分者通常是为了方便或可用性而选择的，很少有研究可以告知哪些素质是这个角色所需要的。在这项研究中，我们评估了几个评分者的特征，作为评分者表现的潜在预测因素。我们为 39 名本科生提供培训，担任心理治疗过程研究的评分者。我们评估了基线特征、人格特征、执行功能，以及访谈者对学生评分潜力的评估，作为评分者与黄金标准评分（即来自一组接受过认知行为治疗培训的临床医生的共识判断）一致性的预测因素。我们还评估了这些变量作为评分者在完成指定评分前退出风险的预测因素。更高的责任心、更高的神经质和更低的大学年级预测了与黄金标准评分更高的一致性。较低的大学年级也预示着较低的辍学风险。这些发现为评分者表现的预测因素提供了经验支持。这些证据提高了使用这些特征来选择评分者的可能性，从而提高心理治疗评分的心理计量学特性。
