Targum Steven D, Cara Pendergrass J, Toner Chelsea, Zumpano Laura, Rauh Philip, DeMartinis Nicolas
Clintara, LLC, Boston, MA, United States.
Clintara, LLC, Boston, MA, United States.
Eur Neuropsychopharmacol. 2015 Mar;25(3):312-8. doi: 10.1016/j.euroneuro.2014.11.023. Epub 2014 Dec 13.
Signal detection in clinical trials relies on ratings reliability. We conducted a reliability analysis of site-independent rater scores derived from audio-digital recordings of site-based rater interviews of the structured Brief Psychiatric Rating Scale (BPRS) in a schizophrenia study. "Dual" ratings assessments were conducted as part of a quality assurance program in a 12-week, double-blind, parallel-group study of PF-02545920 compared to placebo in patients with sub-optimally controlled symptoms of schizophrenia (ClinicalTrials.gov identifier NCT01939548). Blinded, site-independent raters scored the recorded site-based BPRS interviews that were administered in relatively stable patients during two visits prior to the randomization visit. We analyzed the impact of BPRS interview length on "dual" scoring variance and discordance between trained and certified site-based raters and the paired scores of the independent raters. Mean total BPRS scores for 392 interviews conducted at the screen and stabilization visits were 50.4±7.2 (SD) for site-based raters and 49.2±7.2 for site-independent raters (t=2.34; p=0.025). "Dual" rated total BPRS scores were highly correlated (r=0.812). Mean BPRS interview length was 21:05±7:47min ranging from 7 to 59min. 89 interviews (23%) were conducted in less than 15min. These shorter interviews had significantly greater "dual" scoring variability (p=0.0016) and absolute discordance (p=0.0037) between site-based and site-independent raters than longer interviews. In-study ratings reliability cannot be guaranteed by pre-study rater certification. Our findings reveal marked variability of BPRS interview length and that shorter interviews are often incomplete yielding greater "dual" scoring discordance that may affect ratings precision.
临床试验中的信号检测依赖于评分的可靠性。在一项精神分裂症研究中，我们对基于结构化简明精神病评定量表（BPRS）的现场评估者访谈的音频数字记录得出的与现场无关的评估者分数进行了可靠性分析。“双重”评分评估是一项质量保证计划的一部分，该计划是一项为期12周的双盲平行组研究，将PF-02545920与安慰剂用于精神分裂症症状控制欠佳的患者（ClinicalTrials.gov标识符NCT01939548）。在随机分组访视前的两次访视中，对相对稳定的患者进行的基于现场的BPRS访谈录音由不知情的、与现场无关的评估者进行评分。我们分析了BPRS访谈时长对“双重”评分差异以及经过培训和认证的现场评估者与独立评估者的配对分数之间不一致性的影响。在筛查和稳定访视时进行的392次访谈中，现场评估者的BPRS总平均分是50.4±7.2（标准差），与现场无关的评估者为49.2±7.2（t=2.34；p=0.025）。“双重”评分的BPRS总分数高度相关（r=0.812）。BPRS访谈的平均时长为21:05±7:47分钟，范围从7到59分钟。89次访谈（23%）时长不到15分钟。与较长的访谈相比，这些较短的访谈在现场评估者和与现场无关的评估者之间具有显著更大的“双重”评分变异性（p=0.0016）和绝对不一致性（p=0.0037）。研究前评估者的认证无法保证研究中的评分可靠性。我们的研究结果揭示了BPRS访谈时长存在显著变异性，且较短的访谈往往不完整，会产生更大的“双重”评分不一致性，这可能会影响评分的准确性。
