Offord Centre for Child Studies, McMaster University, Hamilton, ON, Canada.
Department of Psychiatry & Behavioural Neurosciences, McMaster University, Hamilton, ON, Canada.
J Child Psychol Psychiatry. 2019 Jan;60(1):16-29. doi: 10.1111/jcpp.12876. Epub 2018 Feb 19.
A better understanding of factors contributing to the observed variability in estimates of test-retest reliability in published studies on standardized diagnostic interviews (SDI) is needed. The objectives of this systematic review and meta-analysis were to estimate the pooled test-retest reliability for parent and youth assessments of seven common disorders, and to examine sources of between-study heterogeneity in reliability.
Following a systematic review of the literature, multilevel random effects meta-analyses were used to analyse 202 reliability estimates (Cohen's kappa = ҡ) from 31 eligible studies and 5,369 assessments of 3,344 children and youth.
Pooled reliability was moderate at ҡ = .58 (CI 95% 0.53-0.63) and between-study heterogeneity was substantial (Q = 2,063 (df = 201), p < .001 and I = 79%). In subgroup analysis, reliability varied across informants for specific types of psychiatric disorder (ҡ = .53-.69 for parent vs. ҡ = .39-.68 for youth) with estimates significantly higher for parents on attention deficit hyperactivity disorder, oppositional defiant disorder and the broad groupings of externalizing and any disorder. Reliability was also significantly higher in studies with indicators of poor or fair study methodology quality (sample size <50, retest interval <7 days).
Our findings raise important questions about the meaningfulness of published evidence on the test-retest reliability of SDIs and the usefulness of these tools in both clinical and research contexts. Potential remedies include the introduction of standardized study and reporting requirements for reliability studies, and exploration of other approaches to assessing and classifying child and adolescent psychiatric disorder.
需要更好地理解导致已发表的标准化诊断访谈（SDI）测试-重测信度估计值存在变异性的因素。本系统评价和荟萃分析的目的是估计七种常见疾病的父母和青少年评估的综合测试-重测信度，并检查信度的研究间异质性的来源。
在对文献进行系统评价后，使用多级随机效应荟萃分析分析了 31 项合格研究中的 202 个可靠性估计值（Cohen's kappa=k）和 3344 名儿童和青少年的 5369 次评估。
汇总的可靠性为中等水平，k=0.58（95%CI 0.53-0.63），研究间异质性较大（Q=2063（df=201），p<0.001，I=79%）。在亚组分析中，特定类型的精神疾病的信息提供者之间的可靠性存在差异（父母的 k=0.53-0.69，而青少年的 k=0.39-0.68），父母对注意缺陷多动障碍、对立违抗性障碍以及外显和任何障碍的广泛分类的估计值明显更高。在研究方法质量指标较差或一般的研究（样本量<50，重测间隔<7 天）中，可靠性也显著更高。
我们的研究结果提出了一些重要的问题，即关于 SDI 测试-重测信度的已发表证据的意义，以及这些工具在临床和研究背景下的有用性。潜在的补救措施包括为可靠性研究引入标准化的研究和报告要求，以及探索评估和分类儿童和青少年精神疾病的其他方法。
