Ernst Marielle, Yoo Albert J, Kriston Levente, Schönfeld Michael H, Vettorazzi Eik, Fiehler Jens
From the Department of Diagnostic and Interventional Neuroradiology (M.E., M.H.S., J.F.), Department of Medical Psychology (L.K.), and Department of Medical Biometry and Epidemiology (E.V.), University Medical Center Hamburg-Eppendorf, Hamburg, Germany; and Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston (A.J.Y.).
Stroke. 2015 Jun;46(6):1574-81. doi: 10.1161/STROKEAHA.114.008513. Epub 2015 May 5.
Angiographic occlusion as a surrogate marker of satisfactory aneurysm treatment is commonly used in clinical trials although some pitfalls have to be considered. To investigate the inter-rater reliability of visual rating of aneurysm occlusion as study end point, we performed a systematic review and meta-analysis.
Electronic databases (MEDLINE, EMBASE, PubMed, and the Cochrane Library) were searched up to June 2014. Assessment of risk for bias was based on the Quality Appraisal Tool for Studies of Diagnostic Reliability and the Guidelines for Reporting Reliability and Agreement studies. Inter-rater reliability estimates were pooled across studies using meta-analysis, and the influence of several factors (eg, imaging methods, grading scales, and occlusion rate) was tested with meta-regression.
From 1193 titles, 644 abstracts and 87 full-text versions were reviewed. Twenty-six articles met the inclusion criteria and provided 77 reliability estimates. Twenty-one different rating scales were used, and statistical analysis varied. Mean inter-rater agreement of the pooled studies was substantial (κ=0.65; 95% confidence interval, 0.60-0.69). Reliability varied significantly as a function of imaging methods, grading scales, occlusion rates, and their interaction. Observer agreement substantially increased with increasing occlusion rate in digital subtraction angiography but not in MR angiography. Reliability was higher in studies using 2- or 3-value grading scales than in studies with 4-value grading scales.
There is significant heterogeneity between studies evaluating the reliability of visual evaluation of aneurysm coiling. On the basis of our analysis, we found that the combination of magnetic resonance angiography, 3-value grading scale, and 2 trained raters seems most promising for usage as surrogate study end points.
血管造影闭塞作为动脉瘤治疗效果令人满意的替代指标，虽存在一些问题，但在临床试验中仍被广泛应用。为研究将动脉瘤闭塞的视觉评估作为研究终点时评估者间的可靠性，我们进行了一项系统评价和荟萃分析。
检索截至2014年6月的电子数据库（MEDLINE、EMBASE、PubMed和Cochrane图书馆）。基于诊断可靠性研究质量评估工具和报告可靠性及一致性研究指南对偏倚风险进行评估。使用荟萃分析汇总各研究的评估者间可靠性估计值，并通过荟萃回归检验几个因素（如成像方法、分级量表和闭塞率）的影响。
共检索1193篇标题、644篇摘要和87篇全文。26篇文章符合纳入标准，提供了77个可靠性估计值。使用了21种不同的分级量表，统计分析各不相同。汇总研究的评估者间平均一致性较高（κ=0.65；95%置信区间，0.60 - 0.69）。可靠性因成像方法、分级量表、闭塞率及其相互作用而有显著差异。在数字减影血管造影中，随着闭塞率增加，观察者一致性显著提高，但在磁共振血管造影中并非如此。使用2分或3分制分级量表的研究比使用4分制分级量表的研究可靠性更高。
评估动脉瘤栓塞视觉评估可靠性的研究之间存在显著异质性。基于我们的分析，我们发现磁共振血管造影、3分制分级量表和2名经过培训的评估者的组合似乎最有希望用作替代研究终点。
