Meta-Analysis Group, MRC Clinical Trials Unit, London WC2B 6NH, UK.
BMJ. 2013 Apr 22;346:f1798. doi: 10.1136/bmj.f1798.
To evaluate the reliability of risk of bias assessments based on published trial reports, for determining trial inclusion in meta-analyses.
Reliability evaluation of risk of bias assessments.
13 published individual participant data (IPD) meta-analyses in cancer were used to source 95 randomised controlled trials.
Risk of bias was assessed using the Cochrane risk of bias tool (RevMan5.1) and accompanying guidance. Assessments were made for individual risk of bias domains and overall for each trial, using information from either trial reports alone or trial reports with additional information collected for IPD meta-analyses. Percentage agreements were calculated for individual domains and overall (<66%= low, ≥ 66% = fair, ≥ 90% = good). The two approaches were considered similarly reliable only when agreement was good.
Percentage agreement between the two methods for sequence generation and incomplete outcome data was fair (69.5% (95% confidence interval 60.2% to 78.7%) and 80.0% (72.0% to 88.0%), respectively). However, percentage agreement was low for allocation concealment, selective outcome reporting, and overall risk of bias (48.4% (38.4% to 58.5%), 42.1% (32.2% to 52.0%), and 54.7% (44.7% to 64.7%), respectively). Supplementary information reduced the proportion of unclear assessments for all individual domains, consequently increasing the number of trials assessed as low risk of bias (and therefore available for inclusion in meta-analyses) from 23 (23%) based on publications alone to 66 (66%) based on publications with additional information.
Using cancer trial publications alone to assess risk of bias could be unreliable; thus, reviewers should be cautious about using them as a basis for trial inclusion, particularly for those trials assessed as unclear risk. Supplementary information from trialists should be sought to enable appropriate assessments and potentially reduce or overcome some risks of bias. Furthermore, guidance should ensure clarity on what constitutes risk of bias, particularly for the more subjective domains.
评估基于已发表试验报告的偏倚风险评估的可靠性，以确定试验是否纳入荟萃分析。
偏倚风险评估的可靠性评估。
为了源 95 项随机对照试验，使用了 13 项已发表的个体参与者数据（IPD）荟萃分析。
使用 Cochrane 偏倚风险工具（RevMan5.1）及其配套指南评估偏倚风险。仅根据试验报告或试验报告（加上为 IPD 荟萃分析收集的额外信息）评估各个试验的个别偏倚风险域和总体偏倚风险，计算各域和总体的百分比一致性（<66%=低，≥66%=中，≥90%=高）。只有当一致性良好时，两种方法才被认为具有相似的可靠性。
两种方法在序列生成和不完整结局数据方面的一致性百分比为中（分别为 69.5%（95%置信区间 60.2%至 78.7%）和 80.0%（72.0%至 88.0%））。然而，在分配隐藏、选择性结局报告和总体偏倚风险方面，一致性百分比较低（分别为 48.4%（38.4%至 58.5%）、42.1%（32.2%至 52.0%）和 54.7%（44.7%至 64.7%））。补充信息减少了所有个别域的不明确评估比例，从而将根据出版物单独评估为低偏倚风险的试验数量从 23 项（23%）增加到 66 项（66%），基于出版物和附加信息。
仅使用癌症试验出版物评估偏倚风险可能不可靠；因此，审查员应谨慎使用它们作为试验纳入的依据，特别是对于那些评估为不明确风险的试验。应向试验者寻求补充信息，以进行适当的评估，并可能减少或克服一些偏倚风险。此外，指南应明确偏倚的构成，特别是对于更主观的领域。
