Poolman Rudolf W, Struijs Peter A A, Krips Rover, Sierevelt Inger N, Lutz Kristina H, Bhandari Mohit
Department Surgery, Division of Orthopaedic surgery, McMaster University, Hamilton General Hospital, 7 North, Room 727, 237 Barton Street East, Hamilton, Ontario L8L 2X2, Canada.
BMC Med Res Methodol. 2006 Sep 11;6:44. doi: 10.1186/1471-2288-6-44.
The Levels of Evidence Rating System is widely believed to categorize studies by quality, with Level I studies representing the highest quality evidence. We aimed to determine the reporting quality of Randomised Controlled Trials (RCTs) published in the most frequently cited general orthopaedic journals.
Two assessors identified orthopaedic journals that reported a level of evidence rating in their abstracts from January 2003 to December 2004 by searching the instructions for authors of the highest impact general orthopaedic journals. Based upon a priori eligibility criteria, two assessors hand searched all issues of the eligible journal from 2003-2004 for RCTs. The assessors extracted the demographic information and the evidence rating from each included RCT and scored the quality of reporting using the reporting quality assessment tool, which was developed by the Cochrane Bone, Joint and Muscle Trauma Group. Scores were conducted in duplicate, and we reached a consensus for any disagreements. We examined the correlation between the level of evidence rating and the Cochrane reporting quality score.
We found that only the Journal of Bone and Joint Surgery - American Volume (JBJS-A) used a level of evidence rating from 2003 to 2004. We identified 938 publications in the JBJS-A from January 2003 to December 2004. Of these publications, 32 (3.4%) were RCTs that fit the inclusion criteria. The 32 RCTs included a total of 3543 patients, with sample sizes ranging from 17 to 514 patients. Despite being labelled as the highest level of evidence (Level 1 and Level II evidence), these studies had low Cochrane reporting quality scores among individual methodological safeguards. The Cochrane reporting quality scores did not differ significantly between Level I and Level II studies. Correlations varied from 0.0 to 0.2 across the 12 items of the Cochrane reporting quality assessment tool (p > 0.05). Among items closely corresponding to the Levels of Evidence Rating System criteria assessors achieved substantial agreement (ICC = 0.80, 95% CI:0.60 to 0.90).
Our findings suggest that readers should not assume that 1) studies labelled as Level I have high reporting quality and 2) Level I studies have better reporting quality than Level II studies. One should address methodological safeguards individually.
证据等级评定系统被广泛认为是依据质量对研究进行分类，其中I级研究代表最高质量的证据。我们旨在确定发表在最常被引用的普通骨科期刊上的随机对照试验（RCT）的报告质量。
两名评估者通过搜索影响最大的普通骨科期刊的作者指南，找出在2003年1月至2004年12月期间在摘要中报告了证据等级评定的骨科期刊。基于预先设定的纳入标准，两名评估者人工检索了2003 - 2004年符合条件期刊的所有期次以查找RCT。评估者从每项纳入的RCT中提取人口统计学信息和证据等级评定，并使用由Cochrane骨、关节和肌肉创伤小组开发的报告质量评估工具对报告质量进行评分。评分进行了两次，对于任何分歧我们达成了共识。我们检查了证据等级评定与Cochrane报告质量得分之间的相关性。
我们发现只有《骨与关节外科杂志 - 美国卷》（JBJS - A）在2003年至2004年使用了证据等级评定。我们在2003年1月至2004年期间在JBJS - A中识别出938篇出版物。在这些出版物中，32篇（3.4%）是符合纳入标准的RCT。这32项RCT共纳入3543例患者，样本量从17例至514例患者不等。尽管被标记为最高等级的证据（I级和II级证据），但在个体方法学保障方面，这些研究的Cochrane报告质量得分较低。I级和II级研究之间的Cochrane报告质量得分没有显著差异。在Cochrane报告质量评估工具的12个项目中，相关性在0.0至0.2之间（p > 0.05）。在与证据等级评定系统标准密切对应的项目中，评估者达成了实质性共识（组内相关系数 = 0.80，95%可信区间：0.60至0.90）。
我们的研究结果表明，读者不应假定：1）标记为I级的研究具有较高的报告质量；2）I级研究的报告质量优于II级研究。应该分别关注方法学保障。
