Sauerland S, Neugebauer E
Biochemical and Experimental Section, 2nd Department of Surgery, University of Cologne, Ostmerheimer Strasse 200, 51109 Cologne, Germany.
Surg Endosc. 2000 Oct;14(10):908-10. doi: 10.1007/s004640000283.
Ideally, a consensus panel combines expert knowledge with external evidence derived from the literature. To date, many consensus conferences do not use a structured approach to search the literature, but simply compile an add-on reference list from all papers cited by the panelists. This study examined how well such panelists retrieved the relevant literature.
We used the reference lists of nine surgeons who took part in a consensus conference on common bile duct stones. We included all papers that were referred to as randomized controlled trials (RCTs). We then compared this list with a database search in order to calculate sensitivity and specificity.
The nine experts cited between 35 and 518 papers, but only eight papers on average were RCTs. Of the 49 papers that the experts believed to be RCTs, only 23 actually were RCTs. The sensitivity resp. specificity for correctly identifying an RCT was 0.21 (95% Cl, 0.11-0.30) resp. 0.80 (95% Cl; 0.64-0.95). RCTs that included the word "randomized" in their title were significantly more likely to be identified (relative risk, 1.31; 95% Cl, 1.18-1.45).
Our data indicate that consensus panelists usually do not perform systematic literature searches, but simply use their favorite papers to back up their arguments. Because this may lead to a biased selection of the evidence base on which the consensus statements are founded, a systematic search of all relevant articles should become a mandatory task in any consensus or guideline process.
理想情况下，共识小组应将专家知识与从文献中获取的外部证据相结合。迄今为止，许多共识会议并未采用结构化方法检索文献，而只是简单地从小组成员引用的所有论文中汇编一份附加参考文献列表。本研究调查了这些小组成员检索相关文献的能力如何。
我们使用了九位参与胆总管结石共识会议的外科医生的参考文献列表。我们纳入了所有被称为随机对照试验（RCT）的论文。然后，我们将该列表与数据库搜索结果进行比较，以计算敏感性和特异性。
这九位专家引用的论文数量在35至518篇之间，但平均而言，只有八篇论文是RCT。在专家们认为是RCT的49篇论文中，实际上只有23篇是RCT。正确识别RCT的敏感性和特异性分别为0.21（95%可信区间，0.11 - 0.30）和0.80（95%可信区间；0.64 - 0.95）。标题中包含“随机”一词的RCT被识别出来的可能性显著更高（相对风险，1.31；95%可信区间，1.18 - 1.45）。
我们的数据表明，共识小组成员通常不会进行系统的文献检索，而只是简单地使用他们喜欢的论文来支持自己的论点。由于这可能导致在形成共识声明所依据的证据基础上出现有偏差的选择，因此在任何共识或指南制定过程中，对所有相关文章进行系统检索应成为一项强制性任务。
