Institute for Information Technology, National Research Council, Ottawa, Ontario, Canada.
BMC Med Inform Decis Mak. 2010 Sep 28;10:56. doi: 10.1186/1472-6947-10-56.
Clinical trials are one of the most important sources of evidence for guiding evidence-based practice and the design of new trials. However, most of this information is available only in free text - e.g., in journal publications - which is labour intensive to process for systematic reviews, meta-analyses, and other evidence synthesis studies. This paper presents an automatic information extraction system, called ExaCT, that assists users with locating and extracting key trial characteristics (e.g., eligibility criteria, sample size, drug dosage, primary outcomes) from full-text journal articles reporting on randomized controlled trials (RCTs).
ExaCT consists of two parts: an information extraction (IE) engine that searches the article for text fragments that best describe the trial characteristics, and a web browser-based user interface that allows human reviewers to assess and modify the suggested selections. The IE engine uses a statistical text classifier to locate those sentences that have the highest probability of describing a trial characteristic. Then, the IE engine's second stage applies simple rules to these sentences to extract text fragments containing the target answer. The same approach is used for all 21 trial characteristics selected for this study.
We evaluated ExaCT using 50 previously unseen articles describing RCTs. The text classifier (first stage) was able to recover 88% of relevant sentences among its top five candidates (top5 recall) with the topmost candidate being relevant in 80% of cases (top1 precision). Precision and recall of the extraction rules (second stage) were 93% and 91%, respectively. Together, the two stages of the extraction engine were able to provide (partially) correct solutions in 992 out of 1050 test tasks (94%), with a majority of these (696) representing fully correct and complete answers.
Our experiments confirmed the applicability and efficacy of ExaCT. Furthermore, they demonstrated that combining a statistical method with 'weak' extraction rules can identify a variety of study characteristics. The system is flexible and can be extended to handle other characteristics and document types (e.g., study protocols).
临床试验是指导循证实践和新试验设计的最重要证据来源之一。然而，这些信息大多只以自由文本的形式存在，例如在期刊出版物中，这对于系统评价、荟萃分析和其他证据综合研究来说是一项劳动密集型的工作。本文介绍了一种名为 ExaCT 的自动信息提取系统，该系统可帮助用户从报告随机对照试验（RCT）的全文期刊文章中定位和提取关键试验特征（例如，纳入标准、样本量、药物剂量、主要结局）。
ExaCT 由两部分组成：一个信息提取（IE）引擎，用于在文章中搜索最能描述试验特征的文本片段；以及一个基于网络浏览器的用户界面，允许人工审阅者评估和修改建议的选择。IE 引擎使用统计文本分类器来定位那些最有可能描述试验特征的句子。然后，IE 引擎的第二阶段对这些句子应用简单的规则来提取包含目标答案的文本片段。这种方法适用于本研究中选择的所有 21 个试验特征。
我们使用 50 篇以前未见过的描述 RCT 的文章来评估 ExaCT。文本分类器（第一阶段）能够在其前五个候选者中（前 5 召回率）恢复 88%的相关句子，其中最上面的候选者在 80%的情况下是相关的（前 1 召回率）。提取规则（第二阶段）的精度和召回率分别为 93%和 91%。提取引擎的两个阶段总共能够在 1050 个测试任务中的 992 个任务中提供（部分）正确的解决方案（94%），其中大多数（696 个）表示完全正确和完整的答案。
我们的实验证实了 ExaCT 的适用性和有效性。此外，它们表明，将统计方法与“弱”提取规则相结合，可以识别各种研究特征。该系统具有灵活性，可以扩展到处理其他特征和文档类型（例如，研究方案）。
