Norman Christopher, Leeflang Mariska, Névéol Aurélie
LIMSI, CNRS, Université Paris Saclay, F-91405 Orsay.
Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands.
AMIA Annu Symp Proc. 2018 Dec 5;2018:817-826. eCollection 2018.
Systematic reviews are critical for obtaining accurate estimates of diagnostic test accuracy, yet these require extracting information buried in free text articles, an often laborious process.
We create a dataset describing the data extraction and synthesis processes in 63 DTA systematic reviews, and demonstrate its utility by using it to replicate the data synthesis in the original reviews.
We construct our dataset using a custom automated extraction pipeline complemented with manual extraction, verification, and post-editing. We evaluate using manual assessment by two annotators and by comparing against data extracted from source files.
The constructed dataset contains 5,848 test results for 1,354 diagnostic tests from 1,738 diagnostic studies. We observe an extraction error rate of 0.06-0.3%.
This constitutes the first dataset describing the later stages of the DTA systematic review process, and is intended to be useful for automating or evaluating the process.
系统评价对于获得诊断试验准确性的准确估计至关重要，但这需要从自由文本文章中提取信息，这通常是一个费力的过程。
我们创建了一个数据集，描述了63项诊断准确性系统评价中的数据提取和综合过程，并通过使用该数据集在原始评价中复制数据综合来证明其效用。
我们使用定制的自动提取管道构建数据集，并辅以手动提取、验证和后期编辑。我们通过两名注释者的人工评估以及与从源文件中提取的数据进行比较来进行评估。
构建的数据集包含来自1738项诊断研究的1354项诊断试验的5848个试验结果。我们观察到提取错误率为0.06 - 0.3%。
这构成了第一个描述诊断准确性系统评价过程后期阶段的数据集，旨在用于自动化或评估该过程。
