Arthritis and Clinical Immunology Research Program, Division of Genomics and Data Sciences, Oklahoma Medical Research Foundation, Oklahoma City, Oklahoma 73104-5005.
Department of Biochemistry and Molecular Biology, University of Oklahoma Health Sciences Center.
Bioinformatics. 2018 May 15;34(10):1758-1766. doi: 10.1093/bioinformatics/btx811.
Studies, mostly from the operations/management literature, have shown that the rate of human error increases with task complexity. What is not known is how many errors make it into the published literature, given that they must slip by peer-review. By identifying paired, dependent values within text for reported calculations of varying complexity, we can identify discrepancies, quantify error rates and identify mitigating factors.
We extracted statistical ratios from MEDLINE abstracts (hazard ratio, odds ratio, relative risk), their 95% CIs, and their P-values. We re-calculated the ratios and P-values using the reported CIs. For comparison, we also extracted percent-ratio pairs, one of the simplest calculation tasks. Over 486 000 published values were found and analyzed for discrepancies, allowing for rounding and significant figures. Per reported item, discrepancies were less frequent in percent-ratio calculations (2.7%) than in ratio-CI and P-value calculations (5.6-7.5%), and smaller discrepancies were more frequent than large ones. Systematic discrepancies (multiple incorrect calculations of the same type) were higher for more complex tasks (14.3%) than simple ones (6.7%). Discrepancy rates decreased with increasing journal impact factor (JIF) and increasing number of authors, but with diminishing returns and JIF accounting for most of the effect. Approximately 87% of the 81 937 extracted P-values were ≤ 0.05.
Using a simple, yet accurate, approach to identifying paired values within text, we offer the first quantitative evaluation of published error frequencies within these types of calculations.
jonathan-wren@omrf.org or jdwren@gmail.com.
Supplementary data are available at Bioinformatics online.
研究主要来自运营/管理文献，表明随着任务复杂性的增加，人为错误的发生率也会增加。目前尚不清楚有多少错误会出现在发表的文献中，因为它们必须通过同行评审。通过在报告的计算中识别文本内配对的、相关的值，可以确定差异、量化错误率并确定减轻因素。
我们从 MEDLINE 摘要中提取了统计比率（危险比、优势比、相对风险）、它们的 95%置信区间（CI）和 P 值。我们使用报告的 CI 重新计算了比率和 P 值。为了比较，我们还提取了最简单计算任务之一的百分比比率对。发现并分析了超过 486000 个已发表的数值差异，允许舍入和有效数字。每个报告的项目中，百分比比率计算的差异（2.7%）比比率-CI 和 P 值计算的差异（5.6-7.5%）更不频繁，小差异比大差异更频繁。对于更复杂的任务（14.3%），系统差异（相同类型的多个不正确计算）比简单任务（6.7%）更高。差异率随着期刊影响因子（JIF）和作者数量的增加而降低，但回报递减，JIF 占大部分影响。在提取的 81937 个 P 值中，约有 87%的值≤0.05。
使用一种简单但准确的方法在文本内识别配对值，我们首次对这些类型的计算中发表的错误频率进行了定量评估。
jonathan-wren@omrf.org 或 jdwren@gmail.com。
补充数据可在生物信息学在线获得。
