Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA.
Division of Nephrology, Boston Children's Hospital, Boston, MA, USA.
Nat Methods. 2020 Dec;17(12):1207-1213. doi: 10.1038/s41592-020-00978-4. Epub 2020 Oct 12.
Ensuring reproducibility of results in high-throughput experiments is crucial for biomedical research. Here, we propose a set of computational methods, INTRIGUE, to evaluate and control reproducibility in high-throughput settings. Our approaches are built on a new definition of reproducibility that emphasizes directional consistency when experimental units are assessed with signed effect size estimates. The proposed methods are designed to (1) assess the overall reproducible quality of multiple studies and (2) evaluate reproducibility at the individual experimental unit levels. We demonstrate the proposed methods in detecting unobserved batch effects via simulations. We further illustrate the versatility of the proposed methods in transcriptome-wide association studies: in addition to reproducible quality control, they are also suited to investigating genuine biological heterogeneity. Finally, we discuss the potential extensions of the proposed methods in other vital areas of reproducible research (for example, publication bias and conceptual replications).
确保高通量实验结果的可重复性对于生物医学研究至关重要。在这里，我们提出了一套计算方法 INTRIGUE，用于评估和控制高通量环境中的可重复性。我们的方法基于可重复性的新定义，当使用有符号效应大小估计值评估实验单位时，强调方向性一致性。所提出的方法旨在 (1) 评估多个研究的整体可重复质量，以及 (2) 在单个实验单位水平上评估可重复性。我们通过模拟来检测未观察到的批次效应，演示了所提出的方法。我们进一步说明了所提出的方法在全转录组关联研究中的多功能性：除了可重复的质量控制之外，它们还适合研究真正的生物学异质性。最后，我们讨论了所提出的方法在可重复研究的其他重要领域（例如，出版偏倚和概念复制）中的潜在扩展。
