Department of Genetics, Harvard Medical School, Boston, MA 02115, USA; Division of Endocrinology and Center for Basic and Translational Obesity Research, Boston Children's Hospital, Boston, MA 02115, USA; Program in Medical and Population Genetics, Broad Institute of MIT and Harvard, Cambridge, MA 02142, USA; Ph.D. Program in Biological and Biomedical Sciences, Graduate School of Arts and Sciences, Harvard University, Cambridge, MA 02138, USA.
The Novo Nordisk Foundation Center for Basic Metabolic Research, Faculty of Health and Medical Sciences, University of Copenhagen, 2200 Copenhagen, Denmark; Department of Epidemiology Research, Statens Serum Institut, 2300 Copenhagen, Denmark.
Am J Hum Genet. 2019 Jun 6;104(6):1025-1039. doi: 10.1016/j.ajhg.2019.03.027. Epub 2019 May 2.
Genome-wide association studies (GWASs) are valuable for understanding human biology, but associated loci typically contain multiple associated variants and genes. Thus, algorithms that prioritize likely causal genes and variants for a given phenotype can provide biological interpretations of association data. However, a critical, currently missing capability is to objectively compare performance of such algorithms. Typical comparisons rely on "gold standard" genes harboring causal coding variants, but such gold standards may be biased and incomplete. To address this issue, we developed Benchmarker, an unbiased, data-driven benchmarking method that compares performance of similarity-based prioritization strategies to each other (and to random chance) by leave-one-chromosome-out cross-validation with stratified linkage disequilibrium (LD) score regression. We first applied Benchmarker to 20 well-powered GWASs and compared gene prioritization based on strategies employing three different data sources, including annotated gene sets and gene expression; genes prioritized based on gene sets had higher per-SNP heritability than those prioritized based on gene expression. Additionally, in a direct comparison of three methods, DEPICT and MAGMA outperformed NetWAS. We also evaluated combinations of methods; our results indicated that combining data sources and algorithms can help prioritize higher-quality genes for follow-up. Benchmarker provides an unbiased approach to evaluate any similarity-based method that provides genome-wide prioritization of genes, variants, or gene sets and can determine the best such method for any particular GWAS. Our method addresses an important unmet need for rigorous tool assessment and can assist in mapping genetic associations to causal function.
全基因组关联研究（GWAS）对于理解人类生物学非常有价值，但相关的基因座通常包含多个相关的变体和基因。因此，优先考虑给定表型的可能因果基因和变体的算法可以为关联数据提供生物学解释。然而，目前缺失的一个关键能力是客观比较这些算法的性能。典型的比较依赖于含有因果编码变体的“黄金标准”基因，但这种黄金标准可能存在偏差和不完整。为了解决这个问题，我们开发了 Benchmarker，这是一种公正、数据驱动的基准测试方法，通过分层连锁不平衡（LD）得分回归的留一染色体交叉验证，比较基于相似性的优先级排序策略之间的性能（以及与随机机会的比较）。我们首先将 Benchmarker 应用于 20 项功能强大的 GWAS，并比较了基于三种不同数据源的策略的基因优先级排序，包括注释基因集和基因表达；基于基因集优先排序的基因比基于基因表达优先排序的基因具有更高的 SNP 遗传力。此外，在三种方法的直接比较中，DEPICT 和 MAGMA 的表现优于 NetWAS。我们还评估了方法的组合；我们的结果表明，结合数据源和算法可以帮助优先考虑更高质量的基因进行后续研究。Benchmarker 为评估任何提供全基因组基因、变体或基因集优先级排序的基于相似性的方法提供了一种公正的方法，并可以确定任何特定 GWAS 的最佳方法。我们的方法满足了严格工具评估的重要未满足需求，并有助于将遗传关联映射到因果功能。
