Department of Experimental Medical Science, BMC B13, Lund University, SE-22 184 Lund, Sweden.
School of Computer Science and Technology, Soochow University, No1. Shizi Street, Suzhou, 215006 Jiangsu, China.
Database (Oxford). 2020 Jan 1;2020. doi: 10.1093/database/baz117.
Development of new computational methods and testing their performance has to be carried out using experimental data. Only in comparison to existing knowledge can method performance be assessed. For that purpose, benchmark datasets with known and verified outcome are needed. High-quality benchmark datasets are valuable and may be difficult, laborious and time consuming to generate. VariBench and VariSNP are the two existing databases for sharing variation benchmark datasets used mainly for variation interpretation. They have been used for training and benchmarking predictors for various types of variations and their effects. VariBench was updated with 419 new datasets from 109 papers containing altogether 329 014 152 variants; however, there is plenty of redundancy between the datasets. VariBench is freely available at http://structure.bmc.lu.se/VariBench/. The contents of the datasets vary depending on information in the original source. The available datasets have been categorized into 20 groups and subgroups. There are datasets for insertions and deletions, substitutions in coding and non-coding region, structure mapped, synonymous and benign variants. Effect-specific datasets include DNA regulatory elements, RNA splicing, and protein property for aggregation, binding free energy, disorder and stability. Then there are several datasets for molecule-specific and disease-specific applications, as well as one dataset for variation phenotype effects. Variants are often described at three molecular levels (DNA, RNA and protein) and sometimes also at the protein structural level including relevant cross references and variant descriptions. The updated VariBench facilitates development and testing of new methods and comparison of obtained performances to previously published methods. We compared the performance of the pathogenicity/tolerance predictor PON-P2 to several benchmark studies, and show that such comparisons are feasible and useful, however, there may be limitations due to lack of provided details and shared data. Database URL: http://structure.bmc.lu.se/VariBench.
开发新的计算方法并测试其性能必须使用实验数据进行。只有与现有知识进行比较，才能评估方法的性能。为此，需要具有已知和经过验证结果的基准数据集。高质量的基准数据集是有价值的，并且可能难以生成、费力且耗时。VariBench 和 VariSNP 是两个现有的用于共享变异基准数据集的数据库，主要用于变异解释。它们已被用于训练和基准测试各种类型的变异及其影响的预测器。VariBench 已使用来自 109 篇论文的 419 个新数据集进行了更新，其中共包含 329014152 个变体；但是，数据集之间存在大量冗余。VariBench 可在 http://structure.bmc.lu.se/VariBench/ 上免费获得。数据集的内容取决于原始来源中的信息。可用数据集已分为 20 个组和子组。有用于插入和缺失、编码和非编码区替换、结构映射、同义和良性变体的数据集。特定于效应的数据集包括 DNA 调控元件、RNA 剪接以及用于聚集、结合自由能、无序和稳定性的蛋白质特性。然后还有几个用于分子特异性和疾病特异性应用的数据集，以及一个用于变异表型效应的数据集。变体通常在三个分子水平（DNA、RNA 和蛋白质）上进行描述，有时也在包括相关交叉引用和变体描述的蛋白质结构水平上进行描述。更新后的 VariBench 促进了新方法的开发和测试，并将获得的性能与以前发表的方法进行比较。我们将致病性/耐受性预测器 PON-P2 的性能与几项基准研究进行了比较，并表明这种比较是可行且有用的，但是，由于缺乏提供的细节和共享的数据，可能存在限制。数据库网址：http://structure.bmc.lu.se/VariBench。
