Madej Dominik, Lam Henry
Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology, Hong Kong 999077, China.
J Proteome Res. 2025 Mar 7;24(3):1118-1134. doi: 10.1021/acs.jproteome.4c00743. Epub 2025 Feb 5.
Validating false discovery rate (FDR) estimation is an essential but surprisingly understudied aspect of method development in shotgun proteomics. Currently available validation protocols mostly rely on ground truth data sets, which typically involve manipulating the properties of the search space or query spectra used. As a result, comparing estimated FDR and ground truth-based false discovery proportion values may not be representative of the scenarios involving natural data sets encountered in practice. In this study, we introduce PyViscount─a Python tool implementing a novel validation protocol based on random search space partition, which enables generating a quasi ground-truth using unaltered search spaces of unique candidate peptides and generic data sets of experimental query spectra. Furthermore, validation of existing FDR estimation methods by PyViscount is consistent with alternative validation protocols. The presented novel approach to validation free from the need for synthetic data sets or dubious manipulation of the data may be an attractive alternative for proteomics practitioners, allowing them to obtain deeper insights into the performance of existing and new FDR estimation methods.
验证错误发现率（FDR）估计是鸟枪法蛋白质组学方法开发中一个重要但却惊人地未得到充分研究的方面。目前可用的验证方案大多依赖于真实数据集，而这些数据集通常涉及操纵所使用的搜索空间或查询光谱的属性。因此，比较估计的FDR和基于真实情况的错误发现比例值可能无法代表实际中遇到的涉及自然数据集的情况。在本研究中，我们引入了PyViscount——一个用Python实现的工具，它基于随机搜索空间划分实现了一种新颖的验证方案，该方案能够使用独特候选肽的未改变搜索空间和实验查询光谱的通用数据集生成准真实情况。此外，PyViscount对现有FDR估计方法的验证与其他验证方案一致。所提出的无需合成数据集或对数据进行可疑操纵的新颖验证方法，对于蛋白质组学从业者来说可能是一个有吸引力的选择，使他们能够更深入地了解现有和新的FDR估计方法的性能。
