Reim Timo, Hartebrodt Anne, Blumenthal David B, Bernett Judith, List Markus
Data Science in Systems Biology, TUM School of Life Sciences, Technical University of Munich, Freising, 85354, Germany.
Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, 91052, Germany.
Bioinformatics. 2025 Jul 1;41(Supplement_1):i590-i598. doi: 10.1093/bioinformatics/btaf192.
As most proteins interact with other proteins to perform their respective functions, methods to computationally predict these interactions have been developed. However, flawed evaluation schemes and data leakage in test sets have obscured the fact that sequence-based protein-protein interaction (PPI) prediction is still an open problem. Recently, methods achieving better-than-random performance on leakage-reduced PPI data have been proposed.
Here, we show that the use of ESM-2 protein embeddings explains this performance gain irrespective of model architecture. We compared the performance of models with varying complexity, per-protein, and per-token embeddings, as well as the influence of self- or cross-attention, where all models plateaued at an accuracy of 0.65. Moreover, we show that the tested sequence-based models cannot implicitly learn a contact map as an intermediate layer. These results imply that other input types, such as structure, might be necessary for producing reliable PPI predictions.
All code for models and execution of the models is available at https://github.com/daisybio/PPI_prediction_study. Python version 3.8.18 and PyTorch version 2.1.1 were used for this study. The environment containing the versions of all other packages used can be found in the GitHub repository. The used data are available at https://doi.org/10.6084/m9.figshare.21591618.v3.
由于大多数蛋白质与其他蛋白质相互作用以执行各自的功能，因此已经开发了用于计算预测这些相互作用的方法。然而，有缺陷的评估方案和测试集中的数据泄漏掩盖了基于序列的蛋白质-蛋白质相互作用（PPI）预测仍然是一个未解决问题的事实。最近，有人提出了在减少泄漏的PPI数据上实现优于随机性能的方法。
在这里，我们表明，无论模型架构如何，使用ESM-2蛋白质嵌入都能解释这种性能提升。我们比较了具有不同复杂度、每个蛋白质和每个token嵌入的模型的性能，以及自注意力或交叉注意力的影响，所有模型的准确率都稳定在0.65。此外，我们表明，经过测试的基于序列的模型不能隐式地将接触图学习为中间层。这些结果意味着，可能需要其他输入类型，如结构，才能产生可靠的PPI预测。
模型的所有代码和模型执行都可在https://github.com/daisybio/PPI_prediction_study获得。本研究使用了Python 3.8.18版本和PyTorch 2.1.1版本。包含所有其他使用包版本的环境可在GitHub存储库中找到。使用的数据可在https://doi.org/10.6084/m9.figshare.21591618.v3获得。
