School of Computer Science and Technology, Nanjing Tech University, 211816, Nanjing, China.
School of Computer Science and Engineering, Central South University,410075, Changsha, China.
Brief Bioinform. 2022 Sep 20;23(5). doi: 10.1093/bib/bbac303.
Accurate and efficient prediction of the molecular property is one of the fundamental problems in drug research and development. Recent advancements in representation learning have been shown to greatly improve the performance of molecular property prediction. However, due to limited labeled data, supervised learning-based molecular representation algorithms can only search limited chemical space and suffer from poor generalizability.
In this work, we proposed a self-supervised learning method, ATMOL, for molecular representation learning and properties prediction. We developed a novel molecular graph augmentation strategy, referred to as attention-wise graph masking, to generate challenging positive samples for contrastive learning. We adopted the graph attention network as the molecular graph encoder, and leveraged the learned attention weights as masking guidance to generate molecular augmentation graphs. By minimization of the contrastive loss between original graph and augmented graph, our model can capture important molecular structure and higher order semantic information. Extensive experiments showed that our attention-wise graph mask contrastive learning exhibited state-of-the-art performance in a couple of downstream molecular property prediction tasks. We also verified that our model pretrained on larger scale of unlabeled data improved the generalization of learned molecular representation. Moreover, visualization of the attention heatmaps showed meaningful patterns indicative of atoms and atomic groups important to specific molecular property.
准确高效地预测分子性质是药物研发的基本问题之一。最近的表示学习进展表明，它极大地提高了分子性质预测的性能。然而，由于标记数据有限，基于监督学习的分子表示算法只能搜索有限的化学空间，并且泛化能力较差。
在这项工作中，我们提出了一种用于分子表示学习和性质预测的自监督学习方法 ATMOL。我们开发了一种新的分子图增强策略，称为注意导向图掩蔽，为对比学习生成具有挑战性的正样本。我们采用图注意网络作为分子图编码器，并利用学习到的注意力权重作为掩蔽指导来生成分子增强图。通过原始图和增强图之间的对比损失最小化，我们的模型可以捕获重要的分子结构和更高阶的语义信息。大量实验表明，我们的注意导向图掩蔽对比学习在几个下游分子性质预测任务中表现出了最先进的性能。我们还验证了我们在更大规模未标记数据上预训练的模型提高了学习分子表示的泛化能力。此外，注意力热图的可视化显示了对特定分子性质重要的原子和原子团的有意义模式。
