Ministry of Education Key Laboratory of Bioinformatics, Bioinformatics Division, Beijing National Research Center for Information Science and Technology, Department of Automation, Tsinghua University, Beijing 100084, China.
Ministry of Education Key Laboratory of Bioinformatics, Bioinformatics Division, Beijing National Research Center for Information Science and Technology, Department of Automation, Tsinghua University, Beijing 100084, China.
Med Image Anal. 2022 Oct;81:102539. doi: 10.1016/j.media.2022.102539. Epub 2022 Jul 20.
Visual representation extraction is a fundamental problem in the field of computational histopathology. Considering the powerful representation capacity of deep learning and the scarcity of annotations, self-supervised learning has emerged as a promising approach to extract effective visual representations from unlabeled histopathological images. Although a few self-supervised learning methods have been specifically proposed for histopathological images, most of them suffer from certain defects that may hurt the versatility or representation capacity. In this work, we propose CS-CO, a hybrid self-supervised visual representation learning method tailored for H&E-stained histopathological images, which integrates advantages of both generative and discriminative approaches. The proposed method consists of two self-supervised learning stages: cross-stain prediction (CS) and contrastive learning (CO). In addition, a novel data augmentation approach named stain vector perturbation is specifically proposed to facilitate contrastive learning. Our CS-CO makes good use of domain-specific knowledge and requires no side information, which means good rationality and versatility. We evaluate and analyze the proposed CS-CO on three H&E-stained histopathological image datasets with downstream tasks of patch-level tissue classification and slide-level cancer prognosis and subtyping. Experimental results demonstrate the effectiveness and robustness of the proposed CS-CO on common computational histopathology tasks. Furthermore, we also conduct ablation studies and prove that cross-staining prediction and contrastive learning in our CS-CO can complement and enhance each other. Our code is made available at https://github.com/easonyang1996/CS-CO.
视觉表示提取是计算组织病理学领域的一个基本问题。考虑到深度学习的强大表示能力和注释的稀缺性，自监督学习已经成为从未标记的组织病理学图像中提取有效视觉表示的一种有前途的方法。尽管已经专门为组织病理学图像提出了几种自监督学习方法，但它们大多数都存在某些缺陷，可能会影响通用性或表示能力。在这项工作中，我们提出了 CS-CO，这是一种针对 H&E 染色组织病理学图像的混合自监督视觉表示学习方法，它结合了生成和判别方法的优势。所提出的方法由两个自监督学习阶段组成：交叉染色预测 (CS) 和对比学习 (CO)。此外，还专门提出了一种名为染色向量扰动的新数据增强方法来促进对比学习。我们的 CS-CO 充分利用了特定领域的知识，并且不需要辅助信息，这意味着它具有良好的合理性和通用性。我们在三个 H&E 染色组织病理学图像数据集上评估和分析了所提出的 CS-CO，并进行了补丁级组织分类和幻灯片级癌症预后和亚型划分的下游任务。实验结果表明了所提出的 CS-CO 在常见的计算组织病理学任务上的有效性和鲁棒性。此外，我们还进行了消融研究，并证明了我们的 CS-CO 中的交叉染色预测和对比学习可以相互补充和增强。我们的代码可在 https://github.com/easonyang1996/CS-CO 上获得。
