Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.
Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
Med Image Anal. 2022 Jul;79:102474. doi: 10.1016/j.media.2022.102474. Epub 2022 May 4.
Artificial intelligence (AI) can extract visual information from histopathological slides and yield biological insight and clinical biomarkers. Whole slide images are cut into thousands of tiles and classification problems are often weakly-supervised: the ground truth is only known for the slide, not for every single tile. In classical weakly-supervised analysis pipelines, all tiles inherit the slide label while in multiple-instance learning (MIL), only bags of tiles inherit the label. However, it is still unclear how these widely used but markedly different approaches perform relative to each other. We implemented and systematically compared six methods in six clinically relevant end-to-end prediction tasks using data from N=2980 patients for training with rigorous external validation. We tested three classical weakly-supervised approaches with convolutional neural networks and vision transformers (ViT) and three MIL-based approaches with and without an additional attention module. Our results empirically demonstrate that histological tumor subtyping of renal cell carcinoma is an easy task in which all approaches achieve an area under the receiver operating curve (AUROC) of above 0.9. In contrast, we report significant performance differences for clinically relevant tasks of mutation prediction in colorectal, gastric, and bladder cancer. In these mutation prediction tasks, classical weakly-supervised workflows outperformed MIL-based weakly-supervised methods for mutation prediction, which is surprising given their simplicity. This shows that new end-to-end image analysis pipelines in computational pathology should be compared to classical weakly-supervised methods. Also, these findings motivate the development of new methods which combine the elegant assumptions of MIL with the empirically observed higher performance of classical weakly-supervised approaches. We make all source codes publicly available at https://github.com/KatherLab/HIA, allowing easy application of all methods to any similar task.
人工智能 (AI) 可以从组织病理学幻灯片中提取视觉信息，并提供生物学见解和临床生物标志物。全切片图像被切成数千个瓦片，分类问题通常是弱监督的：只有幻灯片有真实标签，而不是每个瓦片都有。在经典的弱监督分析管道中，所有瓦片都继承幻灯片标签，而在多实例学习 (MIL) 中，只有瓦片袋继承标签。然而，目前尚不清楚这些广泛使用但明显不同的方法彼此之间的表现如何。我们使用来自 2980 名患者的数据实现并系统比较了六种方法在六个临床相关的端到端预测任务中的表现，使用严格的外部验证进行训练。我们测试了三种基于卷积神经网络和视觉转换器 (ViT) 的经典弱监督方法以及三种基于 MIL 的方法，其中包括和不包括额外的注意力模块。我们的结果从经验上证明，肾细胞癌的组织学肿瘤亚型分类是一项简单的任务，所有方法的接收者操作特征曲线 (AUROC) 都在 0.9 以上。相比之下，我们报告了在结直肠癌、胃癌和膀胱癌的突变预测等临床相关任务中存在显著的性能差异。在这些突变预测任务中，经典的弱监督工作流程优于基于 MIL 的弱监督方法，这令人惊讶，因为它们很简单。这表明计算病理学中的新端到端图像分析管道应该与经典的弱监督方法进行比较。此外，这些发现促使开发新的方法，这些方法将 MIL 的优雅假设与经典弱监督方法观察到的更高性能相结合。我们在 https://github.com/KatherLab/HIA 上公开了所有的源代码，允许将所有方法轻松应用于任何类似的任务。
