Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Mass General Hospital, Harvard Medical School, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Cancer Data Science Program, Dana-Farber/Harvard Cancer Institute, Boston, MA, USA.
Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Mass General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Cancer Data Science Program, Dana-Farber/Harvard Cancer Institute, Boston, MA, USA; Department of Electrical Engineering and Computer Science, MIT, Cambridge, MA, USA.
Cancer Cell. 2022 Aug 8;40(8):865-878.e6. doi: 10.1016/j.ccell.2022.07.004.
The rapidly emerging field of computational pathology has demonstrated promise in developing objective prognostic models from histology images. However, most prognostic models are either based on histology or genomics alone and do not address how these data sources can be integrated to develop joint image-omic prognostic models. Additionally, identifying explainable morphological and molecular descriptors from these models that govern such prognosis is of interest. We use multimodal deep learning to jointly examine pathology whole-slide images and molecular profile data from 14 cancer types. Our weakly supervised, multimodal deep-learning algorithm is able to fuse these heterogeneous modalities to predict outcomes and discover prognostic features that correlate with poor and favorable outcomes. We present all analyses for morphological and molecular correlates of patient prognosis across the 14 cancer types at both a disease and a patient level in an interactive open-access database to allow for further exploration, biomarker discovery, and feature assessment.
计算病理学是一个迅速发展的领域，它在从组织学图像中开发客观的预后模型方面显示出了前景。然而，大多数预后模型要么仅基于组织学，要么仅基于基因组学，而没有解决如何整合这些数据源来开发联合图像组学预后模型的问题。此外，从这些模型中识别出解释性的形态学和分子描述符来预测预后也是很有意义的。我们使用多模态深度学习联合研究了 14 种癌症类型的病理全切片图像和分子谱数据。我们的弱监督多模态深度学习算法能够融合这些异构模态，以预测结果，并发现与不良和良好预后相关的预后特征。我们在一个交互式的开放访问数据库中，在疾病和患者两个层面上，为 14 种癌症类型的患者预后的形态学和分子相关性提供了所有分析，以允许进一步的探索、生物标志物的发现和特征评估。
