Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA.
Department of Pathology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; Cancer Program, Broad Institute of Harvard and MIT, Cambridge, MA, USA; Data Science Program, Dana-Farber Cancer Institute, Boston, MA, USA; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
Cancer Cell. 2022 Oct 10;40(10):1095-1110. doi: 10.1016/j.ccell.2022.09.012.
In oncology, the patient state is characterized by a whole spectrum of modalities, ranging from radiology, histology, and genomics to electronic health records. Current artificial intelligence (AI) models operate mainly in the realm of a single modality, neglecting the broader clinical context, which inevitably diminishes their potential. Integration of different data modalities provides opportunities to increase robustness and accuracy of diagnostic and prognostic models, bringing AI closer to clinical practice. AI models are also capable of discovering novel patterns within and across modalities suitable for explaining differences in patient outcomes or treatment resistance. The insights gleaned from such models can guide exploration studies and contribute to the discovery of novel biomarkers and therapeutic targets. To support these advances, here we present a synopsis of AI methods and strategies for multimodal data fusion and association discovery. We outline approaches for AI interpretability and directions for AI-driven exploration through multimodal data interconnections. We examine challenges in clinical adoption and discuss emerging solutions.
在肿瘤学中，患者状态的特点是涉及多种模式，包括放射学、组织学和基因组学以及电子健康记录。当前的人工智能 (AI) 模型主要在单一模式领域运作，忽略了更广泛的临床背景，这不可避免地限制了它们的潜力。不同数据模式的整合为提高诊断和预后模型的稳健性和准确性提供了机会，使 AI 更接近临床实践。AI 模型还能够发现适合解释患者结局或治疗耐药性差异的新模式，从而促进了新型生物标志物和治疗靶点的发现。为了支持这些进展，我们在这里总结了用于多模态数据融合和关联发现的 AI 方法和策略。我们概述了用于 AI 可解释性的方法以及通过多模态数据互联进行 AI 驱动探索的方向。我们研究了临床应用的挑战，并讨论了新兴的解决方案。
