Zhang Baoyi, Wan Zhuoya, Luo Yige, Zhao Xi, Samayoa Josue, Zhao Weilong, Wu Si
AbbVie Bay Area, South San Francisco, CA, United States.
AbbVie, Inc., North Chicago, IL, United States.
Front Pharmacol. 2025 Aug 20;16:1609079. doi: 10.3389/fphar.2025.1609079. eCollection 2025.
In clinical practice, a variety of techniques are employed to generate diverse data types for each cancer patient. These data types, spanning clinical, genomics, imaging, and other modalities, exhibit significant differences and possess distinct data structures. Therefore, most current analyses focus on a single data modality, limiting the potential of fully utilizing all available data and providing comprehensive insights. Artificial intelligence (AI) methods, adept at handling complex data structures, offer a powerful approach to efficiently integrate multimodal data. The insights derived from such models may ultimately expedite advancements in patient diagnosis, prognosis, and treatment responses. Here, we provide an overview of current advanced multimodal integration strategies and the related clinical potential in oncology field. We start from the key processing methods for single data modalities such as multi-omics, imaging data, and clinical notes. We then include diverse AI methods, covering traditional machine learning, representation learning, and vision language model, tailored to each distinct data modality. We further elaborate on popular multimodal integration strategies and discuss the related strength and weakness. Finally, we explore potential clinical applications including early detection/diagnosis, biomarker discovery, and prediction of clinical outcome. Additionally, we discuss ongoing challenges and outline potential future directions in the field.
在临床实践中，会采用多种技术为每位癌症患者生成不同的数据类型。这些数据类型涵盖临床、基因组学、影像学及其他模式，存在显著差异且具有独特的数据结构。因此，当前大多数分析聚焦于单一数据模式，限制了充分利用所有可用数据并提供全面见解的潜力。擅长处理复杂数据结构的人工智能（AI）方法，为高效整合多模式数据提供了有力途径。从此类模型中获得的见解最终可能会加速患者诊断、预后及治疗反应方面的进展。在此，我们概述当前肿瘤学领域先进的多模式整合策略及其相关临床潜力。我们从单数据模式的关键处理方法入手，如多组学、影像数据和临床记录。然后我们纳入各种AI方法，包括针对每种不同数据模式的传统机器学习、表征学习和视觉语言模型。我们进一步详细阐述流行的多模式整合策略，并讨论其相关的优缺点。最后，我们探索潜在的临床应用，包括早期检测/诊断、生物标志物发现和临床结果预测。此外，我们讨论当前面临的挑战并概述该领域潜在的未来方向。
