Simon Benjamin D, Ozyoruk Kutsev Bengisu, Gelikman David G, Harmon Stephanie A, Türkbey Barış
Molecular Imaging Branch, National Cancer Institute, National Institutes of Health, Bethesda, USA.
University of Oxford, Institute of Biomedical Engineering, Department Engineering Science, Oxford, UK.
Diagn Interv Radiol. 2024 Oct 1. doi: 10.4274/dir.2024.242631.
With the ongoing revolution of artificial intelligence (AI) in medicine, the impact of AI in radiology is more pronounced than ever. An increasing number of technical and clinical AI-focused studies are published each day. As these tools inevitably affect patient care and physician practices, it is crucial that radiologists become more familiar with the leading strategies and underlying principles of AI. Multimodal AI models can combine both imaging and clinical metadata and are quickly becoming a popular approach that is being integrated into the medical ecosystem. This narrative review covers major concepts of multimodal AI through the lens of recent literature. We discuss emerging frameworks, including graph neural networks, which allow for explicit learning from non-Euclidean relationships, and transformers, which allow for parallel computation that scales, highlighting existing literature and advocating for a focus on emerging architectures. We also identify key pitfalls in current studies, including issues with taxonomy, data scarcity, and bias. By informing radiologists and biomedical AI experts about existing practices and challenges, we hope to guide the next wave of imaging-based multimodal AI research.
随着人工智能（AI）在医学领域的不断变革，AI在放射学中的影响比以往任何时候都更加显著。每天都有越来越多以AI技术和临床应用为重点的研究发表。由于这些工具不可避免地会影响患者护理和医生的实践，放射科医生更加熟悉AI的主要策略和基本原理至关重要。多模态AI模型可以结合成像和临床元数据，并且正迅速成为一种被融入医疗生态系统的流行方法。这篇叙述性综述通过近期文献探讨了多模态AI的主要概念。我们讨论了新兴框架，包括允许从非欧几里得关系中进行显式学习的图神经网络，以及允许进行可扩展并行计算的Transformer，突出了现有文献并倡导关注新兴架构。我们还指出了当前研究中的关键缺陷，包括分类法问题、数据稀缺和偏差。通过向放射科医生和生物医学AI专家介绍现有实践和挑战，我们希望引导下一波基于成像的多模态AI研究。
