Avram Oren, Durmus Berkin, Rakocz Nadav, Corradetti Giulia, An Ulzee, Nittala Muneeswar G, Terway Prerit, Rudas Akos, Chen Zeyuan Johnson, Wakatsuki Yu, Hirabayashi Kazutaka, Velaga Swetha, Tiosano Liran, Corvi Federico, Verma Aditya, Karamat Ayesha, Lindenberg Sophiana, Oncel Deniz, Almidani Louay, Hull Victoria, Fasih-Ahmad Sohaib, Esmaeilkhanian Houri, Cannesson Maxime, Wykoff Charles C, Rahmani Elior, Arnold Corey W, Zhou Bolei, Zaitlen Noah, Gronau Ilan, Sankararaman Sriram, Chiang Jeffrey N, Sadda Srinivas R, Halperin Eran
Department of Computational Medicine, University of California, Los Angeles, Los Angeles, CA, USA.
Department of Computer Science, University of California, Los Angeles, Los Angeles, CA, USA.
Nat Biomed Eng. 2025 Apr;9(4):507-520. doi: 10.1038/s41551-024-01257-9. Epub 2024 Oct 1.
The application of machine learning to tasks involving volumetric biomedical imaging is constrained by the limited availability of annotated datasets of three-dimensional (3D) scans for model training. Here we report a deep-learning model pre-trained on 2D scans (for which annotated data are relatively abundant) that accurately predicts disease-risk factors from 3D medical-scan modalities. The model, which we named SLIViT (for 'slice integration by vision transformer'), preprocesses a given volumetric scan into 2D images, extracts their feature map and integrates it into a single prediction. We evaluated the model in eight different learning tasks, including classification and regression for six datasets involving four volumetric imaging modalities (computed tomography, magnetic resonance imaging, optical coherence tomography and ultrasound). SLIViT consistently outperformed domain-specific state-of-the-art models and was typically as accurate as clinical specialists who had spent considerable time manually annotating the analysed scans. Automating diagnosis tasks involving volumetric scans may save valuable clinician hours, reduce data acquisition costs and duration, and help expedite medical research and clinical applications.
将机器学习应用于涉及体积生物医学成像的任务，受到用于模型训练的三维（3D）扫描注释数据集有限的限制。在此，我们报告了一种在二维扫描（注释数据相对丰富）上预训练的深度学习模型，该模型能从3D医学扫描模态中准确预测疾病风险因素。我们将该模型命名为SLIViT（即“基于视觉Transformer的切片整合”），它将给定的体积扫描预处理为二维图像，提取其特征图并将其整合为单个预测。我们在八项不同的学习任务中评估了该模型，包括对涉及四种体积成像模态（计算机断层扫描、磁共振成像、光学相干断层扫描和超声）的六个数据集进行分类和回归。SLIViT始终优于特定领域的最先进模型，并且通常与花费大量时间手动注释分析扫描的临床专家一样准确。自动化涉及体积扫描的诊断任务可以节省临床医生的宝贵时间，降低数据采集成本和持续时间，并有助于加快医学研究和临床应用。
