Wolf Tom Nuno, Pölsterl Sebastian, Wachinger Christian
The Lab for Artificial Intelligence in Medical Imaging (AI-Med), Department of Child and Adolescent Psychiatry, Ludwig-Maximilians-Universität, Nussbaumstraße 5, Munich 80336, Germany; Technical University of Munich, School of Medicine, Department of Radiology, Ismaninger Straße 22, Munich 81675, Germany.
The Lab for Artificial Intelligence in Medical Imaging (AI-Med), Department of Child and Adolescent Psychiatry, Ludwig-Maximilians-Universität, Nussbaumstraße 5, Munich 80336, Germany.
Neuroimage. 2022 Oct 15;260:119505. doi: 10.1016/j.neuroimage.2022.119505. Epub 2022 Jul 22.
Prior work on Alzheimer's Disease (AD) has demonstrated that convolutional neural networks (CNNs) can leverage the high-dimensional image information for diagnosing patients. Beside such data-driven approaches, many established biomarkers exist and are typically represented as tabular data, such as demographics, genetic alterations, or laboratory measurements from cerebrospinal fluid. However, little research has focused on the effective integration of tabular data into existing CNN architectures to improve patient diagnosis. We introduce the Dynamic Affine Feature Map Transform (DAFT), a general-purpose module for CNNs that incites or represses high-level concepts learned from a 3D image by conditioning feature maps of a convolutional layer on both a patient's image and tabular clinical information. This is achieved by using an auxiliary neural network that outputs a scaling factor and offset to dynamically apply an affine transformation to the feature maps of a convolutional layer. In our experiments on AD diagnosis and time-to-dementia prediction, we show that the DAFT is highly effective in combining 3D image and tabular information by achieving a mean balanced accuracy of 0.622 for diagnosis, and mean c-index of 0.748 for time-to-dementia prediction, thus outperforming all baseline methods. Finally, our extensive ablation study and empirical experiments reveal that the performance improvement due to the DAFT is robust with respect to many design choices.
先前关于阿尔茨海默病（AD）的研究表明，卷积神经网络（CNN）可以利用高维图像信息来诊断患者。除了这种数据驱动的方法外，还存在许多已确立的生物标志物，它们通常表示为表格数据，如人口统计学数据、基因改变或脑脊液的实验室测量值。然而，很少有研究专注于将表格数据有效整合到现有的CNN架构中以改善患者诊断。我们引入了动态仿射特征图变换（DAFT），这是一种用于CNN的通用模块，它通过根据患者的图像和表格临床信息对卷积层的特征图进行条件设定，来激发或抑制从3D图像中学到的高级概念。这是通过使用一个辅助神经网络来实现的，该网络输出一个缩放因子和偏移量，以动态地对卷积层的特征图应用仿射变换。在我们关于AD诊断和痴呆症发病时间预测的实验中，我们表明DAFT在结合3D图像和表格信息方面非常有效，诊断的平均平衡准确率达到0.622，痴呆症发病时间预测的平均c指数达到0.748，从而优于所有基线方法。最后，我们广泛的消融研究和实证实验表明，由于DAFT带来的性能提升在许多设计选择方面都具有稳健性。
