Li Junyuan, Wang Wenying, Tivnan Matthew, Sulam Jeremias, Prince Jerry L, McNitt-Gray Michael, Stayman J Webster, Gang Grace J
Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland, USA.
Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, Maryland, USA.
Proc SPIE Int Soc Opt Eng. 2022 Jun;12304. doi: 10.1117/12.2646371. Epub 2022 Oct 17.
The rapid development of deep-learning methods in medical imaging has called for an analysis method suitable for non-linear and data-dependent algorithms. In this work, we investigate a local linearity analysis where a complex neural network can be represented as piecewise linear systems. We recognize that a large number of neural networks consists of alternating linear layers and rectified linear unit (ReLU) activations, and are therefore strictly piecewise linear. We investigated the extent of these locally linear regions by gradually adding perturbations to an operating point. For this work, we explored perturbations based on image features of interest, including lesion contrast, background, and additive noise. We then developed strategies to extend these strictly locally linear regions to include neighboring linear regions with similar gradients. Using these approximately linear regions, we applied singular value decomposition (SVD) analysis to each local linear system to investigate and explain the overall nonlinear and data-dependent behaviors of neural networks. The analysis was applied to an example CT denoising algorithm trained on thorax CT scans. We observed that the strictly local linear regions are highly sensitive to small signal perturbations. Over a range of lesion contrast from 0.007 to 0.04 mm, there is a total of 33992 linear regions. The Jacobians are also shift-variant. However, the Jacobians of neighboring linear regions are very similar. By combining linear regions with similar Jacobians, we narrowed down the number of approximately linear regions to four over lesion contrast from 0.001 to 0.08 mm. The SVD analysis to different linear regions revealed denoising behavior that is highly dependent on the background intensity. Analysis further identified greater amount of noise reduction in uniform regions compared to lesion edges. In summary, the local linearity analysis framework we proposed has the potential for us to better characterize and interpret the non-linear and data-dependent behaviors of neural networks.
深度学习方法在医学成像领域的快速发展，促使人们需要一种适用于非线性和数据依赖型算法的分析方法。在这项工作中，我们研究了一种局部线性分析方法，其中复杂的神经网络可表示为分段线性系统。我们认识到，大量神经网络由交替的线性层和整流线性单元（ReLU）激活组成，因此严格来说是分段线性的。我们通过逐步向一个工作点添加扰动来研究这些局部线性区域的范围。对于这项工作，我们基于感兴趣的图像特征探索了扰动，包括病变对比度、背景和加性噪声。然后，我们开发了一些策略来扩展这些严格的局部线性区域，以纳入具有相似梯度的相邻线性区域。利用这些近似线性区域，我们对每个局部线性系统应用奇异值分解（SVD）分析，以研究和解释神经网络的整体非线性和数据依赖行为。该分析应用于一个在胸部CT扫描上训练的CT去噪算法示例。我们观察到，严格的局部线性区域对小信号扰动高度敏感。在病变对比度从0.007到0.04毫米的范围内，共有33992个线性区域。雅可比矩阵也是移位变体的。然而，相邻线性区域的雅可比矩阵非常相似。通过组合具有相似雅可比矩阵的线性区域，我们将病变对比度从0.001到0.08毫米范围内的近似线性区域数量减少到了四个。对不同线性区域的SVD分析揭示了去噪行为高度依赖于背景强度。分析进一步表明，与病变边缘相比，均匀区域的降噪量更大。总之，我们提出的局部线性分析框架有可能使我们更好地表征和解释神经网络的非线性和数据依赖行为。
