Tahir Waleed, Wang Hao, Tian Lei
Department of Electrical and Computer Engineering, Boston University, Boston, MA, 02215, USA.
Department of Biomedical Engineering, Boston University, Boston, MA, 02215, USA.
Light Sci Appl. 2022 Feb 24;11(1):42. doi: 10.1038/s41377-022-00730-x.
Deep learning has been broadly applied to imaging in scattering applications. A common framework is to train a descattering network for image recovery by removing scattering artifacts. To achieve the best results on a broad spectrum of scattering conditions, individual "expert" networks need to be trained for each condition. However, the expert's performance sharply degrades when the testing condition differs from the training. An alternative brute-force approach is to train a "generalist" network using data from diverse scattering conditions. It generally requires a larger network to encapsulate the diversity in the data and a sufficiently large training set to avoid overfitting. Here, we propose an adaptive learning framework, termed dynamic synthesis network (DSN), which dynamically adjusts the model weights and adapts to different scattering conditions. The adaptability is achieved by a novel "mixture of experts" architecture that enables dynamically synthesizing a network by blending multiple experts using a gating network. We demonstrate the DSN in holographic 3D particle imaging for a variety of scattering conditions. We show in simulation that our DSN provides generalization across a continuum of scattering conditions. In addition, we show that by training the DSN entirely on simulated data, the network can generalize to experiments and achieve robust 3D descattering. We expect the same concept can find many other applications, such as denoising and imaging in scattering media. Broadly, our dynamic synthesis framework opens up a new paradigm for designing highly adaptive deep learning and computational imaging techniques.
深度学习已广泛应用于散射应用中的成像。一个常见的框架是通过去除散射伪影来训练一个去散射网络以进行图像恢复。为了在广泛的散射条件下取得最佳效果，需要针对每种条件训练单独的“专家”网络。然而，当测试条件与训练条件不同时，专家网络的性能会急剧下降。另一种强力方法是使用来自不同散射条件的数据训练一个“通才”网络。这通常需要一个更大的网络来封装数据中的多样性，以及一个足够大的训练集以避免过拟合。在此，我们提出一种自适应学习框架，称为动态合成网络（DSN），它能动态调整模型权重并适应不同的散射条件。这种适应性是通过一种新颖的“专家混合”架构实现的，该架构通过一个门控网络融合多个专家来动态合成一个网络。我们在全息三维粒子成像中针对各种散射条件展示了DSN。我们在模拟中表明，我们的DSN能在连续的散射条件下实现泛化。此外，我们表明通过完全在模拟数据上训练DSN，该网络可以推广到实验中并实现稳健的三维去散射。我们期望同样的概念能找到许多其他应用，例如散射介质中的去噪和成像。总体而言，我们的动态合成框架为设计高度自适应的深度学习和计算成像技术开辟了一种新范式。
