Rehman Azaan, Zhovmer Alexander, Sato Ryo, Mukouyama Yoh-Suke, Chen Jiji, Rissone Alberto, Puertollano Rosa, Liu Jiamin, Vishwasrao Harshad D, Shroff Hari, Combs Christian A, Xue Hui
Office of AI Research, National Heart, Lung and Blood Institute (NHLBI), National Institutes of Health (NIH), Bethesda, MD, 20892, USA.
Center for Biologics Evaluation and Research, U.S. Food and Drug Administration (FDA), Silver Spring, MD, 20903, USA.
Sci Rep. 2024 Aug 6;14(1):18184. doi: 10.1038/s41598-024-68918-2.
Deep neural networks can improve the quality of fluorescence microscopy images. Previous methods, based on Convolutional Neural Networks (CNNs), require time-consuming training of individual models for each experiment, impairing their applicability and generalization. In this study, we propose a novel imaging-transformer based model, Convolutional Neural Network Transformer (CNNT), that outperforms CNN based networks for image denoising. We train a general CNNT based backbone model from pairwise high-low Signal-to-Noise Ratio (SNR) image volumes, gathered from a single type of fluorescence microscope, an instant Structured Illumination Microscope. Fast adaptation to new microscopes is achieved by fine-tuning the backbone on only 5-10 image volume pairs per new experiment. Results show that the CNNT backbone and fine-tuning scheme significantly reduces training time and improves image quality, outperforming models trained using only CNNs such as 3D-RCAN and Noise2Fast. We show three examples of efficacy of this approach in wide-field, two-photon, and confocal fluorescence microscopy.
深度神经网络可以提高荧光显微镜图像的质量。以前基于卷积神经网络（CNN）的方法，需要针对每个实验对单个模型进行耗时的训练，这削弱了它们的适用性和通用性。在本研究中，我们提出了一种基于成像变压器的新型模型，即卷积神经网络变压器（CNNT），它在图像去噪方面优于基于CNN的网络。我们从成对的高-低信噪比（SNR）图像体中训练一个基于通用CNNT的主干模型，这些图像体来自单一类型的荧光显微镜，即即时结构照明显微镜。通过在每个新实验中仅对5-10对图像体对主干进行微调，实现了对新显微镜的快速适应。结果表明，CNNT主干和微调方案显著减少了训练时间并提高了图像质量，优于仅使用CNN训练的模型，如3D-RCAN和Noise2Fast。我们展示了这种方法在宽场、双光子和共聚焦荧光显微镜中的三个有效性示例。
