Department of Automation, Tsinghua University, Beijing, China.
National Laboratory of Biomacromolecules, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing, China.
Nat Methods. 2021 Feb;18(2):194-202. doi: 10.1038/s41592-020-01048-5. Epub 2021 Jan 21.
Deep neural networks have enabled astonishing transformations from low-resolution (LR) to super-resolved images. However, whether, and under what imaging conditions, such deep-learning models outperform super-resolution (SR) microscopy is poorly explored. Here, using multimodality structured illumination microscopy (SIM), we first provide an extensive dataset of LR-SR image pairs and evaluate the deep-learning SR models in terms of structural complexity, signal-to-noise ratio and upscaling factor. Second, we devise the deep Fourier channel attention network (DFCAN), which leverages the frequency content difference across distinct features to learn precise hierarchical representations of high-frequency information about diverse biological structures. Third, we show that DFCAN's Fourier domain focalization enables robust reconstruction of SIM images under low signal-to-noise ratio conditions. We demonstrate that DFCAN achieves comparable image quality to SIM over a tenfold longer duration in multicolor live-cell imaging experiments, which reveal the detailed structures of mitochondrial cristae and nucleoids and the interaction dynamics of organelles and cytoskeleton.
深度神经网络已经实现了从低分辨率（LR）到超分辨率（SR）图像的惊人转换。然而，在什么成像条件下，深度学习模型优于 SR 显微镜，这一点还没有得到很好的探索。在这里，我们首先使用多模态结构光照明显微镜（SIM），提供了广泛的 LR-SR 图像对数据集，并从结构复杂性、信噪比和放大倍数等方面评估了深度学习 SR 模型。其次，我们设计了深度傅里叶通道注意力网络（DFCAN），该网络利用不同特征之间的频率内容差异，学习关于不同生物结构高频信息的精确分层表示。第三，我们表明，DFCAN 的傅里叶域聚焦能够在低信噪比条件下实现稳健的 SIM 图像重建。我们证明了 DFCAN 在多色活细胞成像实验中，在长达十倍的时间内，能够实现与 SIM 相当的图像质量，揭示了线粒体嵴和核仁的详细结构以及细胞器和细胞骨架的相互作用动态。
