School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, Shanghai, China.
Nat Methods. 2024 Aug;21(8):1558-1567. doi: 10.1038/s41592-024-02244-3. Epub 2024 Apr 12.
Fluorescence microscopy-based image restoration has received widespread attention in the life sciences and has led to significant progress, benefiting from deep learning technology. However, most current task-specific methods have limited generalizability to different fluorescence microscopy-based image restoration problems. Here, we seek to improve generalizability and explore the potential of applying a pretrained foundation model to fluorescence microscopy-based image restoration. We provide a universal fluorescence microscopy-based image restoration (UniFMIR) model to address different restoration problems, and show that UniFMIR offers higher image restoration precision, better generalization and increased versatility. Demonstrations on five tasks and 14 datasets covering a wide range of microscopy imaging modalities and biological samples demonstrate that the pretrained UniFMIR can effectively transfer knowledge to a specific situation via fine-tuning, uncover clear nanoscale biomolecular structures and facilitate high-quality imaging. This work has the potential to inspire and trigger new research highlights for fluorescence microscopy-based image restoration.
基于荧光显微镜的图像恢复在生命科学领域受到了广泛关注，并取得了重大进展，这得益于深度学习技术。然而，目前大多数特定于任务的方法对于不同的基于荧光显微镜的图像恢复问题的通用性有限。在这里，我们寻求提高通用性，并探索应用预先训练的基础模型进行基于荧光显微镜的图像恢复的潜力。我们提供了一个通用的基于荧光显微镜的图像恢复（UniFMIR）模型来解决不同的恢复问题，并表明 UniFMIR 提供了更高的图像恢复精度、更好的泛化能力和更高的多功能性。在涵盖广泛显微镜成像模式和生物样本的五个任务和 14 个数据集上的演示表明，预先训练的 UniFMIR 可以通过微调有效地将知识转移到特定情况，揭示清晰的纳米级生物分子结构，并促进高质量成像。这项工作有可能激发和引发基于荧光显微镜的图像恢复的新研究重点。
