Cell Biology and Biophysics Unit, European Molecular Biology Laboratory, Heidelberg, Germany.
Department of Informatics, Technical University of Munich, Garching, Germany.
Nat Methods. 2021 May;18(5):557-563. doi: 10.1038/s41592-021-01136-0. Epub 2021 May 7.
Visualizing dynamic processes over large, three-dimensional fields of view at high speed is essential for many applications in the life sciences. Light-field microscopy (LFM) has emerged as a tool for fast volumetric image acquisition, but its effective throughput and widespread use in biology has been hampered by a computationally demanding and artifact-prone image reconstruction process. Here, we present a framework for artificial intelligence-enhanced microscopy, integrating a hybrid light-field light-sheet microscope and deep learning-based volume reconstruction. In our approach, concomitantly acquired, high-resolution two-dimensional light-sheet images continuously serve as training data and validation for the convolutional neural network reconstructing the raw LFM data during extended volumetric time-lapse imaging experiments. Our network delivers high-quality three-dimensional reconstructions at video-rate throughput, which can be further refined based on the high-resolution light-sheet images. We demonstrate the capabilities of our approach by imaging medaka heart dynamics and zebrafish neural activity with volumetric imaging rates up to 100 Hz.
在生命科学的许多应用中，高速可视化大三维视场中的动态过程至关重要。光场显微镜（LFM）已成为快速体积图像采集的工具，但由于图像重建过程计算要求高且容易出现伪影，其有效吞吐量和在生物学中的广泛应用受到了阻碍。在这里，我们提出了一个人工智能增强显微镜的框架，集成了混合光场光片显微镜和基于深度学习的体积重建。在我们的方法中，同时获取的高分辨率二维光片图像连续作为训练数据和验证数据，用于在扩展的体积延时成像实验期间重建原始 LFM 数据的卷积神经网络。我们的网络以视频帧率的吞吐量提供高质量的三维重建，这些重建可以基于高分辨率光片图像进一步细化。我们通过以高达 100 Hz 的体积成像速率对斑马鱼心脏动力学和斑马鱼神经活动进行成像来证明我们方法的能力。
