Ishii Shin, Lee Sehyung, Urakubo Hidetoshi, Kume Hideaki, Kasai Haruo
Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan.
ATR Neural Information Analysis Laboratories, Kyoto 619-0288, Japan.
Microscopy (Oxf). 2020 Apr 8;69(2):79-91. doi: 10.1093/jmicro/dfaa007.
Image processing is one of the most important applications of recent machine learning (ML) technologies. Convolutional neural networks (CNNs), a popular deep learning-based ML architecture, have been developed for image processing applications. However, the application of ML to microscopic images is limited as microscopic images are often 3D/4D, that is, the image sizes can be very large, and the images may suffer from serious noise generated due to optics. In this review, three types of feature reconstruction applications to microscopic images are discussed, which fully utilize the recent advancements in ML technologies. First, multi-frame super-resolution is introduced, based on the formulation of statistical generative model-based techniques such as Bayesian inference. Second, data-driven image restoration is introduced, based on supervised discriminative model-based ML technique. In this application, CNNs are demonstrated to exhibit preferable restoration performance. Third, image segmentation based on data-driven CNNs is introduced. Image segmentation has become immensely popular in object segmentation based on electron microscopy (EM); therefore, we focus on EM image processing.
图像处理是近期机器学习（ML）技术最重要的应用之一。卷积神经网络（CNN）是一种基于深度学习的流行ML架构，已被开发用于图像处理应用。然而，ML在微观图像上的应用受到限制，因为微观图像通常是3D/4D的，也就是说，图像尺寸可能非常大，并且图像可能会受到光学产生的严重噪声影响。在本综述中，讨论了三种应用于微观图像的特征重建方法，这些方法充分利用了ML技术的最新进展。首先，基于统计生成模型技术（如贝叶斯推理）的公式，引入了多帧超分辨率。其次，基于基于监督判别模型的ML技术，引入了数据驱动的图像恢复。在这个应用中，CNN被证明具有较好的恢复性能。第三，引入了基于数据驱动CNN的图像分割。图像分割在基于电子显微镜（EM）的目标分割中变得非常流行；因此，我们专注于EM图像处理。
