Zheng Siming, Wang Chunyang, Yuan Xin, Xin Huolin L
Department of Physics and Astronomy, University of California, Irvine, Irvine, CA, USA.
Bell Labs, 600 Mountain Avenue, Murray Hill, NJ 07974, USA.
Patterns (N Y). 2021 Jun 24;2(7):100292. doi: 10.1016/j.patter.2021.100292. eCollection 2021 Jul 9.
The development of ultrafast detectors for electron microscopy (EM) opens a new door to exploring dynamics of nanomaterials; however, it raises grand challenges for big data processing and storage. Here, we combine deep learning and temporal compressive sensing (TCS) to propose a novel EM big data compression strategy. Specifically, TCS is employed to compress sequential EM images into a single compressed measurement; an end-to-end deep learning network is leveraged to reconstruct the original images. Owing to the significantly improved compression efficiency and built-in denoising capability of the deep learning framework over conventional JPEG compression, compressed videos with a compression ratio of up to 30 can be reconstructed with high fidelity. Using this approach, considerable encoding power, memory, and transmission bandwidth can be saved, allowing it to be deployed to existing detectors. We anticipate the proposed technique will have far-reaching applications in edge computing for EM and other imaging techniques.
用于电子显微镜（EM）的超快探测器的发展为探索纳米材料的动力学打开了一扇新的大门；然而，它给大数据处理和存储带来了巨大挑战。在此，我们将深度学习与时间压缩感知（TCS）相结合，提出了一种新颖的EM大数据压缩策略。具体而言，采用TCS将连续的EM图像压缩为单个压缩测量值；利用端到端深度学习网络重建原始图像。由于深度学习框架相较于传统JPEG压缩具有显著提高的压缩效率和内置去噪能力，因此可以高保真地重建压缩比高达30的压缩视频。使用这种方法，可以节省大量的编码能力、内存和传输带宽，使其能够部署到现有的探测器上。我们预计所提出的技术将在EM和其他成像技术的边缘计算中具有深远的应用。
