Gui Dan, Chen Yunjiu, Kuang Weibing, Shang Mingtao, Zhang Yingjun, Huang Zhen-Li
Britton Chance Center and MoE Key Laboratory for Biomedical Photonics, School of Engineering Sciences, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, Wuhan 430074, China.
School of Electronic Engineering, Wuhan Vocational College of Software and Engineering, Wuhan 430205, China.
Biomed Opt Express. 2022 May 16;13(6):3401-3415. doi: 10.1364/BOE.459198. eCollection 2022 Jun 1.
Real-time multi-emitter fitting is a key technology for advancing super-resolution localization microscopy (SRLM), especially when it is necessary to achieve dynamic imaging quality control and/or optimization of experimental conditions. However, with the increase of activation densities, the requirements in the computing resources would increase rapidly due to the complexity of the fitting algorithms, making it difficult to realize real-time multi-emitter fitting for emitter density more than 0.6 mol/µm in large field of view (FOV), even after acceleration with the popular Graphics Processing Unit (GPU) computation. Here we adopt the task parallelism strategy in computer science to construct a Peripheral Component Interconnect Express (PCIe) based all-in-one heterogeneous computing platform (AIO-HCP), where the data between two major parallel computing hardware, Field Programmable Gate Array (FPGA) and GPU, are interacted directly and executed simultaneously. Using simulated and experimental data, we verify that AIO-HCP could achieve a data throughput of up to ∼ 1.561 GB/s between FPGA and GPU. With this new platform, we develop a multi-emitter fitting method, called AIO-STORM, under big data stream parallel scheduling. We show that AIO-STORM is capable of providing real-time image processing on raw images with 100 µm × 100 µm FOV, 10 ms exposure time and 5.5 mol/µm structure density, without scarifying image quality. This study overcomes the data throughput limitation of heterogeneous devices, demonstrates the power of the PCIe-based heterogeneous computation platform, and offers opportunities for multi-scale stitching of super-resolution images.
实时多发射器拟合是推动超分辨率定位显微镜（SRLM）发展的一项关键技术，尤其是在需要实现动态成像质量控制和/或优化实验条件时。然而，随着激活密度的增加，由于拟合算法的复杂性，对计算资源的需求会迅速增长，这使得在大视场（FOV）中难以实现对发射体密度超过0.6 mol/µm的实时多发射器拟合，即使采用流行的图形处理单元（GPU）计算进行加速后也是如此。在此，我们采用计算机科学中的任务并行策略，构建了一个基于外围组件互连高速（PCIe）的一体化异构计算平台（AIO-HCP），其中两个主要并行计算硬件——现场可编程门阵列（FPGA）和GPU之间的数据直接交互并同时执行。使用模拟和实验数据，我们验证了AIO-HCP在FPGA和GPU之间可实现高达约1.561 GB/s的数据吞吐量。利用这个新平台，我们在大数据流并行调度下开发了一种名为AIO-STORM的多发射器拟合方法。我们表明，AIO-STORM能够在100 µm×100 µm视场、10 ms曝光时间和5.5 mol/µm结构密度的原始图像上进行实时图像处理，且不牺牲图像质量。本研究克服了异构设备的数据吞吐量限制，展示了基于PCIe的异构计算平台的能力，并为超分辨率图像的多尺度拼接提供了机会。
