Zhou Haoyu, Song Yan, Yao Zhiming, Hei Dongwei, Li Yang, Duan Baojun, Liu Yinong, Sheng Liang
Opt Express. 2024 Apr 22;32(9):16333-16350. doi: 10.1364/OE.519872.
Compressed ultrafast photography (CUP) is a computational imaging technology capable of capturing transient scenes in picosecond scale with a sequence depth of hundreds of frames. Since the inverse problem of CUP is an ill-posed problem, it is challenging to further improve the reconstruction quality under the condition of high noise level and compression ratio. In addition, there are many articles adding an external charge-coupled device (CCD) camera to the CUP system to form the time-unsheared view because the added constraint can improve the reconstruction quality of images. However, since the images are collected by different cameras, slight affine transformation may have great impacts on the reconstruction quality. Here, we propose an algorithm that combines the time-unsheared image constraint CUP system with unsupervised neural networks. Image registration network is also introduced into the network framework to learn the affine transformation parameters of input images. The proposed algorithm effectively utilizes the implicit image prior in the neural network as well as the extra hardware prior information brought by the time-unsheared view. Combined with image registration network, this joint learning model enables our proposed algorithm to further improve the quality of reconstructed images without training datasets. The simulation and experiment results demonstrate the application prospect of our algorithm in ultrafast event capture.
压缩超快摄影（CUP）是一种计算成像技术，能够以皮秒级的时间尺度捕捉瞬态场景，具有数百帧的序列深度。由于CUP的逆问题是一个不适定问题，在高噪声水平和压缩率条件下进一步提高重建质量具有挑战性。此外，有许多文章在CUP系统中添加外部电荷耦合器件（CCD）相机以形成时间未剪切视图，因为添加的约束可以提高图像的重建质量。然而，由于图像是由不同的相机采集的，轻微的仿射变换可能会对重建质量产生很大影响。在此，我们提出一种将时间未剪切图像约束CUP系统与无监督神经网络相结合的算法。图像配准网络也被引入到网络框架中，以学习输入图像的仿射变换参数。所提出的算法有效地利用了神经网络中的隐式图像先验以及时间未剪切视图带来的额外硬件先验信息。结合图像配准网络，这种联合学习模型使我们提出的算法能够在没有训练数据集的情况下进一步提高重建图像的质量。仿真和实验结果证明了我们算法在超快事件捕捉中的应用前景。
