Deng Chao, Zhang Yuanlong, Mao Yifeng, Fan Jingtao, Suo Jinli, Zhang Zhili, Dai Qionghai
IEEE Trans Pattern Anal Mach Intell. 2021 Apr;43(4):1380-1393. doi: 10.1109/TPAMI.2019.2946567. Epub 2021 Mar 5.
Compressive sensing technique allows capturing fast phenomena at a much higher frame rate than the camera sensor, by recovering a frame sequence from their encoded combination. However, most conventional compressive video sensing methods limit the achieved frame rate improvement to tenfold and only support low resolution recovery. Making use of the camera's redundant spatial resolution for further frame rate improve, here we report a novel compressive video acquisition technique termed Sinusoidal Sampling Enhanced Compressive Camera (S2EC2) to encode denser frames within a snapshot. Specifically, we decompose the dense frames into groups and apply combinational coding: random codes within each group for compressive acquisition; group specific sinusoidal codes to multiplex different groups onto the high resolution sensor. The sinusoidal codes designed for these groups would shift their frequency components by different offsets in the Fourier domain and staggered the dominant frequencies of the coded measurements of these groups. Correspondingly, the reconstruction successfully separate coded measurements of different groups and recovers frames within each group. Besides, we also solve the implementation problem of insufficient gray scale spatial light modulation speed, and build a prototype achieving 2000 fps reconstruction with a 15.6 fps camera (the actual compression ratio is 0.009). The extensive experiments validate the proposed approach.
压缩感知技术通过从编码组合中恢复帧序列,能够以比相机传感器高得多的帧率捕捉快速现象。然而,大多数传统的压缩视频感知方法将实现的帧率提升限制在十倍以内,并且仅支持低分辨率恢复。为了利用相机的冗余空间分辨率进一步提高帧率,我们在此报告一种新颖的压缩视频采集技术,称为正弦采样增强压缩相机(S2EC2),用于在单个快照内编码更密集的帧。具体而言,我们将密集帧分解为组并应用组合编码:每组内使用随机码进行压缩采集;使用特定于组的正弦码将不同组复用至高分辨率传感器上。为这些组设计的正弦码会在傅里叶域中将其频率分量偏移不同的量,并使这些组的编码测量的主导频率交错排列。相应地,重建过程成功分离不同组的编码测量并恢复每组内的帧。此外,我们还解决了灰度空间光调制速度不足的实现问题,并构建了一个原型,该原型使用15.6帧每秒的相机实现了2000帧每秒的重建(实际压缩比为0.009)。大量实验验证了所提出的方法。