Ye Zhengnan, Xu Haisong, Huang Yiming, Yang Minhang
State Key Laboratory of Extreme Photonics and Instrumentation, College of Optical Science and Engineering, Zhejiang University, Hangzhou 310027, China.
Sensors (Basel). 2023 Aug 1;23(15):6856. doi: 10.3390/s23156856.
Multispectral imaging is valuable in many vision-related fields as it provides an additional modality to observe the world. Cameras equipped with multispectral filter arrays (MSFAs) are typically impractical for everyday use due to their intractable demosaicking and chromatic reproduction processes, which restrict their applicability beyond academic research. In this work, a novel MSFA design is proposed to enable dual-mode imaging for multispectral cameras. In addition to a conventional multispectral image, the camera is also able to produce a Bayer-formed RGB image from a single shot by grouping and merging adjacent pixels in the proposed MSFA, making it suitable for scenarios where display-ready RGB images are required. Furthermore, a two-stage optimization scheme is implemented to jointly optimize objective functions for both imaging modes. The evaluation results on multiple datasets suggest that the proposed MSFA design is able to simultaneously achieve competitive spectral reconstruction accuracy compared to elaborate multispectral cameras and chromatic accuracy compared to commercial RGB cameras.
多光谱成像在许多与视觉相关的领域都很有价值，因为它提供了一种观察世界的额外方式。配备多光谱滤光片阵列（MSFA）的相机由于其难以处理的去马赛克和色彩再现过程，通常不适合日常使用，这限制了它们在学术研究之外的适用性。在这项工作中，提出了一种新颖的MSFA设计，以实现多光谱相机的双模式成像。除了传统的多光谱图像外，通过在所提出的MSFA中对相邻像素进行分组和合并，相机还能够从单次拍摄中生成拜耳格式的RGB图像，使其适用于需要显示就绪RGB图像的场景。此外，实施了一种两阶段优化方案，以联合优化两种成像模式的目标函数。在多个数据集上的评估结果表明，所提出的MSFA设计能够同时实现与精密多光谱相机相比具有竞争力的光谱重建精度，以及与商用RGB相机相比具有竞争力的色彩精度。
