Deep learning polarimetric three-dimensional integral imaging object recognition in adverse environmental conditions.

Polarimetric imaging is useful for object recognition and material classification because of its ability to discriminate objects based on polarimetric signatures of materials. Polarimetric imaging of an object captures important physical properties such as shape and surface properties and can be effective even in low light environments. Integral imaging is a passive three-dimensional (3D) imaging approach that takes advantage of multiple 2D imaging perspectives to perform 3D reconstruction. In this paper, we propose a unified polarimetric detection and classification of objects in degraded environments such as low light and the presence of occlusion. This task is accomplished using a deep learning model for 3D polarimetric integral imaging data captured in the visible spectral domain. The neural network system is designed and trained for 3D object detection and classification using polarimetric integral images. We compare the detection and classification results between polarimetric and non-polarimetric 2D and 3D imaging. The system performance in degraded environmental conditions is evaluated using average miss rate, average precision, and F-1 score. The results indicate that for the experiments we have performed, polarimetric 3D integral imaging outperforms 2D polarimetric imaging as well as non-polarimetric 2D and 3D imaging for object recognition in adverse conditions such as low light and occlusions. To the best of our knowledge, this is the first report for polarimetric 3D object recognition in low light environments and occlusions using a deep learning-based integral imaging. The proposed approach is attractive because low light polarimetric object recognition in the visible spectral band benefits from much higher spatial resolution, more compact optics, and lower system cost compared with long wave infrared imaging which is the conventional imaging approach for low light environments.