State Key Lab of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China.
Sensors (Basel). 2019 Jan 16;19(2):344. doi: 10.3390/s19020344.
A binocular vision system is a common perception component of an intelligent vehicle. Benefiting from the biomimetic structure, the system is simple and effective. Which are extremely snesitive on external factors, especially missing vision signals. In this paper, a virtual view-generation algorithm based on generative adversarial networks (GAN) is proposed to enhance the robustness of binocular vision systems. The proposed model consists of two parts: generative network and discriminator network. To improve the quality of a virtual view, a generative network structure based on 3D convolutional neural networks (3D-CNN) and attentive mechanisms is introduced to extract the time-series features from image sequences. To avoid gradient vanish during training, the dense block structure is utilized to improve the discriminator network. Meanwhile, three kinds of image features, including image edge, depth map and optical flow are extracted to constrain the supervised training of model. The final results on KITTI and Cityscapes datasets demonstrate that our algorithm outperforms conventional methods, and the missing vision signal can be replaced by a generated virtual view.
双目视觉系统是智能车辆常见的感知组件。得益于仿生结构，该系统简单有效，但对外界因素极其敏感，特别是缺失视觉信号。本文提出了一种基于生成对抗网络（GAN）的虚拟视图生成算法，以增强双目视觉系统的鲁棒性。所提出的模型由生成网络和判别网络两部分组成。为了提高虚拟视图的质量，引入了一种基于三维卷积神经网络（3D-CNN）和注意机制的生成网络结构，从图像序列中提取时间序列特征。为了避免训练过程中的梯度消失，利用密集块结构来改进判别网络。同时，提取三种图像特征，包括图像边缘、深度图和光流，以约束模型的监督训练。在 KITTI 和 Cityscapes 数据集上的最终结果表明，我们的算法优于传统方法，可以用生成的虚拟视图来替代缺失的视觉信号。
