Psychology Division, Faculty of Natural Sciences, University of Stirling, Stirling, UK.
Computing Science and Mathematics Division, Faculty of Natural Sciences, University of Stirling, Stirling, UK.
J Vis. 2021 Jul 6;21(7):13. doi: 10.1167/jov.21.7.13.
The application of deep learning techniques has led to substantial progress in solving a number of critical problems in machine vision, including fundamental problems of scene segmentation and depth estimation. Here, we report a novel deep neural network model, capable of simultaneous scene segmentation and depth estimation from a pair of binocular images. By manipulating the arrangement of binocular image pairs, presenting the model with standard left-right image pairs, identical image pairs or swapped left-right images, we show that performance levels depend on the presence of appropriate binocular image arrangements. Segmentation and depth estimation performance are both impaired when images are swapped. Segmentation performance levels are maintained, however, for identical image pairs, despite the absence of binocular disparity information. Critically, these performance levels exceed those found for an equivalent, monocularly trained, segmentation model. These results provide evidence that binocular image differences support both the direct recovery of depth and segmentation information, and the enhanced learning of monocular segmentation signals. This finding suggests that binocular vision may play an important role in visual development. Better understanding of this role may hold implications for the study and treatment of developmentally acquired perceptual impairments.
深度学习技术的应用在解决机器视觉中的一些关键问题方面取得了重大进展，包括场景分割和深度估计等基本问题。在这里，我们报告了一种新的深度神经网络模型，能够从一对双目图像中同时进行场景分割和深度估计。通过操纵双目图像对的排列，向模型提供标准的左右图像对、相同的图像对或左右图像交换，我们表明性能水平取决于适当的双目图像排列。当图像交换时，分割和深度估计的性能都会受到影响。然而，对于相同的图像对，分割性能水平得以保持，尽管没有双目视差信息。重要的是，这些性能水平超过了等效的、仅用单目训练的分割模型。这些结果提供了证据表明，双目图像差异支持深度和分割信息的直接恢复，以及对单目分割信号的增强学习。这一发现表明，双目视觉可能在视觉发育中发挥重要作用。更好地理解这一作用可能对发育性获得性感知障碍的研究和治疗具有重要意义。
