Tan Xin, Lin Jiaying, Xu Ke, Chen Pan, Ma Lizhuang, Lau Rynson W H
IEEE Trans Pattern Anal Mach Intell. 2023 Mar;45(3):3492-3504. doi: 10.1109/TPAMI.2022.3181030. Epub 2023 Feb 3.
Mirror detection is challenging because the visual appearances of mirrors change depending on those of their surroundings. As existing mirror detection methods are mainly based on extracting contextual contrast and relational similarity between mirror and non-mirror regions, they may fail to identify a mirror region if these assumptions are violated. Inspired by a recent study of applying a CNN to help distinguish whether an image is flipped or not based on the visual chirality property, in this paper, we rethink this image-level visual chirality property and reformulate it as a learnable pixel level cue for mirror detection. Specifically, we first propose a novel flipping-convolution-flipping (FCF) transformation to model visual chirality as learnable commutative residual. We then propose a novel visual chirality embedding (VCE) module to exploit this commutative residual in multi-scale feature maps, to embed the visual chirality features into our mirror detection model. Besides, we also propose a visual chirality-guided edge detection (CED) module to integrate the visual chirality features with contextual features for detection refinement. Extensive experiments show that the proposed method outperforms state-of-the-art methods on three benchmark datasets.
镜像检测具有挑战性，因为镜子的视觉外观会根据其周围环境而变化。由于现有的镜像检测方法主要基于提取镜像区域和非镜像区域之间的上下文对比度和关系相似度，如果这些假设不成立，它们可能无法识别镜像区域。受最近一项关于应用卷积神经网络（CNN）基于视觉手性属性帮助区分图像是否翻转的研究启发，在本文中，我们重新思考这种图像级别的视觉手性属性，并将其重新表述为用于镜像检测的可学习像素级线索。具体而言，我们首先提出一种新颖的翻转-卷积-翻转（FCF）变换，将视觉手性建模为可学习的交换残差。然后，我们提出一种新颖的视觉手性嵌入（VCE）模块，以利用多尺度特征图中的这种交换残差，将视觉手性特征嵌入到我们的镜像检测模型中。此外，我们还提出了一种视觉手性引导的边缘检测（CED）模块，将视觉手性特征与上下文特征集成以进行检测细化。大量实验表明，所提出的方法在三个基准数据集上优于现有方法。
