IEEE Trans Image Process. 2018 Jul;27(7):3264-3274. doi: 10.1109/TIP.2018.2817047.
Traditional saliency models usually adopt hand-crafted image features and human-designed mechanisms to calculate local or global contrast. In this paper, we propose a novel computational saliency model, i.e., deep spatial contextual long-term recurrent convolutional network (DSCLRCN), to predict where people look in natural scenes. DSCLRCN first automatically learns saliency related local features on each image location in parallel. Then, in contrast with most other deep network based saliency models which infer saliency in local contexts, DSCLRCN can mimic the cortical lateral inhibition mechanisms in human visual system to incorporate global contexts to assess the saliency of each image location by leveraging the deep spatial long short-term memory (DSLSTM) model. Moreover, we also integrate scene context modulation in DSLSTM for saliency inference, leading to a novel deep spatial contextual LSTM (DSCLSTM) model. The whole network can be trained end-to-end and works efficiently when testing. Experimental results on two benchmark datasets show that DSCLRCN can achieve state-of-the-art performance on saliency detection. Furthermore, the proposed DSCLSTM model can significantly boost the saliency detection performance by incorporating both global spatial interconnections and scene context modulation, which may uncover novel inspirations for studies on them in computational saliency models.
传统的显著度模型通常采用手工制作的图像特征和人为设计的机制来计算局部或全局对比度。在本文中，我们提出了一种新颖的计算显著度模型，即深度空间上下文长期递归卷积网络（DSCLRCN），用于预测人们在自然场景中看哪里。DSCLRCN 首先在每个图像位置上自动并行学习与显著度相关的局部特征。然后，与大多数其他基于深度网络的显著度模型不同，DSCLRCN 可以模拟人类视觉系统中的皮层侧抑制机制，通过利用深度空间长短期记忆（DSLSTM）模型来整合全局上下文，以评估每个图像位置的显著度。此外，我们还在 DSLSTM 中集成了场景上下文调制，用于显著度推断，从而得到一种新颖的深度空间上下文 LSTM（DSCLSTM）模型。整个网络可以端到端训练，在测试时效率很高。在两个基准数据集上的实验结果表明，DSCLRCN 可以在显著度检测方面达到最先进的性能。此外，所提出的 DSCLSTM 模型通过整合全局空间相互连接和场景上下文调制，可以显著提高显著度检测性能，这可能为计算显著度模型中对它们的研究提供新的启示。
