Tan Xin, Xu Ke, Cao Ying, Zhang Yiheng, Ma Lizhuang, Lau Rynson W H
IEEE Trans Image Process. 2021;30:9085-9098. doi: 10.1109/TIP.2021.3122004. Epub 2021 Nov 3.
Although huge progress has been made on scene analysis in recent years, most existing works assume the input images to be in day-time with good lighting conditions. In this work, we aim to address the night-time scene parsing (NTSP) problem, which has two main challenges: 1) labeled night-time data are scarce, and 2) over- and under-exposures may co-occur in the input night-time images and are not explicitly modeled in existing pipelines. To tackle the scarcity of night-time data, we collect a novel labeled dataset, named NightCity, of 4,297 real night-time images with ground truth pixel-level semantic annotations. To our knowledge, NightCity is the largest dataset for NTSP. In addition, we also propose an exposure-aware framework to address the NTSP problem through augmenting the segmentation process with explicitly learned exposure features. Extensive experiments show that training on NightCity can significantly improve NTSP performances and that our exposure-aware model outperforms the state-of-the-art methods, yielding top performances on our dataset as well as existing datasets.
尽管近年来场景分析取得了巨大进展，但大多数现有工作都假设输入图像处于白天且光照条件良好。在这项工作中，我们旨在解决夜间场景解析（NTSP）问题，该问题有两个主要挑战：1）带标注的夜间数据稀缺，2）输入的夜间图像中可能同时出现过曝光和曝光不足的情况，而现有流程中并未对其进行明确建模。为了解决夜间数据稀缺的问题，我们收集了一个名为NightCity的全新带标注数据集，其中包含4297张真实夜间图像以及像素级语义标注的地面真值。据我们所知，NightCity是用于NTSP的最大数据集。此外，我们还提出了一个曝光感知框架来解决NTSP问题，即通过利用明确学习到的曝光特征增强分割过程。大量实验表明，在NightCity上进行训练可以显著提高NTSP性能，并且我们的曝光感知模型优于现有最先进的方法，在我们的数据集以及现有数据集上均取得了最佳性能。
