IEEE Trans Pattern Anal Mach Intell. 2023 Apr;45(4):4768-4781. doi: 10.1109/TPAMI.2022.3191996. Epub 2023 Mar 7.
Natural disasters, such as floods, tornadoes, or wildfires, are increasingly pervasive as the Earth undergoes global warming. It is difficult to predict when and where an incident will occur, so timely emergency response is critical to saving the lives of those endangered by destructive events. Fortunately, technology can play a role in these situations. Social media posts can be used as a low-latency data source to understand the progression and aftermath of a disaster, yet parsing this data is tedious without automated methods. Prior work has mostly focused on text-based filtering, yet image and video-based filtering remains largely unexplored. In this work, we present the Incidents1M Dataset, a large-scale multi-label dataset which contains 977,088 images, with 43 incident and 49 place categories. We provide details of the dataset construction, statistics and potential biases; introduce and train a model for incident detection; and perform image-filtering experiments on millions of images on Flickr and Twitter. We also present some applications on incident analysis to encourage and enable future work in computer vision for humanitarian aid. Code, data, and models are available at http://incidentsdataset.csail.mit.edu.
自然灾害,如洪水、龙卷风和野火,随着地球全球变暖而日益普遍。很难预测何时何地会发生事件,因此及时的应急响应对于拯救那些受到破坏性事件威胁的生命至关重要。幸运的是,技术可以在这些情况下发挥作用。社交媒体帖子可以作为一个低延迟的数据来源,以了解灾难的进展和后果,但如果没有自动化方法,解析这些数据是很繁琐的。之前的工作主要集中在基于文本的过滤上,但基于图像和视频的过滤仍然在很大程度上未被探索。在这项工作中,我们提出了 Incidents1M 数据集,这是一个大规模的多标签数据集,包含 977,088 张图像,涉及 43 种事件和 49 个地点类别。我们提供了数据集构建、统计和潜在偏差的详细信息;介绍并训练了一个用于事件检测的模型;并在 Flickr 和 Twitter 上对数百万张图像进行了图像过滤实验。我们还展示了一些关于事件分析的应用,以鼓励和促进计算机视觉在人道主义援助中的未来工作。代码、数据和模型可在 http://incidentsdataset.csail.mit.edu 上获得。