Wang Meizhen, Chen Mingzheng, Wang Ziran, Guo Yuxuan, Wu Yong, Zhao Wei, Liu Xuejun
Key Laboratory of Virtual Geographic Environment (Nanjing Normal University), Ministry of Education, Nanjing, 210023, China.
State Key Laboratory Cultivation Base of Geographical Environment Evolution (Jiangsu Province), Nanjing, 210023, China.
Environ Sci Ecotechnol. 2024 Jul 8;22:100450. doi: 10.1016/j.ese.2024.100450. eCollection 2024 Nov.
Rainfall data with high spatial and temporal resolutions are essential for urban hydrological modeling. Ubiquitous surveillance cameras can continuously record rainfall events through video and audio, so they have been recognized as potential rain gauges to supplement professional rainfall observation networks. Since video-based rainfall estimation methods can be affected by variable backgrounds and lighting conditions, audio-based approaches could be a supplement without suffering from these conditions. However, most audio-based approaches focus on rainfall-level classification rather than rainfall intensity estimation. Here, we introduce a dataset named Surveillance Audio Rainfall Intensity Dataset (SARID) and a deep learning model for estimating rainfall intensity. First, we created the dataset through audio of six real-world rainfall events. This dataset's audio recordings are segmented into 12,066 pieces and annotated with rainfall intensity and environmental information, such as underlying surfaces, temperature, humidity, and wind. Then, we developed a deep learning-based baseline using Mel-Frequency Cepstral Coefficients (MFCC) and Transformer architecture to estimate rainfall intensity from surveillance audio. Validated from ground truth data, our baseline achieves a root mean absolute error of 0.88 mm h and a coefficient of correlation of 0.765. Our findings demonstrate the potential of surveillance audio-based models as practical and effective tools for rainfall observation systems, initiating a new chapter in rainfall intensity estimation. It offers a novel data source for high-resolution hydrological sensing and contributes to the broader landscape of urban sensing, emergency response, and resilience.
具有高时空分辨率的降雨数据对于城市水文建模至关重要。无处不在的监控摄像头可以通过视频和音频持续记录降雨事件，因此它们已被视为补充专业降雨观测网络的潜在雨量计。由于基于视频的降雨估计方法可能会受到背景和光照条件变化的影响，基于音频的方法可以作为一种补充，不受这些条件的影响。然而，大多数基于音频的方法侧重于降雨等级分类而非降雨强度估计。在此，我们引入了一个名为监控音频降雨强度数据集（SARID）的数据集以及一个用于估计降雨强度的深度学习模型。首先，我们通过六个真实世界降雨事件的音频创建了该数据集。这个数据集的音频记录被分割成12066段，并标注了降雨强度和环境信息，如底层表面、温度、湿度和风。然后，我们使用梅尔频率倒谱系数（MFCC）和Transformer架构开发了一个基于深度学习的基线，以从监控音频中估计降雨强度。通过地面真值数据验证，我们的基线实现了0.88毫米/小时的均方根绝对误差和0.765的相关系数。我们的研究结果证明了基于监控音频的模型作为降雨观测系统实用有效工具的潜力，开启了降雨强度估计的新篇章。它为高分辨率水文传感提供了一种新颖的数据源，并为城市传感、应急响应和恢复力的更广泛领域做出了贡献。
