Center for Industrial Mathematics, University of Bremen, Bremen 28334, Germany.
Institute of Human Genetics, University of Heidelberg, Heidelberg 69120, Germany.
J Acoust Soc Am. 2024 Oct 1;156(4):2448-2466. doi: 10.1121/10.0030473.
Rodents employ a broad spectrum of ultrasonic vocalizations (USVs) for social communication. As these vocalizations offer valuable insights into affective states, social interactions, and developmental stages of animals, various deep learning approaches have aimed at automating both the quantitative (detection) and qualitative (classification) analysis of USVs. So far, no notable efforts have been made to determine the most suitable architecture. We present the first systematic evaluation of different types of neural networks for USV classification. We assessed various feedforward networks, including a custom-built, fully-connected network, a custom-built convolutional neural network, several residual neural networks, an EfficientNet, and a Vision Transformer. Our analysis concluded that convolutional networks with residual connections specifically adapted to USV data, are the most suitable architecture for analyzing USVs. Paired with a refined, entropy-based detection algorithm (achieving recall of 94.9 % and precision of 99.3 %), the best architecture (achieving 86.79 % accuracy) was integrated into a fully automated pipeline capable of analyzing extensive USV datasets with high reliability. In ongoing projects, our pipeline has proven to be a valuable tool in studying neonatal USVs. By comparing these distinct deep learning architectures side by side, we have established a solid foundation for future research.
啮齿动物使用广泛的超声波发声（USVs）进行社交交流。由于这些发声为动物的情感状态、社交互动和发育阶段提供了有价值的见解，因此各种深度学习方法旨在自动进行 USVs 的定量（检测）和定性（分类）分析。到目前为止，还没有做出确定最合适架构的显著努力。我们首次对用于 USV 分类的不同类型的神经网络进行了系统评估。我们评估了各种前馈网络，包括定制的全连接网络、定制的卷积神经网络、几种残差神经网络、EfficientNet 和 Vision Transformer。我们的分析得出的结论是，具有特别适用于 USV 数据的残差连接的卷积网络是分析 USV 的最合适架构。与经过改进的基于熵的检测算法（实现 94.9%的召回率和 99.3%的精度）相结合，最佳架构（实现 86.79%的准确率）被集成到一个全自动管道中，能够以高可靠性分析广泛的 USV 数据集。在正在进行的项目中，我们的管道已被证明是研究新生儿 USV 的有价值的工具。通过并排比较这些不同的深度学习架构，我们为未来的研究奠定了坚实的基础。
