Department of Bioengineering, Imperial College London, London, United Kingdom.
J Neurophysiol. 2024 Oct 1;132(4):1156-1169. doi: 10.1152/jn.00421.2023. Epub 2024 Aug 28.
Thousands of species use vocal signals to communicate with one another. Vocalizations carry rich information, yet characterizing and analyzing these complex, high-dimensional signals is difficult and prone to human bias. Moreover, animal vocalizations are ethologically relevant stimuli whose representation by auditory neurons is an important subject of research in sensory neuroscience. A method that can efficiently generate naturalistic vocalization waveforms would offer an unlimited supply of stimuli with which to probe neuronal computations. Although unsupervised learning methods allow for the projection of vocalizations into low-dimensional latent spaces learned from the waveforms themselves, and generative modeling allows for the synthesis of novel vocalizations for use in downstream tasks, we are not aware of any model that combines these tasks to synthesize naturalistic vocalizations in the waveform domain for stimulus playback. In this paper, we demonstrate BiWaveGAN: a bidirectional generative adversarial network (GAN) capable of learning a latent representation of ultrasonic vocalizations (USVs) from mice. We show that BiWaveGAN can be used to generate, and interpolate between, realistic vocalization waveforms. We then use these synthesized stimuli along with natural USVs to probe the sensory input space of mouse auditory cortical neurons. We show that stimuli generated from our method evoke neuronal responses as effectively as real vocalizations, and produce receptive fields with the same predictive power. BiWaveGAN is not restricted to mouse USVs but can be used to synthesize naturalistic vocalizations of any animal species and interpolate between vocalizations of the same or different species, which could be useful for probing categorical boundaries in representations of ethologically relevant auditory signals. A new type of artificial neural network is presented that can be used to generate animal vocalization waveforms and interpolate between them to create new vocalizations. We find that our synthetic naturalistic stimuli drive auditory cortical neurons in the mouse equally well and produce receptive field features with the same predictive power as those obtained with natural mouse vocalizations, confirming the quality of the stimuli produced by the neural network.
数千种物种使用声音信号相互交流。声音信号携带着丰富的信息，但对这些复杂的高维信号进行特征描述和分析是困难的，并且容易受到人为偏见的影响。此外，动物的声音信号是具有生态意义的刺激物，听觉神经元对其的表示是感觉神经科学研究的一个重要课题。一种能够有效地生成自然声音波形的方法将提供无限数量的刺激物，用于探测神经元计算。尽管无监督学习方法允许将声音信号投影到从波形本身学习到的低维潜在空间中，生成模型允许合成用于下游任务的新声音信号，但我们不知道有任何模型可以将这些任务结合起来，以在波形域中合成自然声音信号用于刺激回放。在本文中，我们展示了 BiWaveGAN：一种能够从老鼠的超声波声音（USV）中学习潜在表示的双向生成对抗网络（GAN）。我们表明，BiWaveGAN 可用于生成和在真实声音信号之间进行插值。然后，我们使用这些合成的刺激物以及自然 USV 来探测老鼠听觉皮层神经元的感觉输入空间。我们表明，我们的方法生成的刺激物可以像真实声音一样有效地引起神经元反应，并产生具有相同预测能力的感受野。BiWaveGAN 不仅限于老鼠 USV，还可以用于合成任何动物物种的自然声音信号，并在同一物种或不同物种的声音信号之间进行插值，这对于探测与生态相关的听觉信号的表示中的类别边界可能很有用。提出了一种新的人工神经网络类型，可用于生成动物声音信号并在它们之间进行插值以创建新的声音。我们发现，我们的合成自然刺激物同样能够驱动老鼠的听觉皮层神经元，并产生与用自然老鼠声音获得的感受野特征具有相同预测能力的特征，从而确认了神经网络产生的刺激物的质量。
