Galke Lukas, Ram Yoav, Raviv Limor
Department of Mathematics and Computer Science, University of Southern Denmark, Odense, Denmark.
LEADS group, Max Planck Institute for Psycholinguistics, Nijmegen, Netherlands.
Nat Commun. 2024 Dec 30;15(1):10816. doi: 10.1038/s41467-024-55158-1.
Deep neural networks drive the success of natural language processing. A fundamental property of language is its compositional structure, allowing humans to systematically produce forms for new meanings. For humans, languages with more compositional and transparent structures are typically easier to learn than those with opaque and irregular structures. However, this learnability advantage has not yet been shown for deep neural networks, limiting their use as models for human language learning. Here, we directly test how neural networks compare to humans in learning and generalizing different languages that vary in their degree of compositional structure. We evaluate the memorization and generalization capabilities of a large language model and recurrent neural networks, and show that both deep neural networks exhibit a learnability advantage for more structured linguistic input: neural networks exposed to more compositional languages show more systematic generalization, greater agreement between different agents, and greater similarity to human learners.
深度神经网络推动了自然语言处理的成功。语言的一个基本特性是其组合结构，这使人类能够系统地为新含义生成形式。对于人类来说，具有更多组合性和透明结构的语言通常比具有不透明和不规则结构的语言更容易学习。然而，深度神经网络尚未展现出这种可学习性优势，这限制了它们作为人类语言学习模型的应用。在此，我们直接测试神经网络在学习和泛化不同组合结构程度的语言方面与人类相比情况如何。我们评估了一个大语言模型和循环神经网络的记忆与泛化能力，并表明这两种深度神经网络在处理结构更丰富的语言输入时都展现出可学习性优势：接触更多组合性语言的神经网络表现出更系统的泛化能力、不同智能体之间更高的一致性以及与人类学习者更高的相似性。
