Department of Psychology, University of California, Los Angeles, CA 90095;
Department of Statistics, University of California, Los Angeles, CA 90095.
Proc Natl Acad Sci U S A. 2019 Mar 5;116(10):4176-4181. doi: 10.1073/pnas.1814779116. Epub 2019 Feb 15.
By middle childhood, humans are able to learn abstract semantic relations (e.g., antonym, synonym, category membership) and use them to reason by analogy. A deep theoretical challenge is to show how such abstract relations can arise from nonrelational inputs, thereby providing key elements of a protosymbolic representation system. We have developed a computational model that exploits the potential synergy between deep learning from "big data" (to create semantic features for individual words) and supervised learning from "small data" (to create representations of semantic relations between words). Given as inputs labeled pairs of lexical representations extracted by deep learning, the model creates augmented representations by remapping features according to the rank of differences between values for the two words in each pair. These augmented representations aid in coping with the feature alignment problem (e.g., matching those features that make "love-hate" an antonym with the different features that make "rich-poor" an antonym). The model extracts weight distributions that are used to estimate the probabilities that new word pairs instantiate each relation, capturing the pattern of human typicality judgments for a broad range of abstract semantic relations. A measure of relational similarity can be derived and used to solve simple verbal analogies with human-level accuracy. Because each acquired relation has a modular representation, basic symbolic operations are enabled (notably, the converse of any learned relation can be formed without additional training). Abstract semantic relations can be induced by bootstrapping from nonrelational inputs, thereby enabling relational generalization and analogical reasoning.
到了儿童中期，人类能够学习抽象的语义关系（例如，反义词、同义词、类别成员），并利用它们进行类比推理。一个深层次的理论挑战是要展示如何从非关系性输入中产生这种抽象关系，从而为原始符号表示系统提供关键要素。我们已经开发了一种计算模型，利用深度学习从“大数据”中（为单个单词创建语义特征）和从“小数据”中（为单词之间的语义关系创建表示）的潜在协同作用。给定由深度学习提取的标记的词汇表示对作为输入，该模型通过根据每对中两个单词的值之间的差异的等级重新映射特征来创建增强表示。这些增强表示有助于解决特征对齐问题（例如，匹配使“爱-恨”成为反义词的特征与使“富-穷”成为反义词的不同特征）。该模型提取权重分布，用于估计新单词对实例化每个关系的概率，从而捕获广泛的抽象语义关系的人类典型性判断模式。可以得出关系相似性的度量标准，并用于解决具有人类水平准确性的简单口头类比。由于每个获得的关系都有一个模块化的表示，因此可以启用基本的符号操作（值得注意的是，无需额外培训即可形成任何学习关系的逆关系）。可以通过从非关系性输入进行自举来诱导抽象语义关系，从而实现关系泛化和类比推理。
