Yamaguchi Makoto
Department of Educational Psychology, Waseda University, Tokyo, Japan.
Scand J Psychol. 2006 Oct;47(5):339-44. doi: 10.1111/j.1467-9450.2006.00528.x.
Connectionist models with the backpropagation learning rule are said to exhibit catastrophic interference (or forgetting) with sequential training. Subsequent works showed that interference can be reduced by using orthogonal inputs. This study investigated, with a more rigorous assessment method, whether all orthogonal inputs lead to comparable extent of interference using three coding schemes. The results revealed large differences between the coding schemes. With larger networks, dense inputs led to severer interference compared with sparse inputs. With smaller networks, all the three schemes led to comparable extent of interference. Therefore, this study proved that not all the orthogonal inputs cause the same extent of interference, and that severity of interference depends on the interaction of the input coding scheme and the network size.
具有反向传播学习规则的联结主义模型在序列训练时据说会表现出灾难性干扰（或遗忘）。后续研究表明，使用正交输入可以减少干扰。本研究采用更严格的评估方法，调查了使用三种编码方案时，是否所有正交输入都会导致相当程度的干扰。结果显示，编码方案之间存在很大差异。对于较大的网络，密集输入比稀疏输入导致更严重的干扰。对于较小的网络，所有三种方案导致的干扰程度相当。因此，本研究证明并非所有正交输入都会导致相同程度的干扰，并且干扰的严重程度取决于输入编码方案和网络大小的相互作用。
