Huynh Thuan Q, Reggia James A
Department of Computer Science, Universityof Maryland, College Park, MD 20742, USA.
IEEE Trans Neural Netw. 2011 Feb;22(2):264-75. doi: 10.1109/TNN.2010.2094205. Epub 2010 Dec 6.
The production of relatively large and opaque weight matrices by error backpropagation learning has inspired substantial research on how to extract symbolic human-readable rules from trained networks. While considerable progress has been made, the results at present are still relatively limited, in part due to the large numbers of symbolic rules that can be generated. Most past work to address this issue has focused on progressively more powerful methods for rule extraction (RE) that try to minimize the number of weights and/or improve rule expressiveness. In contrast, here we take a different approach in which we modify the error backpropagation training process so that it learns a different hidden layer representation of input patterns than would normally occur. Using five publicly available datasets, we show via computational experiments that the modified learning method helps to extract fewer rules without increasing individual rule complexity and without decreasing classification accuracy. We conclude that modifying error backpropagation so that it more effectively separates learned pattern encodings in the hidden layer is an effective way to improve contemporary RE methods.
通过误差反向传播学习生成相对较大且不透明的权重矩阵，激发了关于如何从训练好的网络中提取符号化的、人类可读规则的大量研究。虽然已经取得了相当大的进展，但目前的结果仍然相对有限，部分原因是可以生成大量的符号规则。过去解决这个问题的大多数工作都集中在越来越强大的规则提取（RE）方法上，这些方法试图最小化权重数量和/或提高规则表现力。相比之下，这里我们采用一种不同的方法，即修改误差反向传播训练过程，使其学习到与正常情况不同的输入模式的隐藏层表示。使用五个公开可用的数据集，我们通过计算实验表明，修改后的学习方法有助于提取更少的规则，而不会增加单个规则的复杂性，也不会降低分类准确率。我们得出结论，修改误差反向传播，使其更有效地在隐藏层中分离学习到的模式编码，是改进当代RE方法的有效途径。
