Department of Informatics, University of Oslo, Norway
Uber AI Labs, USA
Evol Comput. 2020 Spring;28(1):115-140. doi: 10.1162/evco_a_00250. Epub 2019 Feb 15.
The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related to network structure has been shown to improve performance in some cases, especially when modular neural networks are beneficial. However, apart from objectives aiming to make networks more modular, such structural objectives have not been widely explored. We propose two new structural objectives and test their ability to guide evolving neural networks on two problems which can benefit from decomposition into subtasks. The first structural objective guides evolution to align neural networks with a user-recommended decomposition pattern. Intuitively, this should be a powerful guiding target for problems where human users can easily identify a structure. The second structural objective guides evolution towards a population with a high diversity in decomposition patterns. This results in exploration of many different ways to decompose a problem, allowing evolution to find good decompositions faster. Tests on our target problems reveal that both methods perform well on a problem with a very clear and decomposable structure. However, on a problem where the optimal decomposition is less obvious, the structural diversity objective is found to outcompete other structural objectives-and this technique can even increase performance on problems without any decomposable structure at all.
神经网络的结构和性能密切相关，通过使用进化算法，可以探索出最佳适用于给定任务的神经网络结构。在引导这种神经进化时，使用与网络结构相关的额外目标已经被证明可以在某些情况下提高性能，特别是当模块化神经网络有益时。然而，除了旨在使网络更加模块化的目标之外，这种结构目标并没有得到广泛的探索。我们提出了两个新的结构目标，并在两个可以从子任务分解中受益的问题上测试了它们引导进化神经网络的能力。第一个结构目标指导进化使神经网络与用户推荐的分解模式保持一致。直观地说，对于那些人类用户可以轻松识别结构的问题，这应该是一个强大的指导目标。第二个结构目标指导进化朝着具有高度分解模式多样性的种群发展。这导致对分解问题的许多不同方法进行探索，从而使进化能够更快地找到好的分解。在我们的目标问题上的测试表明，这两种方法在具有非常清晰和可分解结构的问题上表现良好。然而，在最优分解不太明显的问题上，结构多样性目标被发现优于其他结构目标——这种技术甚至可以提高根本没有可分解结构的问题的性能。
