Ghosh Ranadhir, Yearwood John, Ghosh Moumita, Bagirov Adil
School of Information Technology and Mathematical Sciences, University of Ballarat, P.O. Box 663, Victoria, Australia.
Int J Neural Syst. 2006 Jun;16(3):201-13. doi: 10.1142/S0129065706000615.
In this paper we investigate a hybrid model based on the Discrete Gradient method and an evolutionary strategy for determining the weights in a feed forward artificial neural network. Also we discuss different variants for hybrid models using the Discrete Gradient method and an evolutionary strategy for determining the weights in a feed forward artificial neural network. The Discrete Gradient method has the advantage of being able to jump over many local minima and find very deep local minima. However, earlier research has shown that a good starting point for the discrete gradient method can improve the quality of the solution point. Evolutionary algorithms are best suited for global optimisation problems. Nevertheless they are cursed with longer training times and often unsuitable for real world application. For optimisation problems such as weight optimisation for ANNs in real world applications the dimensions are large and time complexity is critical. Hence the idea of a hybrid model can be a suitable option. In this paper we propose different fusion strategies for hybrid models combining the evolutionary strategy with the discrete gradient method to obtain an optimal solution much quicker. Three different fusion strategies are discussed: a linear hybrid model, an iterative hybrid model and a restricted local search hybrid model. Comparative results on a range of standard datasets are provided for different fusion hybrid models.
在本文中，我们研究了一种基于离散梯度法和进化策略的混合模型，用于确定前馈人工神经网络中的权重。此外，我们还讨论了使用离散梯度法和进化策略来确定前馈人工神经网络权重的混合模型的不同变体。离散梯度法的优点是能够跳过许多局部最小值并找到非常深的局部最小值。然而，早期研究表明，离散梯度法的一个好的起始点可以提高解点的质量。进化算法最适合全局优化问题。然而，它们存在训练时间长的问题，并且通常不适用于实际应用。对于诸如实际应用中人工神经网络的权重优化等优化问题，维度很大且时间复杂度至关重要。因此，混合模型的想法可能是一个合适的选择。在本文中，我们提出了将进化策略与离散梯度法相结合的混合模型的不同融合策略，以便更快地获得最优解。讨论了三种不同的融合策略：线性混合模型、迭代混合模型和受限局部搜索混合模型。针对不同的融合混合模型，提供了一系列标准数据集上的比较结果。
