Manzhos Sergei, Ihara Manabu
School of Materials and Chemical Technology, Tokyo Institute of Technology, Ookayama 2-12-1, Meguro-ku, Tokyo 152-8552, Japan.
J Phys Chem A. 2023 Sep 21;127(37):7823-7835. doi: 10.1021/acs.jpca.3c02949. Epub 2023 Sep 12.
Feed-forward neural networks (NNs) are a staple machine learning method widely used in many areas of science and technology, including physical chemistry, computational chemistry, and materials informatics. While even a single-hidden-layer NN is a universal approximator, its expressive power is limited by the use of simple neuron activation functions (such as sigmoid functions) that are typically the same for all neurons. More flexible neuron activation functions would allow the use of fewer neurons and layers and thereby save computational cost and improve expressive power. We show that additive Gaussian process regression (GPR) can be used to construct optimal neuron activation functions that are individual to each neuron. An approach is also introduced that avoids nonlinear fitting of neural network parameters by defining them with rules. The resulting method combines the advantage of robustness of a linear regression with the higher expressive power of an NN. We demonstrate the approach by fitting the potential energy surfaces of the water molecule and formaldehyde. Without requiring any nonlinear optimization, the additive-GPR-based approach outperforms a conventional NN in the high-accuracy regime, where a conventional NN suffers more from overfitting.
前馈神经网络（NNs）是一种常用的机器学习方法，广泛应用于许多科学技术领域，包括物理化学、计算化学和材料信息学。虽然即使是单隐藏层神经网络也是通用逼近器，但其表达能力受到简单神经元激活函数（如 sigmoid 函数）使用的限制，这些函数通常对所有神经元都是相同的。更灵活的神经元激活函数将允许使用更少的神经元和层，从而节省计算成本并提高表达能力。我们表明，加性高斯过程回归（GPR）可用于构建每个神经元特有的最优神经元激活函数。还介绍了一种通过规则定义神经网络参数来避免对其进行非线性拟合的方法。由此产生的方法结合了线性回归的稳健性优势和神经网络的更高表达能力。我们通过拟合水分子和甲醛的势能面来演示该方法。基于加性 GPR 的方法在高精度区域优于传统神经网络，在该区域传统神经网络更容易受到过拟合的影响，且无需任何非线性优化。
