Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA.
Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA.
Neural Netw. 2021 Apr;136:112-125. doi: 10.1016/j.neunet.2020.12.028. Epub 2021 Jan 7.
Data sparsity is a common issue to train machine learning tools such as neural networks for engineering and scientific applications, where experiments and simulations are expensive. Recently physics-constrained neural networks (PCNNs) were developed to reduce the required amount of training data. However, the weights of different losses from data and physical constraints are adjusted empirically in PCNNs. In this paper, a new physics-constrained neural network with the minimax architecture (PCNN-MM) is proposed so that the weights of different losses can be adjusted systematically. The training of the PCNN-MM is searching the high-order saddle points of the objective function. A novel saddle point search algorithm called Dual-Dimer method is developed. It is demonstrated that the Dual-Dimer method is computationally more efficient than the gradient descent ascent method for nonconvex-nonconcave functions and provides additional eigenvalue information to verify search results. A heat transfer example also shows that the convergence of PCNN-MMs is faster than that of traditional PCNNs.
数据稀疏性是训练机器学习工具（如神经网络）的一个常见问题，这些工具用于工程和科学应用，其中实验和模拟的成本很高。最近，开发了物理约束神经网络（PCNN）来减少所需的训练数据量。然而，PCNN 中不同数据和物理约束损失的权重是通过经验调整的。在本文中，提出了一种具有最小最大架构的新的物理约束神经网络（PCNN-MM），以便可以系统地调整不同损失的权重。PCNN-MM 的训练是在目标函数的高阶鞍点处进行搜索。开发了一种称为双二聚体方法的新鞍点搜索算法。结果表明，对于非凸非凹函数，双二聚体方法在计算上比梯度上升方法更有效，并提供附加的特征值信息来验证搜索结果。一个传热示例还表明，PCNN-MMs 的收敛速度比传统 PCNN 更快。
