Department of Mathematics & Statistics, University of Massachusetts, Amherst, MA 01003, USA.
Neural Netw. 2024 May;173:106152. doi: 10.1016/j.neunet.2024.106152. Epub 2024 Feb 1.
We introduce the Discrete-Temporal Sobolev Network (DTSN), a neural network loss function that assists dynamical system forecasting by minimizing variational differences between the network output and the training data via a temporal Sobolev norm. This approach is entirely data-driven, architecture agnostic, and does not require derivative information from the estimated system. The DTSN is particularly well suited to chaotic dynamical systems as it minimizes noise in the network output which is crucial for such sensitive systems. For our test cases we consider discrete approximations of the Lorenz-63 system and the Chua circuit. For the network architectures we use the Long Short-Term Memory (LSTM) and the Transformer. The performance of the DTSN is compared with the standard MSE loss for both architectures, as well as with the Physics Informed Neural Network (PINN) loss for the LSTM. The DTSN loss is shown to substantially improve accuracy for both architectures, while requiring less information than the PINN and without noticeably increasing computational time, thereby demonstrating its potential to improve neural network forecasting of dynamical systems.
我们引入了离散时间 Sobolev 网络（DTSN），这是一种神经网络损失函数，通过时间 Sobolev 范数最小化网络输出与训练数据之间的变分差异，从而辅助动力系统预测。这种方法完全是数据驱动的，与架构无关，并且不需要从估计系统中获取导数信息。DTSN 特别适合于混沌动力系统，因为它可以最小化网络输出中的噪声，这对于此类敏感系统至关重要。对于我们的测试案例，我们考虑了 Lorenz-63 系统和 Chua 电路的离散近似。对于网络架构，我们使用了长短期记忆（LSTM）和转换器。将 DTSN 损失与两种架构的标准均方误差损失进行了比较，还与 LSTM 的物理信息神经网络（PINN）损失进行了比较。结果表明，DTSN 损失显著提高了两种架构的准确性，同时所需信息量比 PINN 少，并且计算时间没有明显增加，从而证明了其在提高动力系统神经网络预测方面的潜力。
