Anderson David F, Joshi Badal, Deshpande Abhishek
Department of Mathematics, University of Wisconsin-Madison, Madison, WI, USA.
Department of Mathematics, California State University San Marcos, San Marcos, CA, USA.
J R Soc Interface. 2021 Apr;18(177):20210031. doi: 10.1098/rsif.2021.0031. Epub 2021 Apr 14.
This paper is concerned with the utilization of deterministically modelled chemical reaction networks for the implementation of (feed-forward) neural networks. We develop a general mathematical framework and prove that the ordinary differential equations (ODEs) associated with certain reaction network implementations of neural networks have desirable properties including (i) existence of unique positive fixed points that are smooth in the parameters of the model (necessary for gradient descent) and (ii) fast convergence to the fixed point regardless of initial condition (necessary for efficient implementation). We do so by first making a connection between neural networks and fixed points for systems of ODEs, and then by constructing reaction networks with the correct associated set of ODEs. We demonstrate the theory by constructing a reaction network that implements a neural network with a smoothed ReLU activation function, though we also demonstrate how to generalize the construction to allow for other activation functions (each with the desirable properties listed previously). As there are multiple types of 'networks' used in this paper, we also give a careful introduction to both reaction networks and neural networks, in order to disambiguate the overlapping vocabulary in the two settings and to clearly highlight the role of each network's properties.
本文关注利用确定性建模的化学反应网络来实现（前馈）神经网络。我们开发了一个通用的数学框架，并证明与神经网络的某些反应网络实现相关的常微分方程（ODE）具有理想的性质，包括：（i）存在在模型参数中平滑的唯一正不动点（梯度下降所必需），以及（ii）无论初始条件如何都能快速收敛到不动点（高效实现所必需）。我们通过首先在神经网络和ODE系统的不动点之间建立联系，然后构建具有正确相关ODE集的反应网络来做到这一点。我们通过构建一个实现具有平滑ReLU激活函数的神经网络的反应网络来演示该理论，不过我们也展示了如何将该构建进行推广以允许使用其他激活函数（每个都具有前面列出的理想性质）。由于本文使用了多种类型的“网络”，我们还对反应网络和神经网络都进行了详细介绍，以便消除两种设置中重叠词汇的歧义，并清楚地突出每个网络性质的作用。
