Parihar Abhinav, Jerry Matthew, Datta Suman, Raychowdhury Arijit
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, United States.
Department of Electrical Engineering, University of Notre Dame, Notre Dame, IN, United States.
Front Neurosci. 2018 Apr 4;12:210. doi: 10.3389/fnins.2018.00210. eCollection 2018.
Artificial neural networks can harness stochasticity in multiple ways to enable a vast class of computationally powerful models. Boltzmann machines and other stochastic neural networks have been shown to outperform their deterministic counterparts by allowing dynamical systems to escape local energy minima. Electronic implementation of such stochastic networks is currently limited to addition of algorithmic noise to digital machines which is inherently inefficient; albeit recent efforts to harness physical noise in devices for stochasticity have shown promise. To succeed in fabricating electronic neuromorphic networks we need experimental evidence of devices with measurable and controllable stochasticity which is complemented with the development of reliable statistical models of such observed stochasticity. Current research literature has sparse evidence of the former and a complete lack of the latter. This motivates the current article where we demonstrate a stochastic neuron using an insulator-metal-transition (IMT) device, based on electrically induced phase-transition, in series with a tunable resistance. We show that an IMT neuron has dynamics similar to a piecewise linear FitzHugh-Nagumo (FHN) neuron and incorporates all characteristics of a spiking neuron in the device phenomena. We experimentally demonstrate spontaneous stochastic spiking along with electrically controllable firing probabilities using Vanadium Dioxide (VO) based IMT neurons which show a sigmoid-like transfer function. The stochastic spiking is explained by two noise sources - thermal noise and threshold fluctuations, which act as precursors of bifurcation. As such, the IMT neuron is modeled as an Ornstein-Uhlenbeck (OU) process with a fluctuating boundary resulting in transfer curves that closely match experiments. The moments of interspike intervals are calculated analytically by extending the first-passage-time (FPT) models for Ornstein-Uhlenbeck (OU) process to include a fluctuating boundary. We find that the coefficient of variation of interspike intervals depend on the relative proportion of thermal and threshold noise, where threshold noise is the dominant source in the current experimental demonstrations. As one of the first comprehensive studies of a stochastic neuron hardware and its statistical properties, this article would enable efficient implementation of a large class of neuro-mimetic networks and algorithms.
人工神经网络可以通过多种方式利用随机性，以实现大量具有强大计算能力的模型。玻尔兹曼机和其他随机神经网络已被证明，通过允许动态系统逃离局部能量最小值，其性能优于确定性对应物。目前，此类随机网络的电子实现仅限于向数字机器添加算法噪声，这本质上效率低下；尽管最近在利用器件中的物理噪声实现随机性方面所做的努力已显示出前景。为了成功制造电子神经形态网络，我们需要具有可测量和可控随机性的器件的实验证据，并辅之以针对此类观察到的随机性的可靠统计模型的开发。当前的研究文献对此类器件的证据稀少，而对于后者则完全缺乏。这促使我们撰写本文，在此我们展示了一种基于电诱导相变的绝缘体 - 金属转变（IMT）器件与可调电阻串联构成的随机神经元。我们表明，一个IMT神经元具有与分段线性菲茨休 - 纳古莫（FHN）神经元相似的动力学特性，并且在器件现象中包含了脉冲神经元的所有特征。我们使用基于二氧化钒（VO）的IMT神经元通过实验证明了自发随机脉冲发放以及电可控的发放概率，该神经元呈现出类似S形的传递函数。随机脉冲发放由两个噪声源解释——热噪声和阈值波动，它们充当分岔的先兆。因此，IMT神经元被建模为具有波动边界的奥恩斯坦-乌伦贝克（OU）过程，从而得到与实验紧密匹配的传递曲线。通过将奥恩斯坦 - 乌伦贝克（OU）过程的首通时间（FPT）模型扩展到包含波动边界，解析计算了脉冲间隔的矩。我们发现脉冲间隔的变异系数取决于热噪声和阈值噪声 的相对比例，其中阈值噪声是当前实验演示中的主要噪声源。作为对随机神经元硬件及其统计特性的首批全面研究之一，本文将有助于高效实现大量神经拟态网络和算法。
