Srinivasan Keshav, Plenz Dietmar, Girvan Michelle
Biophysics Program, University of Maryland, College Park, MD 20740, USA.
Section on Critical Brain Dynamics, National Institute of Mental Health, Bethesda, MD 20892, USA.
ArXiv. 2025 Apr 16:arXiv:2504.12480v1.
Reservoir computers (RCs) provide a computationally efficient alternative to deep learning while also offering a framework for incorporating brain-inspired computational principles. By using an internal neural network with random, fixed connections-the 'reservoir'-and training only the output weights, RCs simplify the training process but remain sensitive to the choice of hyperparameters that govern activation functions and network architecture. Moreover, typical RC implementations overlook a critical aspect of neuronal dynamics: the balance between excitatory and inhibitory (E-I) signals, which is essential for robust brain function. We show that RCs characteristically perform best in balanced or slightly over-inhibited regimes, outperforming excitation-dominated ones. To reduce the need for precise hyperparameter tuning, we introduce a self-adapting mechanism that locally adjusts E/I balance to achieve target neuronal firing rates, improving performance by up to 130% in tasks like memory capacity and time series prediction compared with globally tuned RCs. Incorporating brain-inspired heterogeneity in target neuronal firing rates further reduces the need for fine-tuning hyperparameters and enables RCs to excel across linear and non-linear tasks. These results support a shift from static optimization to dynamic adaptation in reservoir design, demonstrating how brain-inspired mechanisms can improve RC performance and robustness while deepening our understanding of neural computation.
储层计算机（RCs）为深度学习提供了一种计算高效的替代方案，同时还提供了一个纳入受大脑启发的计算原理的框架。通过使用具有随机、固定连接的内部神经网络——“储层”，并且只训练输出权重，RCs简化了训练过程，但仍然对控制激活函数和网络架构的超参数选择敏感。此外，典型的RC实现忽略了神经元动力学的一个关键方面：兴奋性和抑制性（E-I）信号之间的平衡，这对强大的大脑功能至关重要。我们表明，RCs在平衡或略微过度抑制的状态下通常表现最佳，优于以兴奋为主导的状态。为了减少对精确超参数调整的需求，我们引入了一种自适应机制，该机制局部调整E/I平衡以实现目标神经元放电率，与全局调整的RCs相比，在诸如记忆容量和时间序列预测等任务中性能提高了多达130%。在目标神经元放电率中纳入受大脑启发的异质性进一步减少了对超参数微调的需求，并使RCs能够在线性和非线性任务中表现出色。这些结果支持在储层设计中从静态优化向动态适应的转变，展示了受大脑启发的机制如何提高RC的性能和鲁棒性，同时加深我们对神经计算的理解。
