Han Xinyu, Zhao Yi, Small Michael
Harbin Institute of Technology, Shenzhen, 518055 Guangdong, China.
Complex Systems Group, Department of Mathematics and Statistics, The University of Western Australia, Crawley, Western Australia 6009, Australia.
Chaos. 2022 Apr;32(4):043115. doi: 10.1063/5.0082258.
While reservoir computing (RC) has demonstrated astonishing performance in many practical scenarios, the understanding of its capability for generalization on previously unseen data is limited. To address this issue, we propose a novel generalization bound for RC based on the empirical Rademacher complexity under the probably approximately correct learning framework. Note that the generalization bound for the RC is derived in terms of the model hyperparameters. For this reason, it can explore the dependencies of the generalization bound for RC on its hyperparameters. Compared with the existing generalization bound, our generalization bound for RC is tighter, which is verified by numerical experiments. Furthermore, we study the generalization bound for the RC corresponding to different reservoir graphs, including directed acyclic graph (DAG) and Erdős-R e´nyi undirected random graph (ER graph). Specifically, the generalization bound for the RC whose reservoir graph is designated as a DAG can be refined by leveraging the structural property (i.e., the longest path length) of the DAG. Finally, both theoretical and experimental findings confirm that the generalization bound for the RC of a DAG is lower and less sensitive to the model hyperparameters than that for the RC of an ER graph.
虽然储层计算（RC）在许多实际场景中已展现出惊人的性能，但其对未见数据的泛化能力的理解仍很有限。为解决这一问题，我们基于可能近似正确学习框架下的经验拉德马赫复杂度，为储层计算提出了一种新颖的泛化界。请注意，储层计算的泛化界是根据模型超参数得出的。因此，它可以探究储层计算的泛化界对其超参数的依赖性。与现有的泛化界相比，我们提出的储层计算泛化界更紧，这一点通过数值实验得到了验证。此外，我们研究了与不同储层图相对应的储层计算的泛化界，包括有向无环图（DAG）和厄多斯 - 雷尼无向随机图（ER图）。具体而言，对于储层图被指定为DAG的储层计算，其泛化界可以通过利用DAG的结构特性（即最长路径长度）来细化。最后，理论和实验结果均证实，DAG的储层计算的泛化界比ER图的储层计算的泛化界更低，且对模型超参数的敏感度更低。
