Sun Xiaochuan, Hao Mingxiang, Wang Yutong, Wang Yu, Li Zhigang, Li Yingqi
College of Artificial Intelligence, North China University of Science and Technology, Bohai Road, Tangshan 063210, China.
Hebei Key Laboratory of Industrial Perception, Tangshan 063210, China.
Entropy (Basel). 2022 Nov 23;24(12):1709. doi: 10.3390/e24121709.
An echo state network (ESN) is an efficient recurrent neural network (RNN) that is widely used in time series prediction tasks due to its simplicity and low training cost. However, the "black-box" nature of reservoirs hinders the development of ESN. Although a large number of studies have concentrated on reservoir interpretability, the perspective of reservoir modeling is relatively single, and the relationship between reservoir richness and reservoir projection capacity has not been effectively established. To tackle this problem, a novel reservoir interpretability framework based on permutation entropy (PE) theory is proposed in this paper. In structure, this framework consists of reservoir state extraction, PE modeling, and PE analysis. Based on these, the instantaneous reservoir states and neuronal time-varying states are extracted, which are followed by phase space reconstruction, sorting, and entropy calculation. Firstly, the obtained instantaneous state entropy (ISE) and global state entropy (GSE) can measure reservoir richness for interpreting good reservoir projection capacity. On the other hand, the multiscale complexity-entropy analysis of global and neuron-level reservoir states is performed to reveal more detailed dynamics. Finally, the relationships between ESN performance and reservoir dynamic are investigated via Pearson correlation, considering different prediction steps and time scales. Experimental evaluations on several benchmarks and real-world datasets demonstrate the effectiveness and superiority of the proposed reservoir interpretability framework.
回声状态网络（ESN）是一种高效的递归神经网络（RNN），因其简单性和低训练成本而被广泛应用于时间序列预测任务。然而，储层的“黑箱”性质阻碍了ESN的发展。尽管大量研究集中在储层可解释性上，但储层建模的视角相对单一，且储层丰富度与储层投影能力之间的关系尚未有效建立。为解决这一问题，本文提出了一种基于排列熵（PE）理论的新型储层可解释性框架。在结构上，该框架由储层状态提取、PE建模和PE分析组成。基于此，提取瞬时储层状态和神经元时变状态，随后进行相空间重构、排序和熵计算。首先，所获得的瞬时状态熵（ISE）和全局状态熵（GSE）可以衡量储层丰富度，以解释良好的储层投影能力。另一方面，对全局和神经元级储层状态进行多尺度复杂性-熵分析，以揭示更详细的动态。最后，通过皮尔逊相关性研究ESN性能与储层动态之间的关系，考虑不同的预测步长和时间尺度。在几个基准数据集和真实世界数据集上的实验评估证明了所提出的储层可解释性框架的有效性和优越性。
