Wang Peng, Wang Ke, Song Yafei, Wang Xiaodan
College of Air and Missile Defense, Air Force Engineering University, Xi'an, 710051, China.
Sci Rep. 2024 Nov 30;14(1):29801. doi: 10.1038/s41598-024-81000-1.
Time series classification finds widespread applications in civil, industrial, and military fields, while the classification performance of time series models has been improving with the recent development of deep learning. However, the issues of feature extraction effectiveness, model complexity, and model design uncertainty constrain the further development of time series classification. To address the above issues, we propose a Lightweight Spatio-Temporal Decoupling Transformer framework based on Automated Machine Learning technique (AutoLDT). The framework proposes a novel lightweight Transformer with fuzzy position encoding, TS-separable linear self-attention mechanism, and convolutional feedforward network, which mine the temporal and spatial features, as well as the local and global relationship of time series. Fuzzy positional encoding integrates fuzzy ideas to enhance the generalization performance of model information mining. TS-separable linear self-attention mechanism and convolutional feedforward network achieve feature extraction in a lightweight way by decoupling temporal and spatial features of time series. Notably, we adopt the Covariance Matrix Adaptation Evolution Strategy and global adaptive pruning technique to realize automated network structure design, which further improves the model training efficiency and automation, and avoids the uncertainty problem of network design. Finally, we validate the effectiveness of the proposed framework on publicly available UCR and UEA time series datasets. The experimental results show that the proposed framework not only improves the model classification performance in a lightweight way but also dramatically improves the model training efficiency.
时间序列分类在民用、工业和军事领域有着广泛的应用，而随着深度学习的发展，时间序列模型的分类性能也在不断提高。然而，特征提取有效性、模型复杂性和模型设计不确定性等问题制约了时间序列分类的进一步发展。为了解决上述问题，我们提出了一种基于自动机器学习技术（AutoLDT）的轻量级时空解耦Transformer框架。该框架提出了一种具有模糊位置编码、TS可分离线性自注意力机制和卷积前馈网络的新型轻量级Transformer，用于挖掘时间序列的时空特征以及局部和全局关系。模糊位置编码整合了模糊思想，以增强模型信息挖掘的泛化性能。TS可分离线性自注意力机制和卷积前馈网络通过解耦时间序列的时空特征，以轻量级方式实现特征提取。值得注意的是，我们采用协方差矩阵自适应进化策略和全局自适应剪枝技术来实现自动化网络结构设计，进一步提高了模型训练效率和自动化程度，避免了网络设计的不确定性问题。最后，我们在公开可用的UCR和UEA时间序列数据集上验证了所提出框架的有效性。实验结果表明，所提出的框架不仅以轻量级方式提高了模型分类性能，而且显著提高了模型训练效率。
