Xue Ye, Lau Vincent K N
IEEE Trans Neural Netw Learn Syst. 2023 Sep;34(9):5774-5788. doi: 10.1109/TNNLS.2021.3131181. Epub 2023 Sep 1.
Dictionary learning is a widely used unsupervised learning method in signal processing and machine learning. Most existing works on dictionary learning adopt an off-line approach, and there are two main off-line ways of conducting it. One is to alternately optimize both the dictionary and the sparse code, while the other is to optimize the dictionary by restricting it over the orthogonal group. The latter, called orthogonal dictionary learning (ODL), has a lower implementation complexity and, hence, is more favorable for low-cost devices. However, existing schemes for ODL only work with batch data and cannot be implemented online, making them inapplicable for real-time applications. This article, thus, proposes a novel online orthogonal dictionary scheme to dynamically learn the dictionary from streaming data, without storing the historical data. The proposed scheme includes a novel problem formulation and an efficient online algorithm design with convergence analysis. In the problem formulation, we relax the orthogonal constraint to enable an efficient online algorithm. We then propose the design of a new Frank-Wolfe-based online algorithm with a convergence rate of O(lnt/t) . The convergence rate in terms of key system parameters is also derived. Experiments with synthetic data and real-world Internet of things (IoT) sensor readings demonstrate the effectiveness and efficiency of the proposed online ODL scheme.
字典学习是信号处理和机器学习中一种广泛使用的无监督学习方法。现有的大多数关于字典学习的工作都采用离线方法，主要有两种离线实现方式。一种是交替优化字典和稀疏编码，另一种是通过在正交群上对字典进行约束来优化字典。后者称为正交字典学习（ODL），其实现复杂度较低，因此更适合低成本设备。然而，现有的ODL方案仅适用于批数据，无法在线实现，这使得它们不适用于实时应用。因此，本文提出了一种新颖的在线正交字典方案，用于从流数据中动态学习字典，而无需存储历史数据。所提出的方案包括一个新颖的问题表述和一个具有收敛性分析的高效在线算法设计。在问题表述中，我们放宽了正交约束以实现高效的在线算法。然后，我们提出了一种基于Frank-Wolfe的新在线算法设计，其收敛速度为O(lnt/t)。还推导了关键系统参数方面的收敛速度。对合成数据和实际物联网（IoT）传感器读数进行的实验证明了所提出的在线ODL方案的有效性和效率。
