Chen Ziheng, Song Yue, Xu Tianyang, Huang Zhiwu, Wu Xiao-Jun, Sebe Nicu
IEEE Trans Image Process. 2024;33:5194-5205. doi: 10.1109/TIP.2024.3451930. Epub 2024 Sep 19.
Symmetric Positive Definite (SPD) matrices have received wide attention in machine learning due to their intrinsic capacity to encode underlying structural correlation in data. Many successful Riemannian metrics have been proposed to reflect the non-Euclidean geometry of SPD manifolds. However, most existing metric tensors are fixed, which might lead to sub-optimal performance for SPD matrix learning, especially for deep SPD neural networks. To remedy this limitation, we leverage the commonly encountered pullback techniques and propose Adaptive Log-Euclidean Metrics (ALEMs), which extend the widely used Log-Euclidean Metric (LEM). Compared with the previous Riemannian metrics, our metrics contain learnable parameters, which can better adapt to the complex dynamics of Riemannian neural networks with minor extra computations. We also present a complete theoretical analysis to support our ALEMs, including algebraic and Riemannian properties. The experimental and theoretical results demonstrate the merit of the proposed metrics in improving the performance of SPD neural networks. The efficacy of our metrics is further showcased on a set of recently developed Riemannian building blocks, including Riemannian batch normalization, Riemannian Residual blocks, and Riemannian classifiers.
对称正定（SPD）矩阵因其在数据中编码潜在结构相关性的内在能力而在机器学习中受到广泛关注。人们已经提出了许多成功的黎曼度量来反映SPD流形的非欧几里得几何。然而，大多数现有的度量张量是固定的，这可能导致SPD矩阵学习的性能次优，特别是对于深度SPD神经网络。为了弥补这一局限性，我们利用常见的拉回技术并提出了自适应对数欧几里得度量（ALEM），它扩展了广泛使用的对数欧几里得度量（LEM）。与先前的黎曼度量相比，我们的度量包含可学习参数，这可以在进行少量额外计算的情况下更好地适应黎曼神经网络的复杂动态。我们还给出了完整的理论分析来支持我们的ALEM，包括代数性质和黎曼性质。实验和理论结果证明了所提出的度量在提高SPD神经网络性能方面的优点。我们的度量的有效性在一组最近开发的黎曼构建块上得到了进一步展示，包括黎曼批量归一化、黎曼残差块和黎曼分类器。
