Department of Electrical and Computer Engineering, Boston University, Boston, MA 02215, USA.
IEEE Trans Image Process. 2013 Jun;22(6):2479-94. doi: 10.1109/TIP.2013.2252622.
We propose a general framework for fast and accurate recognition of actions in video using empirical covariance matrices of features. A dense set of spatio-temporal feature vectors are computed from video to provide a localized description of the action, and subsequently aggregated in an empirical covariance matrix to compactly represent the action. Two supervised learning methods for action recognition are developed using feature covariance matrices. Common to both methods is the transformation of the classification problem in the closed convex cone of covariance matrices into an equivalent problem in the vector space of symmetric matrices via the matrix logarithm. The first method applies nearest-neighbor classification using a suitable Riemannian metric for covariance matrices. The second method approximates the logarithm of a query covariance matrix by a sparse linear combination of the logarithms of training covariance matrices. The action label is then determined from the sparse coefficients. Both methods achieve state-of-the-art classification performance on several datasets, and are robust to action variability, viewpoint changes, and low object resolution. The proposed framework is conceptually simple and has low storage and computational requirements making it attractive for real-time implementation.
我们提出了一种使用特征经验协方差矩阵快速准确识别视频中动作的通用框架。从视频中计算出密集的时空特征向量集，提供动作的局部描述，然后在经验协方差矩阵中聚合，以紧凑地表示动作。使用特征协方差矩阵开发了两种用于动作识别的监督学习方法。这两种方法的共同点是通过矩阵对数将协方差矩阵的闭凸锥中的分类问题转换为对称矩阵向量空间中的等效问题。第一种方法使用适合协方差矩阵的黎曼度量应用最近邻分类。第二种方法通过训练协方差矩阵的对数的稀疏线性组合来近似查询协方差矩阵的对数。然后从稀疏系数中确定动作标签。这两种方法在多个数据集上都实现了最先进的分类性能，并且对动作变化、视角变化和低对象分辨率具有鲁棒性。所提出的框架概念简单，存储和计算要求低，因此非常适合实时实现。
