IEEE Trans Image Process. 2015 Dec;24(12):5904-15. doi: 10.1109/TIP.2015.2490551. Epub 2015 Oct 14.
Slow feature analysis (SFA) extracts slowly varying signals from input data and has been used to model complex cells in the primary visual cortex (V1). It transmits information to both ventral and dorsal pathways to process appearance and motion information, respectively. However, SFA only uses slowly varying features for local feature extraction, because they represent appearance information more effectively than motion information. To better utilize temporal information, we propose temporal variance analysis (TVA) as a generalization of SFA. TVA learns a linear transformation matrix that projects multidimensional temporal data to temporal components with temporal variance. Inspired by the function of V1, we learn receptive fields by TVA and apply convolution and pooling to extract local features. Embedded in the improved dense trajectory framework, TVA for action recognition is proposed to: 1) extract appearance and motion features from gray using slow and fast filters, respectively; 2) extract additional motion features using slow filters from horizontal and vertical optical flows; and 3) separately encode extracted local features with different temporal variances and concatenate all the encoded features as final features. We evaluate the proposed TVA features on several challenging data sets and show that both slow and fast features are useful in the low-level feature extraction. Experimental results show that the proposed TVA features outperform the conventional histogram-based features, and excellent results can be achieved by combining all TVA features.
慢特征分析（Slow Feature Analysis，SFA）从输入数据中提取缓慢变化的信号，并已被用于模拟初级视觉皮层（V1）中的复杂细胞。它将信息传递到腹侧和背侧通路，分别处理外观和运动信息。然而，SFA 仅使用缓慢变化的特征进行局部特征提取，因为它们比运动信息更有效地表示外观信息。为了更好地利用时间信息，我们提出了时间方差分析（Temporal Variance Analysis，TVA）作为 SFA 的推广。TVA 学习一个线性变换矩阵，将多维时间数据投影到具有时间方差的时间分量上。受 V1 功能的启发，我们通过 TVA 学习感受野，并应用卷积和池化来提取局部特征。嵌入改进的密集轨迹框架中，提出了用于动作识别的 TVA 以：1）使用慢滤波器和快滤波器分别从灰度图像中提取外观和运动特征；2）使用慢滤波器从水平和垂直光流中提取额外的运动特征；3）分别对具有不同时间方差的提取的局部特征进行编码，并将所有编码的特征连接作为最终特征。我们在几个具有挑战性的数据集上评估了所提出的 TVA 特征，并表明慢特征和快特征在底层特征提取中都很有用。实验结果表明，所提出的 TVA 特征优于传统的基于直方图的特征，并且通过组合所有 TVA 特征可以获得出色的结果。
