Veeraraghavan Ashok, Srivastava Anuj, Roy-Chowdhury Amit K, Chellappa Rama
Centre for Automation Research and Electrical and Computer Engineering Department, University of Maryland, College Park, MD 20742, USA.
IEEE Trans Image Process. 2009 Jun;18(6):1326-39. doi: 10.1109/TIP.2009.2017143. Epub 2009 Apr 24.
Pattern recognition in video is a challenging task because of the multitude of spatio-temporal variations that occur in different videos capturing the exact same event. While traditional pattern-theoretic approaches account for the spatial changes that occur due to lighting and pose, very little has been done to address the effect of temporal rate changes in the executions of an event. In this paper, we provide a systematic model-based approach to learn the nature of such temporal variations (time warps) while simultaneously allowing for the spatial variations in the descriptors. We illustrate our approach for the problem of action recognition and provide experimental justification for the importance of accounting for rate variations in action recognition. The model is composed of a nominal activity trajectory and a function space capturing the probability distribution of activity-specific time warping transformations. We use the square-root parameterization of time warps to derive geodesics, distance measures, and probability distributions on the space of time warping functions. We then design a Bayesian algorithm which treats the execution rate function as a nuisance variable and integrates it out using Monte Carlo sampling, to generate estimates of class posteriors. This approach allows us to learn the space of time warps for each activity while simultaneously capturing other intra- and interclass variations. Next, we discuss a special case of this approach which assumes a uniform distribution on the space of time warping functions and show how computationally efficient inference algorithms may be derived for this special case. We discuss the relative advantages and disadvantages of both approaches and show their efficacy using experiments on gait-based person identification and activity recognition.
视频中的模式识别是一项具有挑战性的任务，因为在捕捉完全相同事件的不同视频中会出现大量的时空变化。虽然传统的模式理论方法考虑了由于光照和姿势引起的空间变化，但在处理事件执行过程中时间速率变化的影响方面做得很少。在本文中，我们提供了一种基于系统模型的方法来学习这种时间变化（时间扭曲）的本质，同时考虑描述符中的空间变化。我们说明了我们针对动作识别问题的方法，并为在动作识别中考虑速率变化的重要性提供了实验依据。该模型由一个标称活动轨迹和一个捕获特定活动时间扭曲变换概率分布的函数空间组成。我们使用时间扭曲的平方根参数化来推导时间扭曲函数空间上的测地线、距离度量和概率分布。然后，我们设计了一种贝叶斯算法，将执行速率函数视为一个讨厌变量，并使用蒙特卡罗采样将其积分掉，以生成类后验估计。这种方法使我们能够学习每个活动的时间扭曲空间，同时捕获其他类内和类间变化。接下来，我们讨论这种方法的一种特殊情况，即假设时间扭曲函数空间上的均匀分布，并展示如何为这种特殊情况推导计算效率高的推理算法。我们讨论了这两种方法的相对优缺点，并通过基于步态的人员识别和活动识别实验展示了它们的有效性。
