Department of Electrical Engineering, Computer Engineering, and Informatics, Cyprus University of Technology, 33 Saripolou Str., Limassol 3036, Cyprus.
IEEE Trans Pattern Anal Mach Intell. 2013 Jun;35(6):1523-34. doi: 10.1109/TPAMI.2012.208.
Sequential data labeling is a fundamental task in machine learning applications, with speech and natural language processing, activity recognition in video sequences, and biomedical data analysis being characteristic examples, to name just a few. The conditional random field (CRF), a log-linear model representing the conditional distribution of the observation labels, is one of the most successful approaches for sequential data labeling and classification, and has lately received significant attention in machine learning as it achieves superb prediction performance in a variety of scenarios. Nevertheless, existing CRF formulations can capture only one- or few-timestep interactions and neglect higher order dependences, which are potentially useful in many real-life sequential data modeling applications. To resolve these issues, in this paper we introduce a novel CRF formulation, based on the postulation of an energy function which entails infinitely long time-dependences between the modeled data. Building blocks of our novel approach are: 1) the sequence memoizer (SM), a recently proposed nonparametric Bayesian approach for modeling label sequences with infinitely long time dependences, and 2) a mean-field-like approximation of the model marginal likelihood, which allows for the derivation of computationally efficient inference algorithms for our model. The efficacy of the so-obtained infinite-order CRF (CRF(∞)) model is experimentally demonstrated.
顺序数据标注是机器学习应用中的一项基本任务，在语音和自然语言处理、视频序列中的活动识别以及生物医学数据分析等领域都有典型的应用案例。条件随机场 (CRF) 是一种表示观测标签条件分布的对数线性模型，是顺序数据标注和分类的最成功方法之一，最近在机器学习中受到了广泛关注，因为它在各种场景下都能实现出色的预测性能。然而，现有的 CRF 公式只能捕捉一到几个时间步的交互作用，而忽略了更高阶的依赖关系，而这些依赖关系在许多实际的顺序数据建模应用中可能是有用的。为了解决这些问题，在本文中，我们提出了一种新的 CRF 公式，基于一个能量函数的假设，该函数涉及到建模数据之间的无限长时间依赖关系。我们新方法的构建块包括：1) 序列记忆器 (SM)，这是一种最近提出的用于对具有无限长时间依赖关系的标签序列进行建模的非参数贝叶斯方法，以及 2) 模型边际似然的均值场似然近似，它允许为我们的模型导出计算效率高的推断算法。实验证明了所得到的无限阶 CRF (CRF(∞)) 模型的有效性。
