Department of Applied Mathematics and Statistics, Center for Imaging Science and Institute for Computational Medicine, Johns Hopkins University, Clark Hall, 3400 N. Charles St., Baltimore, MD 21218, USA.
IEEE Trans Pattern Anal Mach Intell. 2013 Feb;35(2):398-410. doi: 10.1109/TPAMI.2012.96.
We present a new framework for learning high-dimensional multivariate probability distributions from estimated marginals. The approach is motivated by compositional models and Bayesian networks, and designed to adapt to small sample sizes. We start with a large, overlapping set of elementary statistical building blocks, or "primitives," which are low-dimensional marginal distributions learned from data. Each variable may appear in many primitives. Subsets of primitives are combined in a Lego-like fashion to construct a probabilistic graphical model; only a small fraction of the primitives will participate in any valid construction. Since primitives can be precomputed, parameter estimation and structure search are separated. Model complexity is controlled by strong biases; we adapt the primitives to the amount of training data and impose rules which restrict the merging of them into allowable compositions. The likelihood of the data decomposes into a sum of local gains, one for each primitive in the final structure. We focus on a specific subclass of networks which are binary forests. Structure optimization corresponds to an integer linear program and the maximizing composition can be computed for reasonably large numbers of variables. Performance is evaluated using both synthetic data and real datasets from natural language processing and computational biology.
我们提出了一种从估计的边缘分布中学习高维多元概率分布的新框架。该方法的灵感来自组合模型和贝叶斯网络，旨在适应小样本量。我们从大量重叠的基本统计构建块或“基元”开始，这些基元是从数据中学习到的低维边缘分布。每个变量都可能出现在许多基元中。基元的子集以乐高式的方式组合在一起，构成一个概率图形模型；只有一小部分基元会参与任何有效的构建。由于基元可以预先计算，因此参数估计和结构搜索是分开的。模型复杂度由强偏差控制；我们根据训练数据的数量调整基元，并施加规则限制它们合并为允许的组合。数据的似然分解为局部增益的和，每个基元在最终结构中一个增益。我们专注于一种特定的网络子类，即二进制森林。结构优化对应于整数线性规划，并且可以为相当多的变量计算最大组合。使用来自自然语言处理和计算生物学的合成数据和真实数据集来评估性能。
