IEEE Trans Cybern. 2014 Oct;44(10):1871-83. doi: 10.1109/TCYB.2014.2298401.
This paper concerns model selection for mixtures of probabilistic principal component analyzers (MPCA). The well known Bayesian information criterion (BIC) is frequently used for this purpose. However, it is found that BIC penalizes each analyzer implausibly using the whole sample size. In this paper, we present a new criterion for MPCA called hierarchical BIC in which each analyzer is penalized using its own effective sample size only. Theoretically, hierarchical BIC is a large sample approximation of variational Bayesian lower bound and BIC is a further approximation of hierarchical BIC. To learn hierarchical-BIC-based MPCA, we propose two efficient algorithms: two-stage and one-stage variants. The two-stage algorithm integrates model selection with respect to the subspace dimensions into parameter estimation, and the one-stage variant further integrates the selection of the number of mixture components into a single algorithm. Experiments on a number of synthetic and real-world data sets show that: 1) hierarchical BIC is more accurate than BIC and several related competitors and 2) the two proposed algorithms are not only effective but also much more efficient than the classical two-stage procedure commonly used for BIC.
本文研究了概率主成分分析器（MPCA）混合物的模型选择问题。常用的贝叶斯信息准则（BIC）常用于此目的。但是，我们发现 BIC 对每个分析器的惩罚不合理，使用了整个样本大小。在本文中，我们提出了一种新的 MPCA 准则，称为层次 BIC，其中每个分析器仅使用其自己的有效样本量进行惩罚。从理论上讲，层次 BIC 是变分贝叶斯下界的大样本逼近，BIC 是层次 BIC 的进一步逼近。为了学习基于层次 BIC 的 MPCA，我们提出了两种有效的算法：两阶段和单阶段变体。两阶段算法将子空间维度的模型选择与参数估计集成在一起，而单阶段变体则将混合分量的数量选择进一步集成到单个算法中。在许多合成和真实数据集上的实验表明：1）层次 BIC 比 BIC 和几个相关的竞争对手更准确；2）所提出的两种算法不仅有效，而且比常用的基于 BIC 的经典两阶段过程效率更高。
