Ueda N, Nakano R, Ghahramani Z, Hinton G E
NTT Communication Science Laboratories, Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0237 Japan.
Neural Comput. 2000 Sep;12(9):2109-28. doi: 10.1162/089976600300015088.
We present a split-and-merge expectation-maximization (SMEM) algorithm to overcome the local maxima problem in parameter estimation of finite mixture models. In the case of mixture models, local maxima often involve having too many components of a mixture model in one part of the space and too few in another, widely separated part of the space. To escape from such configurations, we repeatedly perform simultaneous split-and-merge operations using a new criterion for efficiently selecting the split-and-merge candidates. We apply the proposed algorithm to the training of gaussian mixtures and mixtures of factor analyzers using synthetic and real data and show the effectiveness of using the split-and-merge operations to improve the likelihood of both the training data and of held-out test data. We also show the practical usefulness of the proposed algorithm by applying it to image compression and pattern recognition problems.
我们提出了一种分裂合并期望最大化（SMEM）算法，以克服有限混合模型参数估计中的局部最大值问题。在混合模型的情况下，局部最大值通常表现为在空间的一部分中混合模型的组件过多，而在空间中另一处相距甚远的部分中组件过少。为了摆脱这种配置，我们使用一种有效选择分裂合并候选对象的新准则，反复执行同步的分裂合并操作。我们将所提出的算法应用于使用合成数据和真实数据训练高斯混合模型和因子分析器混合模型，并展示了使用分裂合并操作来提高训练数据和留出的测试数据的似然性的有效性。我们还通过将其应用于图像压缩和模式识别问题，展示了所提出算法的实际实用性。
