Information System & Security and Countermeasures Experiments Center, Beijing Institute of Technology, 100081, PR China.
Information System & Security and Countermeasures Experiments Center, Beijing Institute of Technology, 100081, PR China.
Artif Intell Med. 2020 Jan;102:101764. doi: 10.1016/j.artmed.2019.101764. Epub 2019 Nov 18.
Deep Neural Network (DNN), as a deep architectures, has shown excellent performance in classification tasks. However, when the data has different distributions or contains some latent non-observed factors, it is difficult for DNN to train a single model to perform well on the classification tasks. In this paper, we propose mixture model based on DNNs (MoNNs), a supervised approach to perform classification tasks with a gating network and multiple local expert models. We use a neural network as a gating function and use DNNs as local expert models. The gating network split the heterogeneous data into several homogeneous components. DNNs are combined to perform classification tasks in each component. Moreover, we use EM (Expectation Maximization) as an optimization algorithm. Experiments proved that our MoNNs outperformed the other compared methods on determination of diabetes, determination of benign or malignant breast cancer, and handwriting recognition. Therefore, the MoNNs can solve the problem of data heterogeneity and have a good effect on classification tasks.
深度神经网络（DNN）作为一种深度架构，在分类任务中表现出了优异的性能。然而，当数据具有不同的分布或包含一些潜在的未观察到的因素时，DNN 很难训练单个模型来很好地执行分类任务。在本文中，我们提出了基于 DNN 的混合模型（MoNNs），这是一种使用门控网络和多个局部专家模型执行分类任务的有监督方法。我们使用神经网络作为门控函数，并使用 DNN 作为局部专家模型。门控网络将异构数据分为几个同质组件。在每个组件中，DNNs 被组合起来执行分类任务。此外，我们使用 EM（期望最大化）作为优化算法。实验证明，我们的 MoNNs 在糖尿病的确定、良性或恶性乳腺癌的确定以及手写识别方面优于其他比较方法。因此，MoNNs 可以解决数据异质性的问题，并对分类任务有很好的效果。
