IEEE Trans Cybern. 2017 Apr;47(4):1017-1027. doi: 10.1109/TCYB.2016.2536638. Epub 2016 Mar 16.
Deep networks have achieved excellent performance in learning representation from visual data. However, the supervised deep models like convolutional neural network require large quantities of labeled data, which are very expensive to obtain. To solve this problem, this paper proposes an unsupervised deep network, called the stacked convolutional denoising auto-encoders, which can map images to hierarchical representations without any label information. The network, optimized by layer-wise training, is constructed by stacking layers of denoising auto-encoders in a convolutional way. In each layer, high dimensional feature maps are generated by convolving features of the lower layer with kernels learned by a denoising auto-encoder. The auto-encoder is trained on patches extracted from feature maps in the lower layer to learn robust feature detectors. To better train the large network, a layer-wise whitening technique is introduced into the model. Before each convolutional layer, a whitening layer is embedded to sphere the input data. By layers of mapping, raw images are transformed into high-level feature representations which would boost the performance of the subsequent support vector machine classifier. The proposed algorithm is evaluated by extensive experimentations and demonstrates superior classification performance to state-of-the-art unsupervised networks.
深度网络在从视觉数据中学习表示方面取得了优异的性能。然而，像卷积神经网络这样的监督深度学习模型需要大量的标记数据，而这些数据的获取成本非常高。为了解决这个问题，本文提出了一种无监督的深度网络，称为堆叠卷积去噪自动编码器，它可以在没有任何标签信息的情况下将图像映射到分层表示。该网络通过逐层训练进行优化，由堆叠的卷积去噪自动编码器层构成。在每一层中，通过用去噪自动编码器学习的核卷积下层的特征来生成高维特征图。自动编码器在从较低层的特征图中提取的补丁上进行训练，以学习鲁棒的特征检测器。为了更好地训练大型网络，我们在模型中引入了逐层白化技术。在每个卷积层之前，嵌入一个白化层，将输入数据规范化到单位超球体上。通过层层映射，原始图像被转换为高级特征表示，从而提高后续支持向量机分类器的性能。通过广泛的实验评估，所提出的算法显示出优于最先进的无监督网络的分类性能。
