Laboratory of Signal Processing, Tampere University of Technology, Finland.
Department of Engineering, Electrical and Computer Engineering, Aarhus University, Denmark.
Neural Netw. 2018 Sep;105:328-339. doi: 10.1016/j.neunet.2018.05.017. Epub 2018 Jun 7.
The excellent performance of deep neural networks has enabled us to solve several automatization problems, opening an era of autonomous devices. However, current deep net architectures are heavy with millions of parameters and require billions of floating point operations. Several works have been developed to compress a pre-trained deep network to reduce memory footprint and, possibly, computation. Instead of compressing a pre-trained network, in this work, we propose a generic neural network layer structure employing multilinear projection as the primary feature extractor. The proposed architecture requires several times less memory as compared to the traditional Convolutional Neural Networks (CNN), while inherits the similar design principles of a CNN. In addition, the proposed architecture is equipped with two computation schemes that enable computation reduction or scalability. Experimental results show the effectiveness of our compact projection that outperforms traditional CNN, while requiring far fewer parameters.
深度神经网络的卓越性能使我们能够解决许多自动化问题，开创了自主设备的时代。然而，目前的深度网络架构参数众多，需要数十亿次浮点运算。已经开发了几种方法来压缩预训练的深度网络，以减少内存占用并可能减少计算量。在这项工作中，我们不是压缩预训练的网络，而是提出了一种使用多元线性投影作为主要特征提取器的通用神经网络层结构。与传统卷积神经网络（CNN）相比，所提出的架构需要的内存少几倍，同时继承了 CNN 的类似设计原则。此外，所提出的架构配备了两种计算方案，可实现计算减少或可扩展性。实验结果表明，我们的紧凑投影是有效的，优于传统的 CNN，同时需要的参数要少得多。
