Meir Yuval, Tzach Yarden, Hodassman Shiri, Tevet Ofek, Kanter Ido
Department of Physics, Bar-Ilan University, 52900, Ramat-Gan, Israel.
Gonda Interdisciplinary Brain Research Center, Bar-Ilan University, 52900, Ramat-Gan, Israel.
Sci Rep. 2024 Mar 11;14(1):5881. doi: 10.1038/s41598-024-56609-x.
Recently, the underlying mechanism for successful deep learning (DL) was presented based on a quantitative method that measures the quality of a single filter in each layer of a DL model, particularly VGG-16 trained on CIFAR-10. This method exemplifies that each filter identifies small clusters of possible output labels, with additional noise selected as labels outside the clusters. This feature is progressively sharpened with each layer, resulting in an enhanced signal-to-noise ratio (SNR), which leads to an increase in the accuracy of the DL network. In this study, this mechanism is verified for VGG-16 and EfficientNet-B0 trained on the CIFAR-100 and ImageNet datasets, and the main results are as follows. First, the accuracy and SNR progressively increase with the layers. Second, for a given deep architecture, the maximal error rate increases approximately linearly with the number of output labels. Third, similar trends were obtained for dataset labels in the range [3, 1000], thus supporting the universality of this mechanism. Understanding the performance of a single filter and its dominating features paves the way to highly dilute the deep architecture without affecting its overall accuracy, and this can be achieved by applying the filter's cluster connections (AFCC).
最近，基于一种定量方法提出了深度学习（DL）成功的潜在机制，该方法用于衡量DL模型各层中单个滤波器的质量，特别是在CIFAR-10数据集上训练的VGG-16模型。该方法表明，每个滤波器识别可能输出标签的小集群，额外的噪声被选为集群外的标签。随着层数的增加，这一特征逐渐变得更加明显，从而提高了信噪比（SNR），进而提高了DL网络的准确性。在本研究中，对在CIFAR-100和ImageNet数据集上训练的VGG-16和EfficientNet-B0验证了这一机制，主要结果如下。第一，准确率和信噪比随着层数的增加而逐渐提高。第二，对于给定的深度架构，最大错误率随输出标签数量近似线性增加。第三，在[3, 1000]范围内的数据集标签也获得了类似的趋势，从而支持了该机制的普遍性。了解单个滤波器的性能及其主导特征为在不影响整体准确性的情况下大幅简化深度架构铺平了道路，这可以通过应用滤波器的聚类连接（AFCC）来实现。
