Zheng Yufeng, Huang Jun, Chen Tianwen, Ou Yang, Zhou Wu
Department of Data Science, University of Mississippi Medical Centre, Jackson, MS, United States.
Department of Otolaryngology-Head and Neck Surgery, University of Mississippi Medical Centre, Jackson, MS, United States.
Front Comput Neurosci. 2021 Feb 19;15:637144. doi: 10.3389/fncom.2021.637144. eCollection 2021.
The convolutional neural networks (CNNs) are a powerful tool of image classification that has been widely adopted in applications of automated scene segmentation and identification. However, the mechanisms underlying CNN image classification remain to be elucidated. In this study, we developed a new approach to address this issue by investigating transfer of learning in representative CNNs (AlexNet, VGG, ResNet-101, and Inception-ResNet-v2) on classifying geometric shapes based on local/global features or invariants. While the local features are based on simple components, such as orientation of line segment or whether two lines are parallel, the global features are based on the whole object such as whether an object has a hole or whether an object is inside of another object. Six experiments were conducted to test two hypotheses on CNN shape classification. The first hypothesis is that transfer of learning based on local features is higher than transfer of learning based on global features. The second hypothesis is that the CNNs with more layers and advanced architectures have higher transfer of learning based global features. The first two experiments examined how the CNNs transferred learning of discriminating local features (square, rectangle, trapezoid, and parallelogram). The other four experiments examined how the CNNs transferred learning of discriminating global features (presence of a hole, connectivity, and inside/outside relationship). While the CNNs exhibited robust learning on classifying shapes, transfer of learning varied from task to task, and model to model. The results rejected both hypotheses. First, some CNNs exhibited lower transfer of learning based on local features than that based on global features. Second the advanced CNNs exhibited lower transfer of learning on global features than that of the earlier models. Among the tested geometric features, we found that learning of discriminating inside/outside relationship was the most difficult to be transferred, indicating an effective benchmark to develop future CNNs. In contrast to the "ImageNet" approach that employs natural images to train and analyze the CNNs, the results show proof of concept for the "ShapeNet" approach that employs well-defined geometric shapes to elucidate the strengths and limitations of the computation in CNN image classification. This "ShapeNet" approach will also provide insights into understanding visual information processing the primate visual systems.
卷积神经网络（CNNs）是一种强大的图像分类工具，已广泛应用于自动场景分割和识别领域。然而，CNN图像分类背后的机制仍有待阐明。在本研究中，我们开发了一种新方法来解决这个问题，即通过研究代表性CNN（AlexNet、VGG、ResNet - 101和Inception - ResNet - v2）在基于局部/全局特征或不变量对几何形状进行分类时的学习迁移情况。局部特征基于简单的组件，如线段的方向或两条线是否平行，而全局特征基于整个对象，如对象是否有孔或一个对象是否在另一个对象内部。进行了六个实验来检验关于CNN形状分类的两个假设。第一个假设是基于局部特征的学习迁移高于基于全局特征的学习迁移。第二个假设是具有更多层和先进架构的CNN基于全局特征的学习迁移更高。前两个实验研究了CNN如何迁移区分局部特征（正方形、长方形、梯形和平行四边形）的学习。另外四个实验研究了CNN如何迁移区分全局特征（是否有孔、连通性以及内外关系）的学习。虽然CNN在形状分类上表现出强大的学习能力，但学习迁移因任务和模型而异。结果否定了这两个假设。首先，一些CNN基于局部特征的学习迁移低于基于全局特征的学习迁移。其次，先进的CNN基于全局特征的学习迁移低于早期模型。在测试的几何特征中，我们发现区分内外关系的学习最难迁移，这为未来CNN的发展提供了一个有效的基准。与采用自然图像训练和分析CNN的“ImageNet”方法不同，结果证明了采用定义明确的几何形状来阐明CNN图像分类中计算的优势和局限性的“ShapeNet”方法的概念验证。这种“ShapeNet”方法也将为理解灵长类视觉系统中的视觉信息处理提供见解。
