Xiao Anqi, Shen Biluo, Tian Jie, Hu Zhenhua
IEEE Trans Neural Netw Learn Syst. 2024 Sep;35(9):12718-12730. doi: 10.1109/TNNLS.2023.3264551. Epub 2024 Sep 3.
Multiscale features are of great importance in modern convolutional neural networks, showing consistent performance gains on numerous vision tasks. Therefore, many plug-and-play blocks are introduced to upgrade existing convolutional neural networks for stronger multiscale representation ability. However, the design of plug-and-play blocks is getting more and more complex, and these manually designed blocks are not optimal. In this work, we propose PP-NAS to develop plug-and-play blocks based on neural architecture search (NAS). Specifically, we design a new search space PPConv and develop a search algorithm consisting of one-level optimization, zero-one loss, and connection existence loss. PP-NAS minimizes the optimization gap between super-net and subarchitectures and can achieve good performance even without retraining. Extensive experiments on image classification, object detection, and semantic segmentation verify the superiority of PP-NAS over state-of-the-art CNNs (e.g., ResNet, ResNeXt, and Res2Net). Our code is available at https://github.com/ainieli/PP-NAS.
多尺度特征在现代卷积神经网络中非常重要，在众多视觉任务中展现出持续的性能提升。因此，许多即插即用模块被引入以升级现有的卷积神经网络，使其具有更强的多尺度表示能力。然而，即插即用模块的设计变得越来越复杂，并且这些手动设计的模块并非最优。在这项工作中，我们提出了PP-NAS，以基于神经架构搜索（NAS）来开发即插即用模块。具体而言，我们设计了一个新的搜索空间PPConv，并开发了一种由一级优化、零一损失和连接存在损失组成的搜索算法。PP-NAS最小化了超网络和子架构之间的优化差距，甚至无需重新训练就能取得良好性能。在图像分类、目标检测和语义分割上的大量实验验证了PP-NAS优于当前最先进的卷积神经网络（例如，ResNet、ResNeXt和Res2Net）。我们的代码可在https://github.com/ainieli/PP-NAS获取。
