IEEE Trans Neural Netw Learn Syst. 2022 Nov;33(11):6532-6544. doi: 10.1109/TNNLS.2021.3082316. Epub 2022 Oct 27.
As an important and challenging problem, multidomain learning (MDL) typically seeks a set of effective lightweight domain-specific adapter modules plugged into a common domain-agnostic network. Usually, existing ways of adapter plugging and structure design are handcrafted and fixed for all domains before model learning, resulting in learning inflexibility and computational intensiveness. With this motivation, we propose to learn a data-driven adapter plugging strategy with neural architecture search (NAS), which automatically determines where to plug for those adapter modules. Furthermore, we propose an NAS-adapter module for adapter structure design in an NAS-driven learning scheme, which automatically discovers effective adapter module structures for different domains. Experimental results demonstrate the effectiveness of our MDL model against existing approaches under the conditions of comparable performance.
作为一个重要且具有挑战性的问题，多领域学习（MDL）通常寻求一组有效的轻量级领域特定适配器模块，插入到一个通用的领域不可知的网络中。通常，现有的适配器插入和结构设计方法是在模型学习之前针对所有领域手工制作和固定的，导致学习的不灵活性和计算密集性。基于此动机，我们提出使用神经架构搜索（NAS）来学习数据驱动的适配器插入策略，该策略自动确定要为那些适配器模块插入的位置。此外，我们在 NAS 驱动的学习方案中提出了一种 NAS 适配器模块，用于适配器结构设计，它可以自动为不同的领域发现有效的适配器模块结构。实验结果表明，在可比性能的条件下，我们的 MDL 模型相对于现有方法是有效的。
