IEEE Trans Med Imaging. 2022 Mar;41(3):715-726. doi: 10.1109/TMI.2021.3121138. Epub 2022 Mar 2.
Unsupervised domain adaptation (UDA), aiming to adapt the model to an unseen domain without annotations, has drawn sustained attention in surgical instrument segmentation. Existing UDA methods neglect the domain-common knowledge of two datasets, thus failing to grasp the inter-category relationship in the target domain and leading to poor performance. To address these issues, we propose a graph-based unsupervised domain adaptation framework, named Interactive Graph Network (IGNet), to effectively adapt a model to an unlabeled new domain in surgical instrument segmentation tasks. In detail, the Domain-common Prototype Constructor (DPC) is first advanced to adaptively aggregate the feature map into domain-common prototypes using the probability mixture model, and construct a prototypical graph to interact the information among prototypes from the global perspective. In this way, DPC can grasp the co-occurrent and long-range relationship for both domains. To further narrow down the domain gap, we design a Domain-common Knowledge Incorporator (DKI) to guide the evolution of feature maps towards domain-common direction via a common-knowledge guidance graph and category-attentive graph reasoning. At last, the Cross-category Mismatch Estimator (CME) is developed to evaluate the category-level alignment from a graph perspective and assign each pixel with different adversarial weights, so as to refine the feature distribution alignment. The extensive experiments on three types of tasks demonstrate the feasibility and superiority of IGNet compared with other state-of-the-art methods. Furthermore, ablation studies verify the effectiveness of each component of IGNet. The source code is available at https://github.com/CityU-AIM-Group/Prototypical-Graph-DA.
无监督领域自适应 (UDA) 旨在在没有标注的情况下将模型自适应到未见过的领域，在手术器械分割中受到了持续关注。现有的 UDA 方法忽略了两个数据集的领域公共知识，因此无法掌握目标领域中的类别间关系，导致性能较差。为了解决这些问题，我们提出了一种基于图的无监督领域自适应框架，称为交互式图网络 (IGNet)，以有效地将模型自适应到手术器械分割任务中的未标记新领域。具体来说，首先提出了域公共原型构造器 (DPC)，使用概率混合模型自适应地将特征图聚合到域公共原型中，并构建一个原型图，从全局角度交互原型之间的信息。通过这种方式，DPC 可以捕捉到两个域的共同出现和长程关系。为了进一步缩小域差距，我们设计了域公共知识整合器 (DKI)，通过公共知识引导图和类别注意力图推理，引导特征图向域公共方向进化。最后，开发了跨类别不匹配估计器 (CME) ，从图的角度评估类别级对齐，并为每个像素分配不同的对抗权重，从而细化特征分布对齐。在三种类型的任务上的广泛实验表明，与其他最先进的方法相比，IGNet 具有可行性和优越性。此外，消融研究验证了 IGNet 每个组件的有效性。代码可在 https://github.com/CityU-AIM-Group/Prototypical-Graph-DA 获得。
