IEEE Trans Image Process. 2021;30:2045-2059. doi: 10.1109/TIP.2021.3050668. Epub 2021 Jan 21.
Instance segmentation is an important task for biomedical and biological image analysis. Due to the complicated background components, the high variability of object appearances, numerous overlapping objects, and ambiguous object boundaries, this task still remains challenging. Recently, deep learning based methods have been widely employed to solve these problems and can be categorized into proposal-free and proposal-based methods. However, both proposal-free and proposal-based methods suffer from information loss, as they focus on either global-level semantic or local-level instance features. To tackle this issue, we present a Panoptic Feature Fusion Net (PFFNet) that unifies the semantic and instance features in this work. Specifically, our proposed PFFNet contains a residual attention feature fusion mechanism to incorporate the instance prediction with the semantic features, in order to facilitate the semantic contextual information learning in the instance branch. Then, a mask quality sub-branch is designed to align the confidence score of each object with the quality of the mask prediction. Furthermore, a consistency regularization mechanism is designed between the semantic segmentation tasks in the semantic and instance branches, for the robust learning of both tasks. Extensive experiments demonstrate the effectiveness of our proposed PFFNet, which outperforms several state-of-the-art methods on various biomedical and biological datasets.
实例分割是生物医学和生物图像分析的重要任务。由于复杂的背景成分、对象外观的高度可变性、众多重叠的对象和模糊的对象边界，这项任务仍然具有挑战性。最近，基于深度学习的方法已被广泛应用于解决这些问题，可以分为无提议和基于提议的方法。然而，无提议和基于提议的方法都存在信息丢失的问题，因为它们要么侧重于全局级别的语义，要么侧重于局部级别的实例特征。为了解决这个问题，我们提出了一个 Panoptic Feature Fusion Net (PFFNet)，在这项工作中统一了语义和实例特征。具体来说，我们提出的 PFFNet 包含一个残差注意力特征融合机制，将实例预测与语义特征结合起来，以便在实例分支中促进语义上下文信息的学习。然后，设计了一个掩模质量子分支，以将每个对象的置信得分与掩模预测的质量对齐。此外，在语义和实例分支的语义分割任务之间设计了一致性正则化机制，以稳健地学习这两个任务。广泛的实验表明，我们提出的 PFFNet 是有效的，在各种生物医学和生物数据集上优于几种最先进的方法。
