IEEE Trans Med Imaging. 2022 Oct;41(10):2582-2597. doi: 10.1109/TMI.2022.3169449. Epub 2022 Sep 30.
Deep learning (DL) based semantic segmentation methods have achieved excellent performance in biomedical image segmentation, producing high quality probability maps to allow extraction of rich instance information to facilitate good instance segmentation. While numerous efforts were put into developing new DL semantic segmentation models, less attention was paid to a key issue of how to effectively explore their probability maps to attain the best possible instance segmentation. We observe that probability maps by DL semantic segmentation models can be used to generate many possible instance candidates, and accurate instance segmentation can be achieved by selecting from them a set of "optimized" candidates as output instances. Further, the generated instance candidates form a well-behaved hierarchical structure (a forest), which allows selecting instances in an optimized manner. Hence, we propose a novel framework, called hierarchical earth mover's distance (H-EMD), for instance segmentation in biomedical 2D+time videos and 3D images, which judiciously incorporates consistent instance selection with semantic-segmentation-generated probability maps. H-EMD contains two main stages: (1) instance candidate generation: capturing instance-structured information in probability maps by generating many instance candidates in a forest structure; (2) instance candidate selection: selecting instances from the candidate set for final instance segmentation. We formulate a key instance selection problem on the instance candidate forest as an optimization problem based on the earth mover's distance (EMD), and solve it by integer linear programming. Extensive experiments on eight biomedical video or 3D datasets demonstrate that H-EMD consistently boosts DL semantic segmentation models and is highly competitive with state-of-the-art methods.
基于深度学习 (DL) 的语义分割方法在生物医学图像分割中取得了优异的性能，生成高质量的概率图以提取丰富的实例信息，从而实现良好的实例分割。尽管在开发新的 DL 语义分割模型方面付出了大量努力，但对于如何有效地探索其概率图以获得最佳实例分割这一关键问题的关注较少。我们观察到，DL 语义分割模型的概率图可用于生成许多可能的实例候选者，通过从其中选择一组“优化”候选者作为输出实例，可以实现准确的实例分割。此外，生成的实例候选者形成了一个表现良好的层次结构（森林），可以以优化的方式选择实例。因此，我们提出了一种名为层次化 earth mover's distance (H-EMD) 的新框架，用于生物医学 2D+时间视频和 3D 图像的实例分割，该框架巧妙地将一致的实例选择与语义分割生成的概率图相结合。H-EMD 包含两个主要阶段：（1）实例候选者生成：通过生成森林结构中的许多实例候选者来捕获概率图中的实例结构信息；（2）实例候选者选择：从候选集选择实例以进行最终的实例分割。我们将实例候选者森林上的关键实例选择问题表示为基于 earth mover's distance (EMD) 的优化问题，并通过整数线性规划求解。在八个生物医学视频或 3D 数据集上的广泛实验表明，H-EMD 能够显著提升 DL 语义分割模型的性能，并且与最先进的方法具有高度竞争力。
