IEEE Trans Med Imaging. 2023 Dec;42(12):3956-3971. doi: 10.1109/TMI.2023.3320497. Epub 2023 Nov 30.
In this paper, we present the results of the MitoEM challenge on mitochondria 3D instance segmentation from electron microscopy images, organized in conjunction with the IEEE-ISBI 2021 conference. Our benchmark dataset consists of two large-scale 3D volumes, one from human and one from rat cortex tissue, which are 1,986 times larger than previously used datasets. At the time of paper submission, 257 participants had registered for the challenge, 14 teams had submitted their results, and six teams participated in the challenge workshop. Here, we present eight top-performing approaches from the challenge participants, along with our own baseline strategies. Posterior to the challenge, annotation errors in the ground truth were corrected without altering the final ranking. Additionally, we present a retrospective evaluation of the scoring system which revealed that: 1) challenge metric was permissive with the false positive predictions; and 2) size-based grouping of instances did not correctly categorize mitochondria of interest. Thus, we propose a new scoring system that better reflects the correctness of the segmentation results. Although several of the top methods are compared favorably to our own baselines, substantial errors remain unsolved for mitochondria with challenging morphologies. Thus, the challenge remains open for submission and automatic evaluation, with all volumes available for download.
在本文中，我们展示了与 2021 年 IEEE-ISBI 会议联合组织的 MitoEM 挑战赛在从电子显微镜图像中对线粒体 3D 实例分割的结果。我们的基准数据集由两个大规模的 3D 体组成，一个来自人类皮层组织，一个来自大鼠皮层组织，其大小分别是之前使用的数据集的 1986 倍。在提交论文时，已有 257 名参与者注册了挑战赛，有 14 个团队提交了他们的结果，有 6 个团队参加了挑战赛研讨会。在这里，我们展示了挑战赛参与者中的 8 种表现最佳的方法，以及我们自己的基线策略。挑战赛结束后，对地面实况中的注释错误进行了更正，而没有改变最终排名。此外，我们对评分系统进行了回顾性评估，结果表明：1）挑战赛指标对假阳性预测较为宽容；2）基于实例大小的分组没有正确地对感兴趣的线粒体进行分类。因此，我们提出了一种新的评分系统，该系统能更好地反映分割结果的正确性。尽管一些顶级方法与我们自己的基线相比表现良好，但对于具有挑战性形态的线粒体，仍然存在大量未解决的错误。因此，挑战赛仍可提交和自动评估，所有数据集都可下载。
