Cheng Ao, Shi Jiahao, Wang Lirong, Zhang Ruobing
School of Electronic and Information Engineering, Soochow University, Suzhou, China.
Jiangsu Key Laboratory of Medical Optics, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China.
Front Neuroinform. 2023 Jun 8;17:1118419. doi: 10.3389/fninf.2023.1118419. eCollection 2023.
The exorbitant cost of accurately annotating the large-scale serial scanning electron microscope (SEM) images as the ground truth for training has always been a great challenge for brain map reconstruction by deep learning methods in neural connectome studies. The representation ability of the model is strongly correlated with the number of such high-quality labels. Recently, the masked autoencoder (MAE) has been shown to effectively pre-train Vision Transformers (ViT) to improve their representational capabilities.
In this paper, we investigated a self-pre-training paradigm for serial SEM images with MAE to implement downstream segmentation tasks. We randomly masked voxels in three-dimensional brain image patches and trained an autoencoder to reconstruct the neuronal structures.
We tested different pre-training and fine-tuning configurations on three different serial SEM datasets of mouse brains, including two public ones, SNEMI3D and MitoEM-R, and one acquired in our lab. A series of masking ratios were examined and the optimal ratio for pre-training efficiency was spotted for 3D segmentation. The MAE pre-training strategy significantly outperformed the supervised learning from scratch. Our work shows that the general framework of can be a unified approach for effective learning of the representation of heterogeneous neural structural features in serial SEM images to greatly facilitate brain connectome reconstruction.
准确注释大规模连续扫描电子显微镜（SEM）图像作为训练的基准真值成本过高，这一直是神经连接组研究中通过深度学习方法进行脑图谱重建的巨大挑战。模型的表征能力与此类高质量标签的数量密切相关。最近，掩码自动编码器（MAE）已被证明能有效预训练视觉Transformer（ViT）以提高其表征能力。
在本文中，我们研究了一种使用MAE对连续SEM图像进行自预训练的范式，以实现下游分割任务。我们在三维脑图像块中随机掩码体素，并训练一个自动编码器来重建神经元结构。
我们在三个不同的小鼠脑连续SEM数据集上测试了不同的预训练和微调配置，包括两个公开数据集SNEMI3D和MitoEM - R，以及我们实验室获取的数据。研究了一系列掩码比例，并找出了用于3D分割的预训练效率最佳比例。MAE预训练策略显著优于从头开始的监督学习。我们的工作表明，该通用框架可以成为有效学习连续SEM图像中异质神经结构特征表征的统一方法，极大地促进脑连接组重建。
