Medical Device and Robot Institute of Park (MDRIP), Kyungpook National University, Daegu, South Korea.
Robotics Engineering, Daegu Gyungbuk Institute of Science & Technology, Daegu, South Korea.
Int J Comput Assist Radiol Surg. 2020 Jun;15(6):931-941. doi: 10.1007/s11548-020-02192-1. Epub 2020 May 12.
Precise localization of cystic bone lesions is crucial for osteolytic bone tumor surgery. Recently, there is a move toward ultrasound imaging over plain radiographs (X-rays) for intra-operative navigation due to the radiation-free and cost-effectiveness of the modality. In this process, the intra-operative bone model reconstructed from the segmented ultrasound image is registered with the pre-operative bone model. Deep learning approaches have recently shown remarkable success in bone surface segmentation from ultrasound images. However, to train deep learning models effectively with limited dataset size, data augmentation is essential. This study investigates the applicability of the generative approach for data augmentation as well as identifies standard data augmentation approaches for bone surface segmentation from ultrasound images.
The generative approach we used in our work is based on Pix2Pix image-to-image translation network. We have proposed a multiple-snapshot approach, which mitigates the uni-modal deterministic output issue in the Pix2Pix network without using any complex architecture and training process. We also identified standard data augmentation approaches necessary for ultrasound bone surface segmentation through experiments.
We have evaluated our networks using 800 ultrasound images from trained regions (humerus bone) and 1200 ultrasound images from untrained regions (tibia and femur bones) using four different augmentation approaches. The results show that the generative augmentation approach has a positive impact on accuracy in both trained (+ 4.88%) and untrained regions (+ 25.84%) compared to using only standard augmentations. Moreover, compared to standard augmentation approaches, the addition of the generative augmentation approach also showed a similar trend in both trained (+ 8.74%) and untrained (+ 11.55%) regions.
Generative approaches are very beneficial for data augmentation, where limited dataset size is prevalent, such as ultrasound bone segmentation. The proposed multiple-snapshot Pix2Pix approach has the potential to generate multimodal images, which enlarges the dataset considerably.
囊性骨病变的精确定位对于溶骨性骨肿瘤手术至关重要。由于该方法无辐射且具有成本效益，因此最近倾向于使用超声成像替代普通 X 射线（射线）进行术中导航。在此过程中，从分割的超声图像重建的术中骨模型与术前骨模型进行配准。深度学习方法最近在超声图像中的骨表面分割方面取得了显著成功。然而，为了有效地使用有限的数据集大小训练深度学习模型，数据增强是必不可少的。本研究调查了生成方法在数据增强中的适用性，并确定了用于从超声图像中分割骨表面的标准数据增强方法。
我们在工作中使用的生成方法基于 Pix2Pix 图像到图像翻译网络。我们提出了一种多快照方法，该方法减轻了 Pix2Pix 网络中的单模态确定性输出问题，而无需使用任何复杂的架构和训练过程。我们还通过实验确定了用于超声骨表面分割的标准数据增强方法。
我们使用来自训练区域（肱骨）的 800 个超声图像和来自未训练区域（胫骨和股骨）的 1200 个超声图像，通过四种不同的增强方法评估了我们的网络。结果表明，与仅使用标准增强相比，生成增强方法对训练区域（+4.88%）和未训练区域（+25.84%）的准确性都有积极的影响。此外，与标准增强方法相比，生成增强方法的添加在训练区域（+8.74%）和未训练区域（+11.55%）中也表现出相似的趋势。
生成方法对于数据增强非常有益，在数据集有限的情况下，例如超声骨分割。所提出的多快照 Pix2Pix 方法具有生成多模态图像的潜力，这大大增加了数据集的规模。
