Khosravi Bardia, Rouzrokh Pouria, Mickley John P, Faghani Shahriar, Mulford Kellen, Yang Linjun, Larson A Noelle, Howe Benjamin M, Erickson Bradley J, Taunton Michael J, Wyles Cody C
Department of Orthopedic Surgery, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, USA; Department of Radiology, Mayo Clinic, Rochester, MN, USA.
Department of Orthopedic Surgery, Mayo Clinic, 200 First Street SW, Rochester, MN 55905, USA.
Comput Methods Programs Biomed. 2023 Dec;242:107832. doi: 10.1016/j.cmpb.2023.107832. Epub 2023 Sep 26.
Medical image analysis pipelines often involve segmentation, which requires a large amount of annotated training data, which is time-consuming and costly. To address this issue, we proposed leveraging generative models to achieve few-shot image segmentation.
We trained a denoising diffusion probabilistic model (DDPM) on 480,407 pelvis radiographs to generate 256 ✕ 256 px synthetic images. The DDPM was conditioned on demographic and radiologic characteristics and was rigorously validated by domain experts and objective image quality metrics (Frechet inception distance [FID] and inception score [IS]). For the next step, three landmarks (greater trochanter [GT], lesser trochanter [LT], and obturator foramen [OF]) were annotated on 45 real-patient radiographs; 25 for training and 20 for testing. To extract features, each image was passed through the pre-trained DDPM at three timesteps and for each pass, features from specific blocks were extracted. The features were concatenated with the real image to form an image with 4225 channels. The feature-set was broken into random patches, which were fed to a U-Net. Dice Similarity Coefficient (DSC) was used to compare the performance with a vanilla U-Net trained on radiographs.
Expert accuracy was 57.5 % in determining real versus generated images, while the model reached an FID = 7.2 and IS = 210. The segmentation UNet trained on the 20 feature-sets achieved a DSC of 0.90, 0.84, and 0.61 for OF, GT, and LT segmentation, respectively, which was at least 0.30 points higher than the naively trained model.
We demonstrated the applicability of DDPMs as feature extractors, facilitating medical image segmentation with few annotated samples.
医学图像分析流程通常涉及分割，这需要大量带注释的训练数据，既耗时又昂贵。为解决此问题，我们提议利用生成模型来实现少样本图像分割。
我们在480407张骨盆X光片上训练了一个去噪扩散概率模型（DDPM），以生成256×256像素的合成图像。DDPM以人口统计学和放射学特征为条件，并由领域专家和客观图像质量指标（弗雷歇 inception 距离 [FID] 和 inception 分数 [IS]）进行严格验证。下一步，在45张真实患者的X光片上标注了三个地标（大转子 [GT]、小转子 [LT] 和闭孔 [OF]）；25张用于训练，20张用于测试。为了提取特征，每张图像在三个时间步长通过预训练的DDPM，并且每次通过时，从特定块中提取特征。这些特征与真实图像连接起来，形成一个具有4225个通道的图像。特征集被分解为随机补丁，然后输入到一个U-Net中。使用骰子相似系数（DSC）来比较其与在X光片上训练的普通U-Net的性能。
专家在确定真实图像与生成图像时的准确率为57.5%，而模型的FID = 7.2，IS = 210。在20个特征集上训练的分割U-Net在OF、GT和LT分割上的DSC分别达到了0.90、0.84和0.61，比未经训练的模型至少高出0.30分。
我们证明了DDPM作为特征提取器的适用性，有助于在注释样本较少的情况下进行医学图像分割。
