Sano Centre for Computational Medicine, Krakow, Poland.
Department of Computer Science, University of Verona, Verona, Italy.
Int J Comput Assist Radiol Surg. 2024 Mar;19(3):531-539. doi: 10.1007/s11548-023-03030-w. Epub 2023 Nov 7.
Computer-assisted surgical systems provide support information to the surgeon, which can improve the execution and overall outcome of the procedure. These systems are based on deep learning models that are trained on complex and challenging-to-annotate data. Generating synthetic data can overcome these limitations, but it is necessary to reduce the domain gap between real and synthetic data.
We propose a method for image-to-image translation based on a Stable Diffusion model, which generates realistic images starting from synthetic data. Compared to previous works, the proposed method is better suited for clinical application as it requires a much smaller amount of input data and allows finer control over the generation of details by introducing different variants of supporting control networks.
The proposed method is applied in the context of laparoscopic cholecystectomy, using synthetic and real data from public datasets. It achieves a mean Intersection over Union of 69.76%, significantly improving the baseline results (69.76 vs. 42.21%).
The proposed method for translating synthetic images into images with realistic characteristics will enable the training of deep learning methods that can generalize optimally to real-world contexts, thereby improving computer-assisted intervention guidance systems.
计算机辅助手术系统为外科医生提供支持信息，从而提高手术的执行效率和整体效果。这些系统基于经过复杂且难以标注数据训练的深度学习模型。生成合成数据可以克服这些限制，但需要缩小真实数据和合成数据之间的领域差距。
我们提出了一种基于稳定扩散模型的图像到图像转换方法，该方法可以从合成数据生成逼真的图像。与之前的工作相比，该方法更适合临床应用，因为它需要的输入数据量要少得多，并且通过引入不同变体的支持控制网络，可以更好地控制细节的生成。
该方法应用于腹腔镜胆囊切除术，使用来自公共数据集的合成数据和真实数据。它的交并比均值为 69.76%，明显优于基线结果（69.76 比 42.21%）。
将合成图像转换为具有逼真特征的图像的方法将使能够训练出可以最优地推广到真实环境的深度学习方法，从而改进计算机辅助干预指导系统。
