Division of Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany.
Helmholtz Information and Data Science School for Health, Karlsruhe/Heidelberg, Germany.
Int J Comput Assist Radiol Surg. 2024 Jun;19(6):1021-1031. doi: 10.1007/s11548-024-03085-3. Epub 2024 Mar 14.
Surgical scene segmentation is crucial for providing context-aware surgical assistance. Recent studies highlight the significant advantages of hyperspectral imaging (HSI) over traditional RGB data in enhancing segmentation performance. Nevertheless, the current hyperspectral imaging (HSI) datasets remain limited and do not capture the full range of tissue variations encountered clinically.
Based on a total of 615 hyperspectral images from a total of 16 pigs, featuring porcine organs in different perfusion states, we carry out an exploration of distribution shifts in spectral imaging caused by perfusion alterations. We further introduce a novel strategy to mitigate such distribution shifts, utilizing synthetic data for test-time augmentation.
The effect of perfusion changes on state-of-the-art (SOA) segmentation networks depended on the organ and the specific perfusion alteration induced. In the case of the kidney, we observed a performance decline of up to 93% when applying a state-of-the-art (SOA) network under ischemic conditions. Our method improved on the state-of-the-art (SOA) by up to 4.6 times.
Given its potential wide-ranging relevance to diverse pathologies, our approach may serve as a pivotal tool to enhance neural network generalization within the realm of spectral imaging.
手术场景分割对于提供上下文感知的手术辅助至关重要。最近的研究强调了高光谱成像（HSI）相对于传统 RGB 数据在提高分割性能方面的显著优势。然而，当前的高光谱成像（HSI）数据集仍然有限，无法捕捉到临床上遇到的所有组织变化。
基于总共 16 头猪的 615 张高光谱图像，这些猪的器官处于不同的灌注状态，我们探索了由灌注变化引起的光谱成像中的分布偏移。我们进一步引入了一种新策略，利用合成数据进行测试时增强，以减轻这种分布偏移。
灌注变化对最先进（SOA）分割网络的影响取决于器官和诱导的具体灌注变化。在肾脏的情况下，当在缺血条件下应用最先进的（SOA）网络时，我们观察到性能下降高达 93%。我们的方法将最先进的（SOA）提高了 4.6 倍。
鉴于其对多种病理的潜在广泛相关性，我们的方法可以作为一种关键工具，增强光谱成像中神经网络的泛化能力。
