Computer-assisted Applications in Medicine, ETH Zurich, Zürich, Switzerland.
Department of Information Technology, Uppsala University, Uppsala, Sweden.
Int J Comput Assist Radiol Surg. 2021 May;16(5):721-730. doi: 10.1007/s11548-021-02349-6. Epub 2021 Apr 8.
Given the high level of expertise required for navigation and interpretation of ultrasound images, computational simulations can facilitate the training of such skills in virtual reality. With ray-tracing based simulations, realistic ultrasound images can be generated. However, due to computational constraints for interactivity, image quality typically needs to be compromised.
We propose herein to bypass any rendering and simulation process at interactive time, by conducting such simulations during a non-time-critical offline stage and then learning image translation from cross-sectional model slices to such simulated frames. We use a generative adversarial framework with a dedicated generator architecture and input feeding scheme, which both substantially improve image quality without increase in network parameters. Integral attenuation maps derived from cross-sectional model slices, texture-friendly strided convolutions, providing stochastic noise and input maps to intermediate layers in order to preserve locality are all shown herein to greatly facilitate such translation task.
Given several quality metrics, the proposed method with only tissue maps as input is shown to provide comparable or superior results to a state-of-the-art that uses additional images of low-quality ultrasound renderings. An extensive ablation study shows the need and benefits from the individual contributions utilized in this work, based on qualitative examples and quantitative ultrasound similarity metrics. To that end, a local histogram statistics based error metric is proposed and demonstrated for visualization of local dissimilarities between ultrasound images.
A deep-learning based direct transformation from interactive tissue slices to likeness of high quality renderings allow to obviate any complex rendering process in real-time, which could enable extremely realistic ultrasound simulations on consumer-hardware by moving the time-intensive processes to a one-time, offline, preprocessing data preparation stage that can be performed on dedicated high-end hardware.
由于导航和解释超声图像需要高度的专业知识，计算仿真可以在虚拟现实中促进这些技能的培训。基于光线追踪的仿真可以生成逼真的超声图像。然而，由于交互性的计算限制，通常需要折衷图像质量。
我们在此提出通过在非时间关键的离线阶段进行此类仿真，并从横截面模型切片学习到此类模拟帧的图像转换，从而避免在交互时进行任何渲染和仿真过程。我们使用具有专用生成器架构和输入馈送方案的生成对抗网络，这两者都在不增加网络参数的情况下大大提高了图像质量。从横截面模型切片中得出的积分衰减图、纹理友好的跨步卷积、为中间层提供随机噪声和输入图以保持局部性，所有这些都极大地促进了这种转换任务。
给定几个质量指标，仅使用组织图作为输入的所提出的方法被证明可以提供与使用低质量超声渲染的附加图像的最先进方法相当或更好的结果。广泛的消融研究表明，基于定性示例和定量超声相似性指标，所利用的各个贡献都需要并受益于此工作。为此，提出并演示了一种基于局部直方图统计的误差度量，用于可视化超声图像之间的局部差异。
基于深度学习的从交互式组织切片到高质量渲染的直接转换，可以避免在实时中进行任何复杂的渲染过程，这可以通过将时间密集型过程转移到一次性离线预处理数据准备阶段来在消费者硬件上实现极其逼真的超声仿真，该阶段可以在专用的高端硬件上执行。
