Queen's University, Kingston, ON, Canada.
The University of British Columbia, Vancouver, BC, Canada.
Int J Comput Assist Radiol Surg. 2020 Jul;15(7):1215-1223. doi: 10.1007/s11548-020-02172-5. Epub 2020 May 5.
The detection of clinically significant prostate cancer (PCa) is shown to greatly benefit from MRI-ultrasound fusion biopsy, which involves overlaying pre-biopsy MRI volumes (or targets) with real-time ultrasound images. In previous literature, machine learning models trained on either MRI or ultrasound data have been proposed to improve biopsy guidance and PCa detection. However, quantitative fusion of information from MRI and ultrasound has not been explored in depth in a large study. This paper investigates information fusion approaches between MRI and ultrasound to improve targeting of PCa foci in biopsies.
We build models of fully convolutional networks (FCN) using data from a newly proposed ultrasound modality, temporal enhanced ultrasound (TeUS), and apparent diffusion coefficient (ADC) from 107 patients with 145 biopsy cores. The architecture of our models is based on U-Net and U-Net with attention gates. Models are built using joint training through intermediate and late fusion of the data. We also build models with data from each modality, separately, to use as baseline. The performance is evaluated based on the area under the curve (AUC) for predicting clinically significant PCa.
Using our proposed deep learning framework and intermediate fusion, integration of TeUS and ADC outperforms the individual modalities for cancer detection. We achieve an AUC of 0.76 for detection of all PCa foci, and 0.89 for PCa with larger foci. Results indicate a shared representation between multiple modalities outperforms the average unimodal predictions.
We demonstrate the significant potential of multimodality integration of information from MRI and TeUS to improve PCa detection, which is essential for accurate targeting of cancer foci during biopsy. By using FCNs as the architecture of choice, we are able to predict the presence of clinically significant PCa in entire imaging planes immediately, without the need for region-based analysis. This reduces the overall computational time and enables future intra-operative deployment of this technology.
MRI-超声融合活检显示可大大提高临床显著前列腺癌（PCa）的检测能力，该方法涉及将活检前 MRI 体积（或靶区）与实时超声图像叠加。在之前的文献中，已经提出了基于 MRI 或超声数据训练的机器学习模型来改善活检指导和 PCa 检测。然而，在大型研究中，尚未深入探索 MRI 和超声信息的定量融合。本文研究了 MRI 和超声之间的信息融合方法，以改善活检中 PCa 病灶的靶向性。
我们使用来自新提出的超声模态——时变增强超声（TeUS）和 107 名患者的 145 个活检核心的表观扩散系数（ADC）的数据构建全卷积网络（FCN）模型。我们模型的架构基于 U-Net 和带有注意力门的 U-Net。通过数据的中间和晚期融合进行联合训练来构建模型。我们还分别使用每个模态的数据构建模型作为基线。根据预测临床显著 PCa 的曲线下面积（AUC）来评估性能。
使用我们提出的深度学习框架和中间融合，TeUS 和 ADC 的集成在癌症检测方面优于单独的模态。我们实现了所有 PCa 病灶的检测 AUC 为 0.76，较大病灶的 PCa 检测 AUC 为 0.89。结果表明，多模态信息的共享表示优于平均单模态预测。
我们证明了 MRI 和 TeUS 信息的多模态集成在提高 PCa 检测方面具有重要潜力，这对于活检中准确靶向癌症病灶至关重要。通过使用 FCN 作为首选架构，我们能够立即在整个成像平面上立即预测出具有临床意义的 PCa 的存在，而无需进行基于区域的分析。这减少了整体计算时间，并为这项技术在未来的术中部署奠定了基础。
