Wellcome/EPSRC Centre for Interventional and Surgical Sciences and the Centre for Medical Image Computing, UCL, London, UK.
Division of Surgery and Interventional Science, UCL, London, UK.
Int J Comput Assist Radiol Surg. 2022 Jan;17(1):167-176. doi: 10.1007/s11548-021-02518-7. Epub 2021 Oct 26.
The initial registration of a 3D pre-operative CT model to a 2D laparoscopic video image in augmented reality systems for liver surgery needs to be fast, intuitive to perform and with minimal interruptions to the surgical intervention. Several recent methods have focussed on using easily recognisable landmarks across modalities. However, these methods still need manual annotation or manual alignment. We propose a novel, fully automatic pipeline for 3D-2D global registration in laparoscopic liver interventions.
Firstly, we train a fully convolutional network for the semantic detection of liver contours in laparoscopic images. Secondly, we propose a novel contour-based global registration algorithm to estimate the camera pose without any manual input during surgery. The contours used are the anterior ridge and the silhouette of the liver.
We show excellent generalisation of the semantic contour detection on test data from 8 clinical cases. In quantitative experiments, the proposed contour-based registration can successfully estimate a global alignment with as little as 30% of the liver surface, a visibility ratio which is characteristic of laparoscopic interventions. Moreover, the proposed pipeline showed very promising results in clinical data from 5 laparoscopic interventions.
Our proposed automatic global registration could make augmented reality systems more intuitive and usable for surgeons and easier to translate to operating rooms. Yet, as the liver is deformed significantly during surgery, it will be very beneficial to incorporate deformation into our method for more accurate registration.
在用于肝外科手术的增强现实系统中，将术前 CT 三维模型初始配准到二维腹腔镜视频图像需要快速、直观且对手术干预的干扰最小。最近有几种方法集中在使用跨模态的易于识别的地标。然而，这些方法仍然需要手动注释或手动配准。我们提出了一种新颖的、完全自动的腹腔镜肝介入 3D-2D 全局配准流水线。
首先，我们训练了一个全卷积网络来对腹腔镜图像中的肝轮廓进行语义检测。其次，我们提出了一种新颖的基于轮廓的全局配准算法，无需在手术过程中进行任何手动输入即可估计相机姿态。所使用的轮廓是肝的前脊和轮廓。
我们在来自 8 个临床病例的测试数据上展示了语义轮廓检测的出色泛化能力。在定量实验中，所提出的基于轮廓的配准可以成功地以 30%的肝表面作为可见性比例估计全局对齐，这是腹腔镜介入的特征。此外，所提出的流水线在来自 5 个腹腔镜干预的临床数据中显示出非常有前景的结果。
我们提出的自动全局配准可以使增强现实系统对外科医生来说更加直观和易用，并且更容易转化为手术室。然而，由于肝脏在手术过程中会发生显著变形，因此将变形纳入我们的方法中以实现更精确的配准将非常有益。
