Yang Zixin, Simon Richard, Linte Cristian A
Rochester Institute of Technology, Center for Imaging Science, Rochester, New York, United States.
Rochester Institute of Technology, Department of Biomedical Engineering, Rochester, New York, United States.
J Med Imaging (Bellingham). 2023 Jul;10(4):045001. doi: 10.1117/1.JMI.10.4.045001. Epub 2023 Jul 14.
Stereo matching methods that enable depth estimation are crucial for visualization enhancement applications in computer-assisted surgery. Learning-based stereo matching methods have shown great promise in making accurate predictions on laparoscopic images. However, they require a large amount of training data, and their performance may be degraded due to domain shifts.
Maintaining robustness and improving the accuracy of learning-based methods are still open problems. To overcome the limitations of learning-based methods, we propose a disparity refinement framework consisting of a local disparity refinement method and a global disparity refinement method to improve the results of learning-based stereo matching methods in a cross-domain setting. Those learning-based stereo matching methods are pre-trained on a large public dataset of natural images and are tested on two datasets of laparoscopic images.
Qualitative and quantitative results suggest that our proposed disparity framework can effectively refine disparity maps when they are noise-corrupted on an unseen dataset, without compromising prediction accuracy when the network can generalize well on an unseen dataset.
Our proposed disparity refinement framework could work with learning-based methods to achieve robust and accurate disparity prediction. Yet, as a large laparoscopic dataset for training learning-based methods does not exist and the generalization ability of networks remains to be improved, the incorporation of the proposed disparity refinement framework into existing networks will contribute to improving their overall accuracy and robustness associated with depth estimation.
能够进行深度估计的立体匹配方法对于计算机辅助手术中的可视化增强应用至关重要。基于学习的立体匹配方法在对腹腔镜图像进行准确预测方面显示出了巨大的潜力。然而，它们需要大量的训练数据，并且其性能可能会因域偏移而下降。
保持基于学习的方法的鲁棒性并提高其准确性仍然是未解决的问题。为了克服基于学习的方法的局限性，我们提出了一个视差细化框架，该框架由局部视差细化方法和全局视差细化方法组成，以在跨域设置中改进基于学习的立体匹配方法的结果。那些基于学习的立体匹配方法在一个大型自然图像公共数据集上进行预训练，并在两个腹腔镜图像数据集上进行测试。
定性和定量结果表明，我们提出的视差框架可以在未见过的数据集上对视差图进行噪声破坏时有效地细化它们，并且在网络能够很好地在未见过的数据集上进行泛化时不会损害预测准确性。
我们提出的视差细化框架可以与基于学习的方法一起工作，以实现鲁棒且准确的视差预测。然而，由于不存在用于训练基于学习的方法的大型腹腔镜数据集，并且网络的泛化能力仍有待提高，将所提出的视差细化框架纳入现有网络将有助于提高它们与深度估计相关的整体准确性和鲁棒性。
