Shi Peilun, Qiu Jianing, Abaxi Sai Mu Dalike, Wei Hao, Lo Frank P-W, Yuan Wu
Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China.
Department of Computing, Imperial College London, London SW7 2AZ, UK.
Diagnostics (Basel). 2023 Jun 2;13(11):1947. doi: 10.3390/diagnostics13111947.
Medical image analysis plays an important role in clinical diagnosis. In this paper, we examine the recent Segment Anything Model (SAM) on medical images, and report both quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks, covering various imaging modalities, such as optical coherence tomography (OCT), magnetic resonance imaging (MRI), and computed tomography (CT), as well as different applications including dermatology, ophthalmology, and radiology. Those benchmarks are representative and commonly used in model development. Our experimental results indicate that while SAM presents remarkable segmentation performance on images from the general domain, its zero-shot segmentation ability remains restricted for out-of-distribution images, e.g., medical images. In addition, SAM exhibits inconsistent zero-shot segmentation performance across different unseen medical domains. For certain structured targets, e.g., blood vessels, the zero-shot segmentation of SAM completely failed. In contrast, a simple fine-tuning of it with a small amount of data could lead to remarkable improvement of the segmentation quality, showing the great potential and feasibility of using fine-tuned SAM to achieve accurate medical image segmentation for a precision diagnostics. Our study indicates the versatility of generalist vision foundation models on medical imaging, and their great potential to achieve desired performance through fine-turning and eventually address the challenges associated with accessing large and diverse medical datasets in support of clinical diagnostics.
医学图像分析在临床诊断中发挥着重要作用。在本文中，我们研究了最近的医学图像分割模型（SAM），并报告了在九个医学图像分割基准上的定量和定性零样本分割结果，这些基准涵盖了各种成像模态，如光学相干断层扫描（OCT）、磁共振成像（MRI）和计算机断层扫描（CT），以及不同的应用领域，包括皮肤病学、眼科和放射学。这些基准在模型开发中具有代表性且常用。我们的实验结果表明，虽然SAM在通用领域的图像上表现出卓越的分割性能，但其零样本分割能力在分布外图像（如医学图像）上仍然受限。此外，SAM在不同的未见医学领域中表现出不一致的零样本分割性能。对于某些结构化目标，如血管，SAM的零样本分割完全失败。相比之下，用少量数据对其进行简单微调可以显著提高分割质量，这表明使用微调后的SAM实现精确医学图像分割以进行精准诊断具有巨大潜力和可行性。我们的研究表明通用视觉基础模型在医学成像方面的通用性，以及通过微调实现理想性能并最终应对与获取大量多样的医学数据集以支持临床诊断相关挑战的巨大潜力。
