Bao Qin-Zhou, Yang Yi-Xin, Li Qing, Yang Hai-Chao
College of Mathematics and Computer Science, Dali University, Dali, China.
School of Electronic Information, Xijing University, Xian, China.
Front Plant Sci. 2025 May 5;16:1536226. doi: 10.3389/fpls.2025.1536226. eCollection 2025.
Image instance segmentation is essential for plant phenotyping in vertical farms, yet the diversity of plant types and limited annotated image data constrain the performance of traditional supervised techniques. These challenges necessitate a zero-shot approach to enable segmentation without relying on specific training data for each plant type.
We present a zero-shot instance segmentation framework combining Grounding DINO and the Segment Anything Model (SAM). To enhance box prompts, Vegetation Cover Aware Non-Maximum Suppression (VC-NMS) incorporating the Normalized Cover Green Index (NCGI) is used to refine object localization by leveraging vegetation spectral features. For point prompts, similarity maps with a max distance criterion are integrated to improve spatial coherence in sparse annotations, addressing the ambiguity of generic point prompts in agricultural contexts.
Experimental validation on two test datasets shows that our enhanced box and point prompts outperform SAM's everything mode and Grounded SAM in zero-shot segmentation tasks. Compared to the supervised method YOLOv11, our framework demonstrates superior zero-shot generalization, achieving the best segmentation performance on both datasets without target-specific annotations.
This study addresses the critical issue of scarce annotated data in vertical farming by developing a zero-shot segmentation framework. The integration of domain-specific indices (NCGI) and prompt optimization techniques provides an effective solution for plant phenotyping, highlighting the potential of weakly supervised models in agricultural computer vision where extensive manual annotation is impractical.
图像实例分割对于垂直农场中的植物表型分析至关重要，然而植物类型的多样性和有限的标注图像数据限制了传统监督技术的性能。这些挑战需要一种零样本方法，以便在不依赖每种植物类型的特定训练数据的情况下进行分割。
我们提出了一个结合Grounding DINO和分割一切模型（SAM）的零样本实例分割框架。为了增强框提示，采用结合归一化覆盖绿色指数（NCGI）的植被覆盖感知非极大值抑制（VC-NMS），通过利用植被光谱特征来优化目标定位。对于点提示，集成了具有最大距离准则的相似性图，以提高稀疏标注中的空间一致性，解决农业环境中通用点提示的模糊性问题。
在两个测试数据集上的实验验证表明，我们增强的框提示和点提示在零样本分割任务中优于SAM的全模式和有基础的SAM。与监督方法YOLOv11相比，我们的框架展示了卓越的零样本泛化能力，在没有特定目标标注的情况下，在两个数据集上均实现了最佳分割性能。
本研究通过开发一个零样本分割框架，解决了垂直农业中注释数据稀缺的关键问题。特定领域指数（NCGI）和提示优化技术的整合为植物表型分析提供了有效的解决方案，突出了弱监督模型在农业计算机视觉中的潜力，在这种情况下，广泛的人工标注是不切实际的。
