Wang Qiang, Fan Xijian, Zhuang Ziqing, Tjahjadi Tardi, Jin Shichao, Huan Honghua, Ye Qiaolin
Nanjing Forestry University, Nanjing 210037, China.
University of Warwick, Coventry CV4 7AL, UK.
Plant Phenomics. 2024 Nov 28;6:0271. doi: 10.34133/plantphenomics.0271. eCollection 2024.
Accurate counting of cereals crops, e.g., maize, rice, sorghum, and wheat, is crucial for estimating grain production and ensuring food security. However, existing methods for counting cereal crops focus predominantly on building models for specific crop head; thus, they lack generalizability to different crop varieties. This paper presents Counting Heads of Cereal Crops Net (CHCNet), which is a unified model designed for counting multiple cereal crop heads by few-shot learning, which effectively reduces labeling costs. Specifically, a refined vision encoder is developed to enhance feature embedding, where a foundation model, namely, the segment anything model (SAM), is employed to emphasize the marked crop heads while mitigating complex background effects. Furthermore, a multiscale feature interaction module is proposed for integrating a similarity metric to facilitate automatic learning of crop-specific features across varying scales, which enhances the ability to describe crop heads of various sizes and shapes. The CHCNet model adopts a 2-stage training procedure. The initial stage focuses on latent feature mining to capture common feature representations of cereal crops. In the subsequent stage, inference is performed without additional training, by extracting domain-specific features of the target crop from selected exemplars to accomplish the counting task. In extensive experiments on 6 diverse crop datasets captured from ground cameras and drones, CHCNet substantially outperformed state-of-the-art counting methods in terms of cross-crop generalization ability, achieving mean absolute errors (MAEs) of 9.96 and 9.38 for maize, 13.94 for sorghum, 7.94 for rice, and 15.62 for mixed crops. A user-friendly interactive demo is available at http://cerealcropnet.com/, where researchers are invited to personally evaluate the proposed CHCNet. The source code for implementing CHCNet is available at https://github.com/Small-flyguy/CHCNet.
准确计算谷物作物，如玉米、水稻、高粱和小麦的数量，对于估计粮食产量和确保粮食安全至关重要。然而，现有的谷物作物计数方法主要集中在为特定作物穗构建模型；因此，它们缺乏对不同作物品种的通用性。本文提出了谷物作物穗计数网络（CHCNet），这是一个通过少样本学习设计的用于计数多种谷物作物穗的统一模型，有效降低了标注成本。具体而言，开发了一种改进的视觉编码器来增强特征嵌入，其中使用了一个基础模型，即分割一切模型（SAM），以突出标记的作物穗，同时减轻复杂背景的影响。此外，还提出了一个多尺度特征交互模块，用于整合相似性度量，以促进跨不同尺度自动学习作物特定特征，增强了描述各种大小和形状作物穗的能力。CHCNet模型采用两阶段训练过程。初始阶段专注于潜在特征挖掘，以捕获谷物作物的共同特征表示。在随后的阶段，无需额外训练即可进行推理，通过从选定的示例中提取目标作物的特定领域特征来完成计数任务。在从地面相机和无人机捕获的6个不同作物数据集上进行的广泛实验中，CHCNet在跨作物泛化能力方面显著优于现有最先进的计数方法，玉米的平均绝对误差（MAE）为9.96和9.38，高粱为13.94，水稻为7.94，混合作物为15.62。可在http://cerealcropnet.com/上获得一个用户友好的交互式演示，邀请研究人员亲自评估所提出的CHCNet。实现CHCNet的源代码可在https://github.com/Small-flyguy/CHCNet上获取。
