Tang Can, Chen Du, Wang Xin, Ni Xindong, Liu Yehong, Liu Yihao, Mao Xu, Wang Shumao
College of Engineering, China Agricultural University, Beijing, China.
State Key Laboratory of Intelligent Agricultural Power Equipment, Henan, China.
Front Plant Sci. 2023 Jul 28;14:1211830. doi: 10.3389/fpls.2023.1211830. eCollection 2023.
As a fruit with high economic value, strawberry has a short ripeness period, and harvesting at an incorrect time will seriously affect the quality of strawberries, thereby reducing economic benefits. Therefore, the timing of its harvesting is very demanding. A fine ripeness recognition can provide more accurate crop information, and guide strawberry harvest management more timely and effectively. This study proposes a fine recognition method for field strawberry ripeness that combines deep learning and image processing. The method is divided into three stages: In the first stage, self-calibrated convolutions are added to the Mask R-CNN backbone network to improve the model performance, and then the model is used to extract the strawberry target in the image. In the second stage, the strawberry target is divided into four sub-regions by region segmentation method, and the color feature values of B, G, L, a and S channels are extracted for each sub-region. In the third stage, the strawberry ripeness is classified according to the color feature values and the results are visualized. Experimental results show that with the incorporation of self-calibrated convolutions into the Mask R-CNN, the model's performance has been substantially enhanced, leading to increased robustness against diverse occlusion interferences. As a result, the final average precision (AP) has improved to 0.937, representing a significant increase of 0.039 compared to the previous version. The strawberry ripeness classification effect is the best on the SVM classifier, and the accuracy under the combined channel BGLaS reaches 0.866. The classification results are better than common manual feature extraction methods and AlexNet, ResNet18 models. In order to clarify the role of the region segmentation method, the contribution of different sub-regions to each ripeness is also explored. The comprehensive results demonstrate that the proposed method enables the evaluation of six distinct ripeness levels of strawberries in the complex field environment. This method can provide accurate decision support for strawberry refined planting management.
草莓作为一种具有较高经济价值的水果，成熟周期短，采摘时机不当会严重影响草莓品质，进而降低经济效益。因此，其采摘时机要求非常苛刻。精准的成熟度识别能够提供更准确的作物信息，更及时有效地指导草莓收获管理。本研究提出一种结合深度学习与图像处理的田间草莓成熟度精准识别方法。该方法分为三个阶段：第一阶段，在Mask R-CNN骨干网络中添加自校准卷积以提升模型性能，然后用该模型提取图像中的草莓目标；第二阶段，采用区域分割方法将草莓目标划分为四个子区域，为每个子区域提取B、G、L、a和S通道的颜色特征值；第三阶段，根据颜色特征值对草莓成熟度进行分类并将结果可视化。实验结果表明，将自校准卷积纳入Mask R-CNN后，模型性能大幅提升，对各种遮挡干扰的鲁棒性增强。最终平均精度（AP）提高到0.937，相比之前版本显著提高了0.039。草莓成熟度分类效果在支持向量机（SVM）分类器上最佳，组合通道BGLaS下的准确率达到0.866。分类结果优于常见的手工特征提取方法以及AlexNet、ResNet18模型。为阐明区域分割方法的作用，还探究了不同子区域对各成熟度的贡献。综合结果表明，该方法能够在复杂田间环境中评估草莓的六个不同成熟度等级。此方法可为草莓精细化种植管理提供准确的决策支持。
