Jia Weikuan, Wei Jinmeng, Zhang Qi, Pan Ningning, Niu Yi, Yin Xiang, Ding Yanhui, Ge Xinting
School of Information Science and Engineering, Shandong Normal University, Jinan, China.
Key Laboratory of Intelligent Control and Manufacturing of Agricultural Machinery, Wuyi University, Wuyishan, China.
Front Plant Sci. 2022 Aug 10;13:955256. doi: 10.3389/fpls.2022.955256. eCollection 2022.
Fruit and vegetable picking robots are affected by the complex orchard environment, resulting in poor recognition and segmentation of target fruits by the vision system. The orchard environment is complex and changeable. For example, the change of light intensity will lead to the unclear surface characteristics of the target fruit; the target fruits are easy to overlap with each other and blocked by branches and leaves, which makes the shape of the fruits incomplete and difficult to accurately identify and segment one by one. Aiming at various difficulties in complex orchard environment, a two-stage instance segmentation method based on the optimized mask region convolutional neural network (mask RCNN) was proposed. The new model proposed to apply the lightweight backbone network MobileNetv3, which not only speeds up the model but also greatly improves the accuracy of the model and meets the storage resource requirements of the mobile robot. To further improve the segmentation quality of the model, the boundary patch refinement (BPR) post-processing module is added to the new model to optimize the rough mask boundaries of the model output to reduce the error pixels. The new model has a high-precision recognition rate and an efficient segmentation strategy, which improves the robustness and stability of the model. This study validates the effect of the new model using the persimmon dataset. The optimized mask RCNN achieved mean average precision (mAP) and mean average recall (mAR) of 76.3 and 81.1%, respectively, which are 3.1 and 3.7% improvement over the baseline mask RCNN, respectively. The new model is experimentally proven to bring higher accuracy and segmentation quality and can be widely deployed in smart agriculture.
水果和蔬菜采摘机器人受到复杂果园环境的影响，导致视觉系统对目标水果的识别和分割效果不佳。果园环境复杂多变。例如，光照强度的变化会导致目标水果表面特征不清晰；目标水果容易相互重叠，并被枝叶遮挡，使得水果形状不完整，难以准确逐一识别和分割。针对复杂果园环境中的各种难题，提出了一种基于优化的掩膜区域卷积神经网络（Mask RCNN）的两阶段实例分割方法。新模型提议应用轻量级骨干网络MobileNetv3，这不仅加快了模型速度，还大大提高了模型的准确率，同时满足了移动机器人的存储资源需求。为了进一步提高模型的分割质量，在新模型中添加了边界补丁细化（BPR）后处理模块，以优化模型输出的粗糙掩膜边界，减少错误像素。新模型具有高精度识别率和高效的分割策略，提高了模型的鲁棒性和稳定性。本研究使用柿子数据集验证了新模型的效果。优化后的Mask RCNN分别实现了76.3%和81.1%的平均精度均值（mAP）和平均召回率均值（mAR），分别比基线Mask RCNN提高了3.1%和3.7%。实验证明，新模型能带来更高的准确率和分割质量，可广泛应用于智慧农业。
