Song Guozhu, Wang Jian, Ma Rongting, Shi Yan, Wang Yaqi
College of Software, Shanxi Agricultural University, Taigu, China.
Front Plant Sci. 2024 Nov 29;15:1447855. doi: 10.3389/fpls.2024.1447855. eCollection 2024.
When harvesting bunch tomatoes, accurately identifying certain fruiting stems proves challenging due to their obstruction by branches and leaves, or their similarity in colour to the branches, main vines, and lateral vines. Additionally, irregularities in the growth pattern of the fruiting pedicels further complicate precise picking point localization, thus impacting harvesting efficiency. Moreover, the fruit stalks being too short or slender poses an obstacle, rendering it impossible for the depth camera to accurately obtain depth information during depth value acquisition. To address these challenges, this paper proposes an enhanced YOLOv8 model integrated with a depth camera for string tomato fruit stalk picking point identification and localization research. Initially, the Fasternet bottleneck in YOLOv8 is replaced with the c2f bottleneck, and the MLCA attention mechanism is added after the backbone network to construct the FastMLCA-YOLOv8 model for fruit stalk recognition. Subsequently, the optimized K-means algorithm, utilizing K-means++ for clustering centre initialization and determining the optimal number of clusters via Silhouette coefficients, is employed to segment the fruit stalk region. Following this, the corrosion operation and Zhang refinement algorithm are used to denoise the segmented fruit stalk region and extract the refined skeletal line, thereby determining the coordinate position of the fruit stalk picking point in the binarized image. Finally, the issue of missing depth values of fruit stalks is addressed by the secondary extraction method to obtain the depth values and 3D coordinate information of the picking points in RGB-D camera coordinates. The experimental results demonstrate that the algorithm accurately identifies and locates the picking points of string tomatoes under complex background conditions, with the identification success rate of the picking points reaching 91.3%. Compared with the YOLOv8 model, the accuracy is improved by 2.8%, and the error of the depth value of the picking points is only ±2.5 mm. This research meets the needs of string tomato picking robots in fruit stalk target detection and provides strong support for the development of string tomato picking technology.
在收获成串番茄时，由于某些结果茎被枝叶遮挡，或者其颜色与树枝、主蔓和侧蔓相似，准确识别它们具有挑战性。此外，结果花梗生长模式的不规则性进一步使精确采摘点定位复杂化，从而影响收获效率。此外，果柄过短或过细也构成了障碍，使得深度相机在获取深度值时无法准确获得深度信息。为应对这些挑战，本文提出了一种集成深度相机的增强型YOLOv8模型，用于成串番茄果柄采摘点的识别与定位研究。首先，将YOLOv8中的Fasternet瓶颈替换为c2f瓶颈，并在骨干网络后添加MLCA注意力机制，构建用于果柄识别的FastMLCA-YOLOv8模型。随后，采用优化的K-means算法，利用K-means++初始化聚类中心，并通过轮廓系数确定最优聚类数，对果柄区域进行分割。在此之后，使用腐蚀操作和Zhang细化算法对分割后的果柄区域进行去噪，并提取细化的骨架线，从而确定二值化图像中果柄采摘点的坐标位置。最后，通过二次提取方法解决果柄深度值缺失的问题，以获取RGB-D相机坐标下采摘点的深度值和三维坐标信息。实验结果表明，该算法能够在复杂背景条件下准确识别和定位成串番茄的采摘点，采摘点的识别成功率达到91.3%。与YOLOv8模型相比，准确率提高了2.8%，采摘点深度值的误差仅为±2.5毫米。本研究满足了成串番茄采摘机器人在果柄目标检测方面的需求，为成串番茄采摘技术发展提供了有力支持。
