Kang Rui, Huang Jiaxin, Zhou Xuehai, Ren Ni, Sun Shangpeng
Institute of Agricultural Information, Jiangsu Academy of Agricultural Sciences, Nanjing 210044, China.
Bioresource Engineering Department, McGill University, Montreal, QC H9X 3V9, Canada.
Plant Phenomics. 2024 Apr 15;6:0174. doi: 10.34133/plantphenomics.0174. eCollection 2024.
The deployment of intelligent surveillance systems to monitor tomato plant growth poses substantial challenges due to the dynamic nature of disease patterns and the complexity of environmental conditions such as background and lighting. In this study, an integrated cascade framework that synergizes detectors and trackers was introduced for the simultaneous identification of tomato leaf diseases and fruit counting. We applied an autonomous robot with smartphone camera to collect images for leaf disease and fruits in greenhouses. Further, we improved the deep learning network YOLO-TGI by incorporating Ghost and CBAM modules, which was trained and tested in conjunction with premier lightweight detection models like YOLOX and NanoDet in evaluating leaf health conditions. For the cascading with various base detectors, we integrated state-of-the-art trackers such as Byte-Track, Motpy, and FairMot to enable fruit counting in video streams. Experimental results indicated that the combination of YOLO-TGI and Byte-Track achieved the most robust performance. Particularly, YOLO-TGI-N emerged as the model with the least computational demands, registering the lowest FLOPs at 2.05 G and checkpoint weights at 3.7 M, while still maintaining a mAP of 0.72 for leaf disease detection. Regarding the fruit counting, the combination of YOLO-TGI-S and Byte-Track achieved the best of 0.93 and the lowest RMSE of 9.17, boasting an inference speed that doubles that of the YOLOX series, and is 2.5 times faster than the NanoDet series. The developed network framework is a potential solution for researchers facilitating the deployment of similar surveillance models for a broad spectrum of fruit and vegetable crops.
由于疾病模式的动态性以及诸如背景和光照等环境条件的复杂性，部署智能监测系统来监控番茄植株生长面临着重大挑战。在本研究中，引入了一种将探测器和跟踪器协同工作的集成级联框架，用于同时识别番茄叶部病害和果实计数。我们应用配备智能手机摄像头的自主机器人在温室中采集叶片病害和果实的图像。此外，我们通过合并Ghost和CBAM模块改进了深度学习网络YOLO-TGI，并结合YOLOX和NanoDet等一流的轻量级检测模型对其进行训练和测试，以评估叶片健康状况。为了与各种基础探测器进行级联，我们集成了诸如Byte-Track、Motpy和FairMot等先进的跟踪器，以便在视频流中进行果实计数。实验结果表明，YOLO-TGI和Byte-Track的组合实现了最稳健的性能。特别是，YOLO-TGI-N成为计算需求最少的模型，其浮点运算次数最低为2.05 G，检查点权重为3.7 M，同时在叶部病害检测方面仍保持0.72的平均精度均值。在果实计数方面，YOLO-TGI-S和Byte-Track的组合实现了最佳召回率0.93和最低均方根误差9.17，其推理速度是YOLOX系列的两倍，比NanoDet系列快2.5倍。所开发的网络框架为研究人员提供了一种潜在的解决方案，有助于为广泛的水果和蔬菜作物部署类似的监测模型。
