Shuai Luyu, Chen Ziao, Li Zhiyong, Li Hongdan, Zhang Boda, Wang Yuchao, Mu Jiong
College of Information Engineering, Sichuan Agricultural University, Ya'an, China.
Ya'an Digital Agricultural Engineering Technology Research Center, Sichuan Agricultural University, Ya'an, China.
Front Plant Sci. 2023 Jul 18;14:1224884. doi: 10.3389/fpls.2023.1224884. eCollection 2023.
The difficulties in tea shoot recognition are that the recognition is affected by lighting conditions, it is challenging to segment images with similar backgrounds to the shoot color, and the occlusion and overlap between leaves.
To solve the problem of low accuracy of dense small object detection of tea shoots, this paper proposes a real-time dense small object detection algorithm based on multimodal optimization. First, RGB, depth, and infrared images are collected form a multimodal image set, and a complete shoot object labeling is performed. Then, the YOLOv5 model is improved and applied to dense and tiny tea shoot detection. Secondly, based on the improved YOLOv5 model, this paper designs two data layer-based multimodal image fusion methods and a feature layerbased multimodal image fusion method; meanwhile, a cross-modal fusion module (FFA) based on frequency domain and attention mechanisms is designed for the feature layer fusion method to adaptively align and focus critical regions in intra- and inter-modal channel and frequency domain dimensions. Finally, an objective-based scale matching method is developed to further improve the detection performance of small dense objects in natural environments with the assistance of transfer learning techniques.
The experimental results indicate that the improved YOLOv5 model increases the mAP50 value by 1.7% compared to the benchmark model with fewer parameters and less computational effort. Compared with the single modality, the multimodal image fusion method increases the mAP50 value in all cases, with the method introducing the FFA module obtaining the highest mAP50 value of 0.827. After the pre-training strategy is used after scale matching, the mAP values can be improved by 1% and 1.4% on the two datasets. The research idea of multimodal optimization in this paper can provide a basis and technical support for dense small object detection.
茶梢识别的困难在于识别受光照条件影响，分割与茶梢颜色背景相似的图像具有挑战性，以及叶片之间的遮挡和重叠。
为解决茶梢密集小目标检测准确率低的问题，本文提出一种基于多模态优化的实时密集小目标检测算法。首先，从多模态图像集中采集RGB、深度和红外图像，并进行完整的梢目标标注。然后，对YOLOv5模型进行改进并应用于密集微小茶梢检测。其次，基于改进的YOLOv5模型，本文设计了两种基于数据层的多模态图像融合方法和一种基于特征层的多模态图像融合方法；同时，为特征层融合方法设计了一种基于频域和注意力机制的跨模态融合模块（FFA），以在模态内和模态间的通道和频域维度上自适应地对齐和聚焦关键区域。最后，开发了一种基于目标的尺度匹配方法，借助迁移学习技术进一步提高自然环境中密集小目标的检测性能。
实验结果表明，改进后的YOLOv5模型与基准模型相比，mAP50值提高了1.7%，且参数更少、计算量更小。与单模态相比，多模态图像融合方法在所有情况下均提高了mAP50值，引入FFA模块的方法获得了最高的mAP50值0.827。在尺度匹配后使用预训练策略，两个数据集上的mAP值可分别提高1%和1.4%。本文的多模态优化研究思路可为密集小目标检测提供依据和技术支持。
