School of Computer Science and Statistics, Trinity College Dublin, D02 PN40 Dublin, Ireland.
School of Biosystems and Food Engineering, University College Dublin, D04 V1W8 Dublin, Ireland.
Sensors (Basel). 2023 Sep 3;23(17):7639. doi: 10.3390/s23177639.
Currently, strawberry harvesting relies heavily on human labour and subjective assessments of ripeness, resulting in inconsistent post-harvest quality. Therefore, the aim of this work is to automate this process and provide a more accurate and efficient way of assessing ripeness. We explored a unique combination of YOLOv7 object detection and augmented reality technology to detect and visualise the ripeness of strawberries. Our results showed that the proposed YOLOv7 object detection model, which employed transfer learning, fine-tuning and multi-scale training, accurately identified the level of ripeness of each strawberry with an mAP of 0.89 and an F1 score of 0.92. The tiny models have an average detection time of 18 ms per frame at a resolution of 1280 × 720 using a high-performance computer, thereby enabling real-time detection in the field. Our findings distinctly establish the superior performance of YOLOv7 when compared to other cutting-edge methodologies. We also suggest using Microsoft HoloLens 2 to overlay predicted ripeness labels onto each strawberry in the real world, providing a visual representation of the ripeness level. Despite some challenges, this work highlights the potential of augmented reality to assist farmers in harvesting support, which could have significant implications for current agricultural practices.
目前，草莓采摘主要依赖人工和主观判断成熟度，导致采后质量不一致。因此，本工作旨在实现该过程自动化，并提供更准确和高效的成熟度评估方法。我们探索了 YOLOv7 目标检测和增强现实技术的独特组合，以检测和可视化草莓的成熟度。结果表明，所提出的使用迁移学习、微调、多尺度训练的 YOLOv7 目标检测模型，能够以 0.89 的 mAP 和 0.92 的 F1 分数准确识别每个草莓的成熟度水平。使用高性能计算机，在 1280×720 的分辨率下，小模型的平均检测时间为每帧 18 毫秒，从而能够在现场实时检测。与其他前沿方法相比，我们的研究结果明显确立了 YOLOv7 的优越性能。我们还建议使用 Microsoft HoloLens 2 将预测的成熟度标签叠加在每个真实世界中的草莓上，提供成熟度水平的直观表示。尽管存在一些挑战，但这项工作突出了增强现实在支持农民收获方面的潜力，这可能对当前的农业实践产生重大影响。
