Faculty of Engineering and Applied Science, Ontario Tech University, Oshawa, ON L1H 7K4, Canada.
Sensors (Basel). 2021 Oct 7;21(19):6657. doi: 10.3390/s21196657.
We present an end-to-end smart harvesting solution for precision agriculture. Our proposed pipeline begins with yield estimation that is done through the use of object detection and tracking to count fruit within a video. We use and train You Only Look Once model (YOLO) on video clips of apples, oranges and pumpkins. The bounding boxes obtained through objection detection are used as an input to our selected tracking model, DeepSORT. The original version of DeepSORT is unusable with fruit data, as the appearance feature extractor only works with people. We implement ResNet as DeepSORT's new feature extractor, which is lightweight, accurate and generically works on different fruits. Our yield estimation module shows accuracy between 91-95% on real footage of apple trees. Our modification successfully works for counting oranges and pumpkins, with an accuracy of 79% and 93.9% with no need for training. Our framework additionally includes a visualization of the yield. This is done through the incorporation of geospatial data. We also propose a mechanism to annotate a set of frames with a respective GPS coordinate. During counting, the count within the set of frames and the matching GPS coordinate are recorded, which we then visualize on a map. We leverage this information to propose an optimal container placement solution. Our proposed solution involves minimizing the number of containers to place across the field before harvest, based on a set of constraints. This acts as a decision support system for the farmer to make efficient plans for logistics, such as labor, equipment and gathering paths before harvest. Our work serves as a blueprint for future agriculture decision support systems that can aid in many other aspects of farming.
我们提出了一个端到端的智能收割解决方案，用于精准农业。我们的流水线从产量估计开始，通过使用目标检测和跟踪来计算视频中的水果数量。我们使用并在苹果、橙子和南瓜的视频剪辑上训练了 You Only Look Once 模型（YOLO）。通过目标检测获得的边界框被用作我们选择的跟踪模型 DeepSORT 的输入。原始版本的 DeepSORT 无法用于水果数据，因为外观特征提取器仅适用于人。我们实现了 ResNet 作为 DeepSORT 的新特征提取器，它重量轻、准确，并且可以在不同的水果上通用。我们的产量估计模块在真实的苹果树视频中显示出 91-95%的准确率。我们的修改成功地用于计数橙子和南瓜，准确率分别为 79%和 93.9%，无需训练。我们的框架还包括产量的可视化。这是通过合并地理空间数据来实现的。我们还提出了一种用相应的 GPS 坐标注释一组帧的机制。在计数过程中，会记录一组帧内的计数和匹配的 GPS 坐标，然后我们将其在地图上可视化。我们利用这些信息提出了一种最优的容器放置解决方案。我们提出的解决方案涉及在收获前根据一组约束在整个田地里最小化要放置的容器数量。这可以作为农民的决策支持系统，帮助他们在收获前制定有效的物流计划，例如劳动力、设备和收集路径。我们的工作为未来的农业决策支持系统提供了蓝图，可以在农业的许多其他方面提供帮助。
