Xin Bolai, Sun Ji, Bartholomeus Harm, Kootstra Gert
Department of Plant Science, Wageningen University and Research, Wageningen, Netherlands.
Laboratory of Geo-Information Science and Remote Sensing, Wageningen University and Research, Wageningen, Netherlands.
Front Plant Sci. 2023 Jun 12;14:1045545. doi: 10.3389/fpls.2023.1045545. eCollection 2023.
3D semantic segmentation of plant point clouds is an important step towards automatic plant phenotyping and crop modeling. Since traditional hand-designed methods for point-cloud processing face challenges in generalisation, current methods are based on deep neural network that learn to perform the 3D segmentation based on training data. However, these methods require a large annotated training set to perform well. Especially for 3D semantic segmentation, the collection of training data is highly labour intensitive and time consuming. Data augmentation has been shown to improve training on small training sets. However, it is unclear which data-augmentation methods are effective for 3D plant-part segmentation.
In the proposed work, five novel data-augmentation methods (global cropping, brightness adjustment, leaf translation, leaf rotation, and leaf crossover) were proposed and compared to five existing methods (online down sampling, global jittering, global scaling, global rotation, and global translation). The methods were applied to PointNet++ for 3D semantic segmentation of the point clouds of three cultivars of tomato plants (Merlice, Brioso, and Gardener Delight). The point clouds were segmented into soil base, stick, stemwork, and other bio-structures.
Among the data augmentation methods being proposed in this paper, leaf crossover indicated the most promising result which outperformed the existing ones. Leaf rotation (around Z axis), leaf translation, and cropping also performed well on the 3D tomato plant point clouds, which outperformed most of the existing work apart from global jittering. The proposed 3D data augmentation approaches significantly improve the overfitting caused by the limited training data. The improved plant-part segmentation further enables a more accurate reconstruction of the plant architecture.
植物点云的三维语义分割是迈向自动植物表型分析和作物建模的重要一步。由于传统的手工设计的点云处理方法在泛化方面面临挑战，当前的方法基于深度神经网络，该网络通过训练数据学习执行三维分割。然而，这些方法需要大量带注释的训练集才能表现良好。特别是对于三维语义分割，训练数据的收集非常耗费人力且耗时。数据增强已被证明可以改善小训练集上的训练效果。然而，尚不清楚哪些数据增强方法对三维植物部分分割有效。
在本研究中，提出了五种新颖的数据增强方法（全局裁剪、亮度调整、叶片平移、叶片旋转和叶片交叉），并与五种现有方法（在线下采样、全局抖动、全局缩放、全局旋转和全局平移）进行了比较。这些方法应用于PointNet++，用于对三个番茄品种（Merlice、Brioso和Gardener Delight）的点云进行三维语义分割。点云被分割为土壤基部、茎杆、茎干和其他生物结构。
在本文提出的数据增强方法中，叶片交叉显示出最有前景的结果，优于现有方法。叶片旋转（绕Z轴）、叶片平移和裁剪在三维番茄植物点云上也表现良好，除了全局抖动外，优于大多数现有工作。所提出的三维数据增强方法显著改善了由有限训练数据导致的过拟合。改进的植物部分分割进一步实现了对植物结构更准确的重建。
