Graduate School of Engineering, Kochi University of Technology, Kami, Kochi 782-8502, Japan.
School of Information, Kochi University of Technology, Kami, Kochi 782-8502, Japan.
Sensors (Basel). 2021 Sep 27;21(19):6437. doi: 10.3390/s21196437.
There is a growing demand for developing image sensor systems to aid fruit and vegetable harvesting, and crop growth prediction in precision agriculture. In this paper, we present an end-to-end optimization approach for the simultaneous design of optical filters and green pepper segmentation neural networks. Our optimization method modeled the optical filter as one learnable neural network layer and attached it to the subsequent camera spectral response (CSR) layer and segmentation neural network for green pepper segmentation. We used not only the standard red-green-blue output from the CSR layer but also the color-ratio maps as additional cues in the visible wavelength and to augment the feature maps as the input for segmentation. We evaluated how well our proposed color-ratio maps enhanced optical filter design methods in our collected dataset. We find that our proposed method can yield a better performance than both an optical filter RGB system without color-ratio maps and a raw RGB camera (without an optical filter) system. The proposed learning-based framework can potentially build better image sensor systems for green pepper segmentation.
在精准农业中，人们对于开发图像传感器系统以辅助水果和蔬菜采摘以及作物生长预测的需求日益增长。在本文中，我们提出了一种端到端的优化方法，用于同时设计光学滤波器和用于青辣椒分割的神经网络。我们的优化方法将光学滤波器建模为一个可学习的神经网络层，并将其附加到后续的相机光谱响应 (CSR) 层和用于青辣椒分割的分割神经网络上。我们不仅使用了 CSR 层的标准红绿蓝输出，还使用了颜色比图作为可见光波段的附加线索，并将其作为分割的输入来扩充特征图。我们评估了我们提出的颜色比图在我们收集的数据集上如何增强光学滤波器设计方法。我们发现，与没有颜色比图的光学滤波器 RGB 系统和没有光学滤波器的原始 RGB 相机系统相比，我们提出的方法可以获得更好的性能。基于学习的框架可以为青辣椒分割构建更好的图像传感器系统。
