Institute of Vegetables, Key Laboratory of Vegetable Legumes Germplasm Enhancement and Southern China of the Ministry of Agriculture and Rural Affairs, Zhejiang Academy of Agricultural Sciences, Hangzhou, China.
Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou, China.
Sci Rep. 2024 Jan 31;14(1):2568. doi: 10.1038/s41598-024-51668-6.
The freshness of vegetable soybean (VS) is an important indicator for quality evaluation. Currently, deep learning-based image recognition technology provides a fast, efficient, and low-cost method for analyzing the freshness of food. The RGB (red, green, and blue) image recognition technology is widely used in the study of food appearance evaluation. In addition, the hyperspectral image has outstanding performance in predicting the nutrient content of samples. However, there are few reports on the research of classification models based on the fusion data of these two sources of images. We collected RGB and hyperspectral images at four different storage times of VS. The ENVI software was adopted to extract the hyperspectral information, and the RGB images were reconstructed based on the downsampling technology. Then, the one-dimensional hyperspectral data was transformed into a two-dimensional space, which allows it to be overlaid and concatenated with the RGB image data in the channel direction, thereby generating fused data. Compared with four commonly used machine learning models, the deep learning model ResNet18 has higher classification accuracy and computational efficiency. Based on the above results, a novel classification model named ResNet-R &H, which is based on the residual networks (ResNet) structure and incorporates the fusion data of RGB and hyperspectral images, was proposed. The ResNet-R &H can achieve a testing accuracy of 97.6%, which demonstrates a significant enhancement of 4.0% and 7.2% compared to the distinct utilization of hyperspectral data and RGB data, respectively. Overall, this research is significant in providing a unique, efficient, and more accurate classification approach in evaluating the freshness of vegetable soybean. The method proposed in this study can provide a theoretical reference for classifying the freshness of fruits and vegetables to improve classification accuracy and reduce human error and variability.
蔬菜大豆（VS）的新鲜度是评价其质量的一个重要指标。目前，基于深度学习的图像识别技术为分析食品新鲜度提供了一种快速、高效、低成本的方法。RGB（红、绿、蓝）图像识别技术广泛应用于食品外观评价研究中。此外，高光谱图像在预测样品营养含量方面表现出色。然而，基于这两种来源的图像融合数据的分类模型研究报告较少。我们采集了 VS 在四个不同储存时间的 RGB 和高光谱图像。采用 ENVI 软件提取高光谱信息，并采用下采样技术对 RGB 图像进行重建。然后，将一维高光谱数据转换到二维空间，使其能够在通道方向上与 RGB 图像数据进行叠加和连接，从而生成融合数据。与四种常用的机器学习模型相比，深度学习模型 ResNet18 具有更高的分类准确率和计算效率。基于以上结果，提出了一种名为 ResNet-R &H 的新型分类模型，该模型基于残差网络（ResNet）结构，并融合了 RGB 和高光谱图像的融合数据。ResNet-R &H 的测试准确率达到 97.6%，与单独使用高光谱数据和 RGB 数据相比，分别提高了 4.0%和 7.2%。总的来说，这项研究为评估蔬菜大豆的新鲜度提供了一种独特、高效且更准确的分类方法。本研究提出的方法可以为水果和蔬菜的新鲜度分类提供理论参考，以提高分类精度并减少人为误差和变异性。
