Konstantakopoulos Fotios S, Georga Eleni I, Fotiadis Dimitrios I
Unit of Medical Technology and Intelligent Information Systems, Materials Science and Engineering DepartmentUniversity of Ioannina GR 45110 Ioannina Greece.
Biomedical Research InstituteFORTH, University of Ioannina GR 45110 Ioannina Greece.
IEEE Open J Eng Med Biol. 2023 Apr 13;4:45-54. doi: 10.1109/OJEMB.2023.3266135. eCollection 2023.
: The modern way of living has significantly influenced the daily diet. The ever-increasing number of people with obesity, diabetes and cardiovascular diseases stresses the need to find tools that could help in the daily intake of the necessary nutrients. In this paper, we present an automated image-based dietary assessment system of Mediterranean food, based on: 1) an image dataset of Mediterranean foods, 2) on a pre-trained Convolutional Neural Network (CNN) for food image classification, and 3) on stereo vision techniques for the volume and nutrition estimation of the food. We use a pre-trained CNN in the Food-101 dataset to train a deep learning classification model employing our dataset Mediterranean Greek Food (MedGRFood). Based on the EfficientNet family of CNNs, we use the EfficientNetB2 both for the pre-trained model and its weights evaluation, as well as for classifying food images in the MedGRFood dataset. Next, we estimate the volume of the food, through 3D food reconstruction of two images taken by a smartphone camera. The proposed volume estimation subsystem uses stereo vision techniques and algorithms, and needs the input of two food images to reconstruct the point cloud of the food and to compute its quantity. The classification accuracy where true class matches with the most probable class predicted by the model (Top-1 accuracy) is 83.8%, while the accuracy where true class matches with any one of the 5 most probable classes predicted by the model (Top-5 accuracy) is 97.6%, for the food classification subsystem. The food volume estimation subsystem achieves an overall mean absolute percentage error 10.5% for 148 different food dishes. The proposed automated image-based dietary assessment system provides the capability of continuous recording of health data in real time.
现代生活方式对日常饮食产生了重大影响。肥胖、糖尿病和心血管疾病患者人数不断增加，这凸显了寻找有助于日常摄入必需营养素的工具的必要性。在本文中，我们提出了一种基于图像的地中海食物自动饮食评估系统，该系统基于：1）地中海食物的图像数据集；2）用于食物图像分类的预训练卷积神经网络（CNN）；3）用于食物体积和营养估计的立体视觉技术。我们在Food-101数据集中使用预训练的CNN来训练一个深度学习分类模型，该模型采用我们的数据集“地中海希腊食物”（MedGRFood）。基于CNN的EfficientNet系列，我们将EfficientNetB2用于预训练模型及其权重评估，以及对MedGRFood数据集中的食物图像进行分类。接下来，我们通过智能手机摄像头拍摄的两张图像进行3D食物重建来估计食物的体积。所提出的体积估计子系统使用立体视觉技术和算法，需要输入两张食物图像来重建食物的点云并计算其数量。对于食物分类子系统，真实类别与模型预测的最可能类别匹配时的分类准确率（Top-1准确率）为83.8%，而真实类别与模型预测的5个最可能类别中的任何一个匹配时的准确率（Top-5准确率）为97.6%。对于148种不同的食物菜肴，食物体积估计子系统的总体平均绝对百分比误差为10.5%。所提出的基于图像的自动饮食评估系统提供了实时连续记录健康数据的能力。
