Department of Food Science and Technology, University of the Peloponnese, Kalamata, Greece.
Laboratory of Food Quality Control and Hygiene, Department of Food Science and Human Nutrition, Agricultural University of Athens, Athens, Greece.
Adv Nutr. 2022 Dec 22;13(6):2590-2619. doi: 10.1093/advances/nmac078.
Dietary assessment can be crucial for the overall well-being of humans and, at least in some instances, for the prevention and management of chronic, life-threatening diseases. Recall and manual record-keeping methods for food-intake monitoring are available, but often inaccurate when applied for a long period of time. On the other hand, automatic record-keeping approaches that adopt mobile cameras and computer vision methods seem to simplify the process and can improve current human-centric diet-monitoring methods. Here we present an extended critical literature overview of image-based food-recognition systems (IBFRS) combining a camera of the user's mobile device with computer vision methods and publicly available food datasets (PAFDs). In brief, such systems consist of several phases, such as the segmentation of the food items on the plate, the classification of the food items in a specific food category, and the estimation phase of volume, calories, or nutrients of each food item. A total of 159 studies were screened in this systematic review of IBFRS. A detailed overview of the methods adopted in each of the 78 included studies of this systematic review of IBFRS is provided along with their performance on PAFDs. Studies that included IBFRS without presenting their performance in at least 1 of the above-mentioned phases were excluded. Among the included studies, 45 (58%) studies adopted deep learning methods and especially convolutional neural networks (CNNs) in at least 1 phase of the IBFRS with input PAFDs. Among the implemented techniques, CNNs outperform all other approaches on the PAFDs with a large volume of data, since the richness of these datasets provides adequate training resources for such algorithms. We also present evidence for the benefits of application of IBFRS in professional dietetic practice. Furthermore, challenges related to the IBFRS presented here are also thoroughly discussed along with future directions.
饮食评估对于人类的整体健康至关重要，至少在某些情况下，对于预防和管理慢性、危及生命的疾病也是如此。目前已有用于监测食物摄入量的回忆和手动记录方法，但在长时间应用时往往不够准确。另一方面，采用移动摄像头和计算机视觉方法的自动记录方法似乎简化了流程，并可以改进当前以人为中心的饮食监测方法。在这里，我们结合用户移动设备的摄像头和计算机视觉方法以及公开可用的食物数据集（PAFDs），对基于图像的食物识别系统（IBFRS）进行了扩展的批判性文献综述。简而言之，此类系统由几个阶段组成，例如在盘子上分割食物、对特定食物类别中的食物进行分类以及估计每种食物的体积、卡路里或营养成分。在这项基于图像的食物识别系统综述中，共筛选出 159 项研究。详细概述了该系统综述中包含的 78 项研究中采用的方法，以及它们在 PAFD 上的性能。在这项基于图像的食物识别系统综述中，未在上述至少一个阶段展示其性能的研究被排除在外。在包括的研究中，45 项（58%）研究在 IBFRS 的至少一个阶段中采用了深度学习方法，尤其是卷积神经网络（CNNs），并使用 PAFD 作为输入。在所实施的技术中，CNN 在具有大量数据的 PAFD 上优于所有其他方法，因为这些数据集的丰富性为这些算法提供了足够的训练资源。我们还提供了在专业饮食实践中应用 IBFRS 的好处的证据。此外，还彻底讨论了与 IBFRS 相关的挑战以及未来的发展方向。
