Zhou Jun, Bell Dane, Nusrat Sabrina, Hingle Melanie, Surdeanu Mihai, Kobourov Stephen
Department of Computer Science, Columbia University, New York, NY, United States.
Department of Linguistics, University of Arizona, Tucson, AZ, United States.
Interact J Med Res. 2018 Nov 5;7(2):e17. doi: 10.2196/ijmr.9359.
Software designed to accurately estimate food calories from still images could help users and health professionals identify dietary patterns and food choices associated with health and health risks more effectively. However, calorie estimation from images is difficult, and no publicly available software can do so accurately while minimizing the burden associated with data collection and analysis.
The aim of this study was to determine the accuracy of crowdsourced annotations of calorie content in food images and to identify and quantify sources of bias and noise as a function of respondent characteristics and food qualities (eg, energy density).
We invited adult social media users to provide calorie estimates for 20 food images (for which ground truth calorie data were known) using a custom-built webpage that administers an online quiz. The images were selected to provide a range of food types and energy density. Participants optionally provided age range, gender, and their height and weight. In addition, 5 nutrition experts provided annotations for the same data to form a basis of comparison. We examined estimated accuracy on the basis of expertise, demographic data, and food qualities using linear mixed-effects models with participant and image index as random variables. We also analyzed the advantage of aggregating nonexpert estimates.
A total of 2028 respondents agreed to participate in the study (males: 770/2028, 37.97%, mean body mass index: 27.5 kg/m). Average accuracy was 5 out of 20 correct guesses, where "correct" was defined as a number within 20% of the ground truth. Even a small crowd of 10 individuals achieved an accuracy of 7, exceeding the average individual and expert annotator's accuracy of 5. Women were more accurate than men (P<.001), and younger people were more accurate than older people (P<.001). The calorie content of energy-dense foods was overestimated (P=.02). Participants performed worse when images contained reference objects, such as credit cards, for scale (P=.01).
Our findings provide new information about how calories are estimated from food images, which can inform the design of related software and analyses.
旨在根据静止图像准确估算食物热量的软件，可帮助用户和健康专业人员更有效地识别与健康及健康风险相关的饮食模式和食物选择。然而，从图像中估算热量很困难，且没有公开可用的软件能在尽量减少与数据收集和分析相关负担的同时准确做到这一点。
本研究的目的是确定众包标注食物图像热量含量的准确性，并识别和量化作为受访者特征和食物质量（如能量密度）函数的偏差和噪声来源。
我们邀请成年社交媒体用户使用管理在线测验的定制网页，为20张食物图像（已知其真实热量数据）提供热量估算。选择这些图像以提供一系列食物类型和能量密度。参与者可选择提供年龄范围、性别以及身高和体重。此外，5名营养专家为相同数据提供标注以形成比较基础。我们使用以参与者和图像索引为随机变量的线性混合效应模型，根据专业知识、人口统计学数据和食物质量检查估算准确性。我们还分析了汇总非专业估算的优势。
共有2028名受访者同意参与研究（男性：770/2028，37.97%，平均体重指数：27.5kg/m²）。平均准确率为20次猜测中有5次正确，其中“正确”定义为在真实值的20%范围内的数字。即使是一小群10个人也达到了7的准确率，超过了个体和专家标注者的平均准确率5。女性比男性更准确（P<.001），年轻人比年长者更准确（P<.001）。能量密集型食物的热量含量被高估（P=.02）。当图像包含用于比例参考的物体（如信用卡）时，参与者表现更差（P=.01）。
我们的研究结果提供了有关如何从食物图像中估算热量的新信息，这可为相关软件的设计和分析提供参考。
