(PO14-35-22) The State of the Science on Artificial Intelligence-Based Dietary Assessment Methods That Use Digital Images: A Scoping Review

The ePoster content is available to NUTRITION 2022 LIVE ONLINE attendees who are logged in.<br><br>If you are registered, log in and then come back here to use this button.

Presenting Author(s)

Eleanor T. Shonkoff, PhD
– Assistant Clinical Professor, Merrimack College, North Andover, Massachusetts, United States

Co-Author(s)

Kelly C. Cara, MS
– Graduate Research Assistant, Tufts University, Friedman School of Nutrition Science and Policy, Springfield, Missouri, United States
XP

Xuechen (Anna) Pei
– Tufts University
MC

Mei Chung, PhD, MPH
– Associate Professor of Public Health and Community Medicine, Tufts University, School of Medicine, Boston, Massachusetts, United States
SK

Shreyas Kamath
– Tufts University
KP

Karen Panetta
– Tufts University
Erin Hennessy, PhD
– Assistant Professor, Tufts University, Boston, Massachusetts, United States

Disclosure(s):

Eleanor T. Shonkoff, PhD: No relevant financial relationship(s) with ineligible companies to disclose.

Kelly C. Cara, MS: No relevant financial relationship(s) with ineligible companies to disclose.

Objectives: Human error when estimating food intake is a major source of bias in nutrition research. This study provides an overview of literature on the accuracy of artificial intelligence (AI) methods used to analyze digital images of food compared to human coders and ground truth.

Methods: This scoping review included peer-reviewed journal articles reporting AI (e.g., deep learning) and image coding for food analysis. Literature was searched through August 2021 in 4 databases plus reference mining in conference papers and reviews. Eligible articles reported volume, energy, or nutrients estimated from digital food images via fully automatic AI methods. Two investigators independently screened and extracted data. Study characteristics, methods, and type of results were extracted.

Results: In total, 8,761 unique publications were identified; 35 papers published from 2010 to 2021 (77% after 2015) were included. Preliminary results are reported. Most papers used newly-captured images or created a digital image database (77%). Some used named image databases (e.g., Rakuten18, UEC Food-100); only ImageNet and Inselspital’s dataset were used in more than one study. Most (63%) reported relying on a nutrient database to derive nutrient values; 31% specified using USDA FoodData Central databases.

The most frequently reported estimation accuracy results were absolute error (AE; 46%), relative error (RE; 40%), correlation coefficient (26%), and error rate (23%). Other reported measures were Bland-Altman plots; actual and estimated values (e.g., volumes, nutrients); and tests of mean differences. Results were reported for calories (AE, 26% of all studies; RE 20%); macronutrients (AE 20%, RE 9%); volume, mass, weight, or area (AE 17%; RE 26%); and salt (AE 6%; RE 6%).

Conclusions: Significant resources have been devoted to investigating the ability of AI methods to conduct accurate dietary assessment using digital food images. Yet, substantial variability in the databases used and results reported prevents quantitative synthesis of overall accuracy across studies. A validated risk of bias tool is needed to compare study quality. Future research should consider using a limited number of valid databases for food images and nutrition information, and reports should at least include absolute and relative error for volume estimations.

Funding Sources: Supported by an NIH grant