A Review of Image-Based Food Recognition and Volume Estimation Artificial Intelligence Systems.

The daily healthy diet and balanced intake of essential nutrients play an important role in modern lifestyle. The estimation of a meal's nutrient content is an integral component of significant diseases, such as diabetes, obesity and cardiovascular disease. Lately, there has been an increasing interest towards the development and utilization of smartphone applications with the aim of promoting healthy behaviours. The semi - automatic or automatic, precise and in real-time estimation of the nutrients of daily consumed meals is approached in relevant literature as a computer vision problem using food images which are taken via a user's smartphone. Herein, we present the state-of-the-art on automatic food recognition and food volume estimation methods starting from their basis, i.e., the food image databases. First, by methodically organizing the extracted information from the reviewed studies, this review study enables the comprehensive fair assessment of the methods and techniques applied for segmenting food images, classifying their food content and computing the food volume, associating their results with the characteristics of the used datasets. Second, by unbiasedly reporting the strengths and limitations of these methods and proposing pragmatic solutions to the latter, this review can inspire future directions in the field of dietary assessment systems.

Weight Estimation of Mediterranean Food Images using Random Forest Regression Algorithm.

The daily nutrition management is one of the most important issues that concern individuals in the modern lifestyle. Over the years, the development of dietary assessment systems and applications based on food images has assisted experts to manage people's nutritional facts and eating habits. In these systems, the food volume estimation is the most important task for calculating food quantity and nutritional information. In this study, we present a novel methodology for food weight estimation based on a food image, using the Random Forest regression algorithm. The weight estimation model was trained on a unique dataset of 5,177 annotated Mediterranean food images, consisting of 50 different foods with a reference card placed next to the plate. Then, we created a data frame of 6,425 records from the annotated food images with features such as: food area, reference object area, food id, food category and food weight. Finally, using the Random Forest regression algorithm and applying nested cross validation and hyperparameters tuning, we trained the weight estimation model. The proposed model achieves 22.6 grams average difference between predicted and real weight values for each food item record in the data frame and 15.1% mean absolute percentage error for each food item, opening new perspectives in food image-based volume and nutrition estimation models and systems.Clinical Relevance- The proposed methodology is suitable for healthcare systems and applications that monitor an individual's malnutrition, offering the ability to estimate the energy and nutrients consumed using an image of the meal.

A novel approach to estimate the weight of food items based on features extracted from an image using boosting algorithms.

Managing daily nutrition is a prominent concern among individuals in contemporary society. The advancement of dietary assessment systems and applications utilizing images has facilitated the effective management of individuals' nutritional information and dietary habits over time. The determination of food weight or volume is a vital part in these systems for assessing food quantities and nutritional information. This study presents a novel methodology for evaluating the weight of food by utilizing extracted features from images and training them through advanced boosting regression algorithms. Α unique dataset of 23,052 annotated food images of Mediterranean cuisine, including 226 different dishes with a reference object placed next to the dish, was used to train the proposed pipeline. Then, using extracted features from the annotated images, such as food area, reference object area, food id, food category, and food weight, we built a dataframe with 24,996 records. Finally, we trained the weight estimation model by applying cross validation, hyperparameter tuning, and boosting regression algorithms such as XGBoost, CatBoost, and LightGBM. Between the predicted and actual weight values for each food in the proposed dataset, the proposed model achieves a mean weight absolute error 3.93 g, a mean absolute percentage error 3.73% and a root mean square error 6.05 g for the 226 food items of the Mediterranean Greek Food database (MedGRFood), setting new perspectives in food image-based weight and nutrition estimate models and systems.

An Automated Image-Based Dietary Assessment System for Mediterranean Foods.

Goal: The modern way of living has significantly influenced the daily diet. The ever-increasing number of people with obesity, diabetes and cardiovascular diseases stresses the need to find tools that could help in the daily intake of the necessary nutrients. Methods: In this paper, we present an automated image-based dietary assessment system of Mediterranean food, based on: 1) an image dataset of Mediterranean foods, 2) on a pre-trained Convolutional Neural Network (CNN) for food image classification, and 3) on stereo vision techniques for the volume and nutrition estimation of the food. We use a pre-trained CNN in the Food-101 dataset to train a deep learning classification model employing our dataset Mediterranean Greek Food (MedGRFood). Based on the EfficientNet family of CNNs, we use the EfficientNetB2 both for the pre-trained model and its weights evaluation, as well as for classifying food images in the MedGRFood dataset. Next, we estimate the volume of the food, through 3D food reconstruction of two images taken by a smartphone camera. The proposed volume estimation subsystem uses stereo vision techniques and algorithms, and needs the input of two food images to reconstruct the point cloud of the food and to compute its quantity. Results: The classification accuracy where true class matches with the most probable class predicted by the model (Top-1 accuracy) is 83.8%, while the accuracy where true class matches with any one of the 5 most probable classes predicted by the model (Top-5 accuracy) is 97.6%, for the food classification subsystem. The food volume estimation subsystem achieves an overall mean absolute percentage error 10.5% for 148 different food dishes. Conclusions: The proposed automated image-based dietary assessment system provides the capability of continuous recording of health data in real time.

GlucoseML Mobile Application for Automated Dietary Assessment of Mediterranean Food.

Over the years and with the help of technology, the daily care of type 1 diabetes has been improved significantly. The increased adoption of continuous glucose monitoring, the continuous subcutaneous insulin injection and the accurate behavioral monitoring mHealth solutions have contributed to this phenomenon. In this study we present a mobile application for automated dietary assessment of Mediterranean food images as part of the GlucoseML system. Based on short-term predictive analysis of the glucose trajectory, GlucoseML is a type-1 diabetes self-management system. A computer vision approach is used as main part of the GlucoseML dietary assessment system calculating food carbohydrates, fats and proteins, relying on: (i) a deep learning subsystem for food image classification, and (ii) a 3D food image reconstruction subsystem for the volume estimation of food. The deep learning subsystem achieves 82.4% and 97.5% top-1 and top-5 accuracy, respectively, for food image classification while the subsystem for volume estimation of food achieves a mean absolute percentage error 10.7% for the four main categories of MedGRFood dataset.

Mediterranean Food Image Recognition Using Deep Convolutional Networks.

We present a new dataset of food images that can be used to evaluate food recognition systems and dietary assessment systems. The Mediterranean Greek food -MedGRFood dataset consists of food images from the Mediterranean cuisine, and mainly from the Greek cuisine. The dataset contains 42,880 food images belonging to 132 food classes which have been collected from the web. Based on the EfficientNet family of convolutional neural networks, specifically the EfficientNetB2, we propose a new deep learning schema that achieves 83.4% top-1 accuracy and 97.8% top-5 accuracy in the MedGRFood dataset for food recognition. This schema includes the use of the fine tuning, transfer learning and data augmentation technique.