A Review of Image-Based Food Recognition and Volume Estimation Artificial Intelligence Systems.

The daily healthy diet and balanced intake of essential nutrients play an important role in modern lifestyle. The estimation of a meal's nutrient content is an integral component of significant diseases, such as diabetes, obesity and cardiovascular disease. Lately, there has been an increasing interest towards the development and utilization of smartphone applications with the aim of promoting healthy behaviours. The semi - automatic or automatic, precise and in real-time estimation of the nutrients of daily consumed meals is approached in relevant literature as a computer vision problem using food images which are taken via a user's smartphone. Herein, we present the state-of-the-art on automatic food recognition and food volume estimation methods starting from their basis, i.e., the food image databases. First, by methodically organizing the extracted information from the reviewed studies, this review study enables the comprehensive fair assessment of the methods and techniques applied for segmenting food images, classifying their food content and computing the food volume, associating their results with the characteristics of the used datasets. Second, by unbiasedly reporting the strengths and limitations of these methods and proposing pragmatic solutions to the latter, this review can inspire future directions in the field of dietary assessment systems.

Weight Estimation of Mediterranean Food Images using Random Forest Regression Algorithm.

The daily nutrition management is one of the most important issues that concern individuals in the modern lifestyle. Over the years, the development of dietary assessment systems and applications based on food images has assisted experts to manage people's nutritional facts and eating habits. In these systems, the food volume estimation is the most important task for calculating food quantity and nutritional information. In this study, we present a novel methodology for food weight estimation based on a food image, using the Random Forest regression algorithm. The weight estimation model was trained on a unique dataset of 5,177 annotated Mediterranean food images, consisting of 50 different foods with a reference card placed next to the plate. Then, we created a data frame of 6,425 records from the annotated food images with features such as: food area, reference object area, food id, food category and food weight. Finally, using the Random Forest regression algorithm and applying nested cross validation and hyperparameters tuning, we trained the weight estimation model. The proposed model achieves 22.6 grams average difference between predicted and real weight values for each food item record in the data frame and 15.1% mean absolute percentage error for each food item, opening new perspectives in food image-based volume and nutrition estimation models and systems.Clinical Relevance- The proposed methodology is suitable for healthcare systems and applications that monitor an individual's malnutrition, offering the ability to estimate the energy and nutrients consumed using an image of the meal.

A novel approach to estimate the weight of food items based on features extracted from an image using boosting algorithms.

Managing daily nutrition is a prominent concern among individuals in contemporary society. The advancement of dietary assessment systems and applications utilizing images has facilitated the effective management of individuals' nutritional information and dietary habits over time. The determination of food weight or volume is a vital part in these systems for assessing food quantities and nutritional information. This study presents a novel methodology for evaluating the weight of food by utilizing extracted features from images and training them through advanced boosting regression algorithms. Α unique dataset of 23,052 annotated food images of Mediterranean cuisine, including 226 different dishes with a reference object placed next to the dish, was used to train the proposed pipeline. Then, using extracted features from the annotated images, such as food area, reference object area, food id, food category, and food weight, we built a dataframe with 24,996 records. Finally, we trained the weight estimation model by applying cross validation, hyperparameter tuning, and boosting regression algorithms such as XGBoost, CatBoost, and LightGBM. Between the predicted and actual weight values for each food in the proposed dataset, the proposed model achieves a mean weight absolute error 3.93 g, a mean absolute percentage error 3.73% and a root mean square error 6.05 g for the 226 food items of the Mediterranean Greek Food database (MedGRFood), setting new perspectives in food image-based weight and nutrition estimate models and systems.

An Automated Image-Based Dietary Assessment System for Mediterranean Foods.

Goal: The modern way of living has significantly influenced the daily diet. The ever-increasing number of people with obesity, diabetes and cardiovascular diseases stresses the need to find tools that could help in the daily intake of the necessary nutrients. Methods: In this paper, we present an automated image-based dietary assessment system of Mediterranean food, based on: 1) an image dataset of Mediterranean foods, 2) on a pre-trained Convolutional Neural Network (CNN) for food image classification, and 3) on stereo vision techniques for the volume and nutrition estimation of the food. We use a pre-trained CNN in the Food-101 dataset to train a deep learning classification model employing our dataset Mediterranean Greek Food (MedGRFood). Based on the EfficientNet family of CNNs, we use the EfficientNetB2 both for the pre-trained model and its weights evaluation, as well as for classifying food images in the MedGRFood dataset. Next, we estimate the volume of the food, through 3D food reconstruction of two images taken by a smartphone camera. The proposed volume estimation subsystem uses stereo vision techniques and algorithms, and needs the input of two food images to reconstruct the point cloud of the food and to compute its quantity. Results: The classification accuracy where true class matches with the most probable class predicted by the model (Top-1 accuracy) is 83.8%, while the accuracy where true class matches with any one of the 5 most probable classes predicted by the model (Top-5 accuracy) is 97.6%, for the food classification subsystem. The food volume estimation subsystem achieves an overall mean absolute percentage error 10.5% for 148 different food dishes. Conclusions: The proposed automated image-based dietary assessment system provides the capability of continuous recording of health data in real time.

Short Term Glucose Prediction in Patients with Type 1 Diabetes Mellitus.

Glucose prediction is used in diabetes self-management as it allows to take suitable actions for proper glycemic regulation of the patient. The aim of this work is the short-term personalized glucose prediction in patients with Type 1 diabetes mellitus (T1DM). In this scope, we compared two different models, an autoregressive moving average (ARMA) model and a long short-term memory (LSTM) model for different prediction horizons. The comparison of two models was performed using the evaluation metrics of root mean square error (RMSE) and mean absolute error (MAE). The models were trained and tested in 29 real patients. The results shown that the LSTM model had better performance than ARMA with RMSE 3.13, 6.41 and 8.81 mg/dL and MAE 1.98, 5.06 and 6.47 mg/dL for 5-, 15- and 30-minutes prediction horizon.

GlucoseML Mobile Application for Automated Dietary Assessment of Mediterranean Food.

Over the years and with the help of technology, the daily care of type 1 diabetes has been improved significantly. The increased adoption of continuous glucose monitoring, the continuous subcutaneous insulin injection and the accurate behavioral monitoring mHealth solutions have contributed to this phenomenon. In this study we present a mobile application for automated dietary assessment of Mediterranean food images as part of the GlucoseML system. Based on short-term predictive analysis of the glucose trajectory, GlucoseML is a type-1 diabetes self-management system. A computer vision approach is used as main part of the GlucoseML dietary assessment system calculating food carbohydrates, fats and proteins, relying on: (i) a deep learning subsystem for food image classification, and (ii) a 3D food image reconstruction subsystem for the volume estimation of food. The deep learning subsystem achieves 82.4% and 97.5% top-1 and top-5 accuracy, respectively, for food image classification while the subsystem for volume estimation of food achieves a mean absolute percentage error 10.7% for the four main categories of MedGRFood dataset.

Automated Assessment of Balance Rehabilitation Exercises With a Data-Driven Scoring Model: Algorithm Development and Validation Study.

BACKGROUND: Balance rehabilitation programs represent the most common treatments for balance disorders. Nonetheless, lack of resources and lack of highly expert physiotherapists are barriers for patients to undergo individualized rehabilitation sessions. Therefore, balance rehabilitation programs are often transferred to the home environment, with a considerable risk of the patient misperforming the exercises or failing to follow the program at all. Holobalance is a persuasive coaching system with the capacity to offer full-scale rehabilitation services at home. Holobalance involves several modules, from rehabilitation program management to augmented reality coach presentation. OBJECTIVE: The aim of this study was to design, implement, test, and evaluate a scoring model for the accurate assessment of balance rehabilitation exercises, based on data-driven techniques. METHODS: The data-driven scoring module is based on an extensive data set (approximately 1300 rehabilitation exercise sessions) collected during the Holobalance pilot study. It can be used as a training and testing data set for training machine learning (ML) models, which can infer the scoring components of all physical rehabilitation exercises. In that direction, for creating the data set, 2 independent experts monitored (in the clinic) 19 patients performing 1313 balance rehabilitation exercises and scored their performance based on a predefined scoring rubric. On the collected data, preprocessing, data cleansing, and normalization techniques were applied before deploying feature selection techniques. Finally, a wide set of ML algorithms, like random forests and neural networks, were used to identify the most suitable model for each scoring component. RESULTS: The results of the trained model improved the performance of the scoring module in terms of more accurate assessment of a performed exercise, when compared with a rule-based scoring model deployed at an early phase of the system (k-statistic value of 15.9% for sitting exercises, 20.8% for standing exercises, and 26.8% for walking exercises). Finally, the resulting performance of the model resembled the threshold of the interobserver variability, enabling trustworthy usage of the scoring module in the closed-loop chain of the Holobalance coaching system. CONCLUSIONS: The proposed set of ML models can effectively score the balance rehabilitation exercises of the Holobalance system. The models had similar accuracy in terms of Cohen kappa analysis, with interobserver variability, enabling the scoring module to infer the score of an exercise based on the collected signals from sensing devices. More specifically, for sitting exercises, the scoring model had high classification accuracy, ranging from 0.86 to 0.90. Similarly, for standing exercises, the classification accuracy ranged from 0.85 to 0.92, while for walking exercises, it ranged from 0.81 to 0.90. TRIAL REGISTRATION: ClinicalTrials.gov NCT04053829; https://clinicaltrials.gov/ct2/show/NCT04053829.

Machine Learning Coronary Artery Disease Prediction Based on Imaging and Non-Imaging Data.

The prediction of obstructive atherosclerotic disease has significant clinical meaning for the decision making. In this study, a machine learning predictive model based on gradient boosting classifier is presented, aiming to identify the patients of high CAD risk and those of low CAD risk. The machine learning methodology includes five steps: the preprocessing of the input data, the class imbalance handling applying the Easy Ensemble algorithm, the recursive feature elimination technique implementation, the implementation of gradient boosting classifier, and finally the model evaluation, while the fine tuning of the presented model was implemented through a randomized search optimization of the model's hyper-parameters over an internal 3-fold cross-validation. In total, 187 participants with suspicion of CAD previously underwent CTCA during EVINCI and ARTreat clinical studies and were prospectively included to undergo follow-up CTCA. The predictive model was trained using imaging data (geometrical and blood flow based) and non-imaging data. The overall predictive accuracy of the model was 0.81, using both imaging and non-imaging data. The innovative aspect of the proposed study is the combination of imaging-based data with the typical CAD risk factors to provide an integrated CAD risk-predictive model.

Mediterranean Food Image Recognition Using Deep Convolutional Networks.

We present a new dataset of food images that can be used to evaluate food recognition systems and dietary assessment systems. The Mediterranean Greek food -MedGRFood dataset consists of food images from the Mediterranean cuisine, and mainly from the Greek cuisine. The dataset contains 42,880 food images belonging to 132 food classes which have been collected from the web. Based on the EfficientNet family of convolutional neural networks, specifically the EfficientNetB2, we propose a new deep learning schema that achieves 83.4% top-1 accuracy and 97.8% top-5 accuracy in the MedGRFood dataset for food recognition. This schema includes the use of the fine tuning, transfer learning and data augmentation technique.

Site specific prediction of PCI stenting based on imaging and biomechanics data using gradient boosting tree ensembles.

Cardiovascular diseases are nowadays considered as the main cause of morbidity and mortality worldwide. Coronary Artery Disease (CAD), the most typical form of cardiovascular disease is diagnosed by a variety of imaging modalities, both invasive and non-invasive, which involve either risk implications or high cost. Therefore, several attempts have been undertaken to early diagnose and predict either the high CAD risk patients or the cardiovascular events, implementing machine learning techniques. The purpose of this study is to present a classification scheme for the prediction of Percutaneous Coronary Intervention (PCI) stenting placement, using image-based data. The proposed classification model is a gradient boosting classifier, incorporated into a class imbalance handling technique, the Easy ensemble scheme and aims to classify coronary segments into high CAD risk and low CAD risk, based on their PCI placement. Through this study, we investigate the importance of image based features, concluding that the combination of the coronary degree of stenosis and the fractional flow reserve achieves accuracy 78%.

A Dynamic Bayesian Network Approach to Behavioral Modelling of Elderly People during a Home-based Augmented Reality Balance Physiotherapy Programme.

In this study, we propose a dynamic Bayesian network (DBN)-based approach to behavioral modelling of community dwelling older adults at risk for falls during the daily sessions of a hologram-enabled vestibular rehabilitation therapy programme. The component of human behavior being modelled is the level of frustration experienced by the user at each exercise, as it is assessed by the NASA Task Load Index. Herein, we present the topology of the DBN and test its inference performance on real-patient data.Clinical Relevance- Precise behavioral modelling will provide an indicator for tailoring the rehabilitation programme to each individual's personal psychological needs.

Predicting Lymphoma Development by Exploiting Genetic Variants and Clinical Findings in a Machine Learning-Based Methodology With Ensemble Classifiers in a Cohort of Sjögren's Syndrome Patients.

Lymphoma development constitutes one of the most serious clinico-pathological manifestations of patients with Sjögren's Syndrome (SS). Over the last decades the risk for lymphomagenesis in SS patients has been studied aiming to identify novel biomarkers and risk factors predicting lymphoma development in this patient population. Objective: The current study aims to explore whether genetic susceptibility profiles of SS patients along with known clinical, serological and histological risk factors enhance the accuracy of predicting lymphoma development in this patient population. Methods: The potential predicting role of both genetic variants, clinical and laboratory risk factors were investigated through a Machine Learning-based (ML) framework which encapsulates ensemble classifiers. Results: Ensemble methods empower the classification accuracy with approaches which are sensitive to minor perturbations in the training phase. The evaluation of the proposed methodology based on a 10-fold stratified cross validation procedure yielded considerable results in terms of balanced accuracy (GB: 0.7780 ± 0.1514, RF Gini: 0.7626 ± 0.1787, RF Entropy: 0.7590 ± 0.1837). Conclusions: The initial clinical, serological, histological and genetic findings at an early diagnosis have been exploited in an attempt to establish predictive tools in clinical practice and further enhance our understanding towards lymphoma development in SS.

Short-term prediction of glucose in type 1 diabetes using kernel adaptive filters.

This study aims at presenting a nonlinear, recursive, multivariate prediction model of the subcutaneous glucose concentration in type 1 diabetes. Nonlinear regression is performed in a reproducing kernel Hilbert space, by either the fixed budget quantized kernel least mean square (QKLMS-FB) or the approximate linear dependency kernel recursive least-squares (KRLS-ALD) algorithm, such that a sparse model structure is accomplished. A multivariate feature set (i.e., subcutaneous glucose, food carbohydrates, insulin regime and physical activity) is used and its influence on short-term glucose prediction is investigated. The method is evaluated using data from 15 patients with type 1 diabetes in free-living conditions. In the case when all the input variables are considered: (i) the average root mean squared error (RMSE) of QKLMS-FB increases from 13.1 mg dL-1 (mean absolute percentage error (MAPE) 6.6%) for a 15-min prediction horizon (PH) to 37.7 mg dL-1 (MAPE 20.8%) for a 60-min PH and (ii) the RMSE of KRLS-ALD, being predictably lower, increases from 10.5 mg dL-1 (MAPE 5.2%) for a 15-min PH to 31.8 mg dL-1 (MAPE 18.0%) for a 60-min PH. Multivariate data improve systematically both the regularity and the time lag of the predictions, reducing the errors in critical glucose value regions for a PH ≥ 30 min. Graphical abstract ᅟ.

Cohort Harmonization and Integrative Analysis From a Biomedical Engineering Perspective.

In this review, the critical parts and milestones for data harmonization, from the biomedical engineering perspective, are outlined. The need for data sharing between heterogeneous sources paves the way for cohort harmonization; thus, fostering data integration and interdisciplinary research. Unmet needs in chronic diseases, as well as in other diseases, can be addressed based on the integration of patient health records and the sharing of information of the clinical picture and outcome. The stratification of patients, the determination of various clinical and outcome features, and the identification of novel biomarkers for the different phenotypes of the disease characterize the impact of cohort harmonization in patient-centered clinical research and in precision medicine. Subsequently, the establishment of matching techniques and ontologies for the creation of data schemas are also presented. The exploitation of web technologies and data-collection tools supports the opportunities to achieve new levels of integration and interoperability. Ethical and legal issues that arise when sharing and harmonizing individual-level data are discussed in order to evaluate the harmonization potential. Use cases that shape and test the harmonization approach are explicitly analyzed along with their significant results on their research objectives. Finally, future trends and directions are discussed and critically reviewed toward a roadmap in cohort harmonization for clinical medicine.

A Clinical Decision Support Platform for the Risk Stratification, Diagnosis, and Prediction of Coronary Artery Disease Evolution.

SMARTool aims to perform accurate risk stratification of coronary artery disease patients as well as to provide early diagnosis and prediction of disease progression. This is achieved by the acquisition of data from about 263 patients including computed tomography angiographic images, clinical, molecular, biohumoral, exposome, inflammatory and omics data. Data are collected in two time points with a followup period of approximately 5 years. In the first step, data mining techniques are implemented for the estimation of risk stratification. In the next step, patients, who are classified as medium to high risk are considered for coronary imaging and computational modelling of blood flow, plaque growth and stenosis severity assessment. Additionally, patients with increased stenosis are selected for stent deployment. All the above modules are integrated in a cloud-based platform for the clinical decision support (CDSS) of patients with coronary artery disease. The work presents preliminary results employing the SMARTool dataset as well as the concept and architecture of the under development platform.

A Machine Learning Approach for the Prediction of the Progression of Cardiovascular Disease based on Clinical and Non-Invasive Imaging Data.

Nowadays, cardiovascular diseases are very common and are considered as the main cause of morbidity and mortality worldwide. Coronary Artery Disease (CAD), the most typical cardiovascular disease is diagnosed by a variety of medical imaging modalities, which involve costs and complications. Therefore, several attempts have been undertaken to early diagnose and predict CAD status and progression through machine learning approaches. The purpose of this study is to present a machine learning technique for the prediction of CAD, using image-based data and clinical data. We investigate the effect of vascular anatomical features of the three coronary arteries on the graduation of CAD. A classification model is built to predict the future status of CAD, including cases of "no CAD" patients, "non-obstructive CAD" patients and "obstructive CAD" patients. The best accuracy was achieved by the implementation of a tree-based classifier, J48 classifier, after a ranking feature selection methodology. The majority of the selected features are the vessel geometry derived features, among the traditional risk factors. The combination of geometrical risk factors with the conventional ones constitutes a novel scheme for the CAD prediction.

Kernel-based adaptive learning improves accuracy of glucose predictive modelling in type 1 diabetes: A proof-of-concept study.

This study aims at demonstrating the need for nonlinear recursive models to the identification and prediction of the dynamic glucose system in type 1 diabetes. Nonlinear regression is performed in a reproducing kernel Hilbert space, by the Approximate Linear Dependency Kernel Recursive Least Squares (KRLS-ALD) algorithm, such that a sparse model structure is accomplished. The method is evaluated on seven people with type 1 diabetes in free-living conditions, where a change in glycaemic dynamics is forced by increasing the level of physical activity in the middle of the observational period. The univariate input allows for short-term (≤30 min) predictions with KRLS-ALD reaching an average root mean square error of 15.22±5.95 mgdL-1 and an average time lag of 17.14±2.67 min for an horizon of 30 min. Its performance is considerably better than that of time-invariant (regularized) linear regression models.

Non-linear dynamic modeling of glucose in type 1 diabetes with kernel adaptive filters.

We propose a non-linear recursive solution to the problem of short-term prediction of glucose in type 1 diabetes. The Fixed Budget Quantized Kernel Least Mean Square (QKLMS-FB) algorithm is employed to construct a univariate model of subcutaneous glucose concentration, which: (i) handles nonlinearities by transforming the input space into a high-dimensional Reproducing Kernel Hilbert Space and, (ii) finds a sparse solution by retaining a representative subset of the training input vectors. The dataset comes from the continuous multi-day recordings of 15 type 1 patients in free-living conditions. QKLMS-FB produces an average root mean squared error of 18.66±3.19 mg/dl for a prediction horizon of 30 min with 82.04% of hypoglycemic readings and 93.30% of hyperglycemic ones being classified as clinically accurate or with benign errors. The effect of the prediction horizon is more evident in the hypoglycemic range.

Evaluation of short-term predictors of glucose concentration in type 1 diabetes combining feature ranking with regression models.

Glucose concentration in type 1 diabetes is a function of biological and environmental factors which present high inter-patient variability. The objective of this study is to evaluate a number of features, which are extracted from medical and lifestyle self-monitoring data, with respect to their ability to predict the short-term subcutaneous (s.c.) glucose concentration of an individual. Random forests (RF) and RReliefF algorithms are first employed to rank the candidate feature set. Then, a forward selection procedure follows to build a glucose predictive model, where features are sequentially added to it in decreasing order of importance. Predictions are performed using support vector regression or Gaussian processes. The proposed method is validated on a dataset of 15 type diabetics in real-life conditions. The s.c. glucose profile along with time of the day and plasma insulin concentration are systematically highly ranked, while the effect of food intake and physical activity varies considerably among patients. Moreover, the average prediction error converges in less than d/2 iterations (d is the number of features). Our results suggest that RF and RReliefF can find the most informative features and can be successfully used to customize the input of glucose models.

Online prediction of glucose concentration in type 1 diabetes using extreme learning machines.

We propose an online machine-learning solution to the problem of nonlinear glucose time series prediction in type 1 diabetes. Recently, extreme learning machine (ELM) has been proposed for training single hidden layer feed-forward neural networks. The high accuracy and fast learning speed of ELM drive us to investigate its applicability to the glucose prediction problem. Given that diabetes self-monitoring data are received sequentially, we focus on online sequential ELM (OS-ELM) and online sequential ELM kernels (KOS-ELM). A multivariate feature set is utilized concerning subcutaneous glucose, insulin therapy, carbohydrates intake and physical activity. The dataset comes from the continuous multi-day recordings of 15 type 1 patients in free-living conditions. Assuming stationarity and evaluating the performance of the proposed method by 10-fold cross- validation, KOS-ELM were found to perform better than OS-ELM in terms of prediction error, temporal gain and regularity of predictions for a 30-min prediction horizon.