Benefits and Limitations of MARD as a Performance Parameter for Continuous Glucose Monitoring in the Interstitial Space.

High-quality performance of medical devices for glucose monitoring is important for a safe and efficient usage of this diagnostic option by patients with diabetes. The mean absolute relative difference (MARD) parameter is used most often to characterize the measurement performance of systems for continuous glucose monitoring (CGM). Calculation of this parameter is relatively easy and comparison of the MARD numbers between different CGM systems appears to be straightforward on the first glance. However, a closer look reveals that a number of complex aspects make interpretation of the MARD numbers provided by the manufacturer for their CGM systems difficult. In this review, these aspects are discussed and considerations are made for a systematic and appropriate evaluation of the MARD in clinical trials. The MARD should not be used as the sole parameter to characterize CGM systems, especially when it comes to nonadjunctive usage of such systems.

Limits to the Evaluation of the Accuracy of Continuous Glucose Monitoring Systems by Clinical Trials.

Systems for continuous glucose monitoring (CGM) are evolving quickly, and the data obtained are expected to become the basis for clinical decisions for many patients with diabetes in the near future. However, this requires that their analytical accuracy is sufficient. This accuracy is usually determined with clinical studies by comparing the data obtained by the given CGM system with blood glucose (BG) point measurements made with a so-called reference method. The latter is assumed to indicate the correct value of the target quantity. Unfortunately, due to the nature of the clinical trials and the approach used, such a comparison is subject to several effects which may lead to misleading results. While some reasons for the differences between the values obtained with CGM and BG point measurements are relatively well-known (e.g., measurement in different body compartments), others related to the clinical study protocols are less visible, but also quite important. In this review, we present a general picture of the topic as well as tools which allow to correct or at least to estimate the uncertainty of measures of CGM system performance.

Analyzing the Potential of Advanced Insulin Dosing Strategies in Patients With Type 2 Diabetes: Results From a Hybrid In Silico Study.

BACKGROUND: The ongoing improvement of continuous glucose monitoring (CGM) sensors and of insulin pumps are paving the way for a fast implementation of artificial pancreas (AP) for type 1 diabetes (T1D) patients. The case for type 2 diabetes (T2D) patients is less obvious since usually some residual beta cell function allows for simpler therapy approaches, and even multiple daily injections (MDI) therapy is not very widespread. However, the number of insulin dependent T2D patients is vastly increasing and therefore a need for understanding chances and challenges of an automated insulin therapy arises. Based on this background, this article analyzes conditions under which the use of more advanced therapeutic approaches, particularly AP, could bring a substantial improvement and should be considered as a viable therapy option. METHOD: Data of 14 insulin-treated T2D patients on MDI wearing a CGM device and deviation analysis methods were used to estimate the expected improvements in the clinical outcome by using self-monitoring of blood glucose (SMBG) with advanced carbohydrate counting, a full AP or intermediate approaches, either CGM measurements with MDI therapy or SMBG with insulin pump. HbA1C and time in range (70-140 mg/dl, 70-180 mg/dl, respectively) were used as a performance measure. Outcome measures beyond glycemic control (eg, compliance, patient acceptance) have not been analyzed in this study. RESULTS: AP has the potential to improve the condition of many poorly controlled insulin-treated T2D patients. However, as the interpatient variability is much higher than in T1D, a prescreening is recommended to select suitable patients. CONCLUSIONS: Clinical criteria need to be developed for inclusion/exclusion of T2D patients for AP related therapies.

Significance and Reliability of MARD for the Accuracy of CGM Systems.

BACKGROUND: There is a need to assess the accuracy of continuous glucose monitoring (CGM) systems for several uses. Mean absolute relative difference (MARD) is the measure of choice for this. Unfortunately, it is frequently overlooked that MARD values computed with data acquired during clinical studies do not reflect the accuracy of the CGM system only, but are strongly influenced by the design of the study. Thus, published MARD values must be understood not as precise values but as indications with some uncertainty. DATA AND METHODS: Data from a recent clinical trial, Monte Carlo simulations, and assumptions about the error distribution of the reference measurements have been used to determine the confidence region of MARD as a function of the number and the accuracy of the reference measurements. RESULTS: The uncertainty of the computed MARD values can be quantified by a newly introduced MARD reliability index (MRI), which independently mirrors the reliability of the evaluation. Thus MARD conveys information on the accuracy of the CGM system, while MRI conveys information on the uncertainty of the computed MARD values. CONCLUSIONS: MARD values from clinical studies should not be used blindly but the reliability of the evaluation should be considered as well. Furthermore, it should not be ignored that MARD does not take into account the key feature of CGM sensors, the frequency of the measurements. Additional metrics, such as precision absolute relative difference (PARD) should be used as well to obtain a better evaluation of the CGM performance for specific uses, for example, for artificial pancreas.

Performance Comparison of CGM Systems: MARD Values Are Not Always a Reliable Indicator of CGM System Accuracy.

BACKGROUND: The ongoing progress of continuous glucose monitoring (CGM) systems results in an increasing interest in comparing their performance, in particular in terms of accuracy, that is, matching CGM readings with reference values measured at the same time. Most often accuracy is evaluated by the mean absolute relative difference (MARD). It is frequently overseen that MARD does not only reflect accuracy, but also the study protocol and evaluation procedure, making a cross-study comparison problematic. METHODS: We evaluate the effect of several factors on the MARD statistical properties: number of paired reference and CGM values, distribution of the paired values, accuracy of the reference measurement device itself and the time delay between data pairs. All analysis is done using clinical data from 12 patients wearing 6 sensors each. RESULTS: We have found that a few paired points can have a potentially high impact on MARD. Leaving out those points for evaluation thus reduces the MARD. Similarly, accuracy of the reference measurements greatly affects the MARD as numerical and graphical data show. Results also show that a log-normal distribution of the paired references provides a significantly different MARD than, for example, a uniform distribution. CONCLUSIONS: MARD is a reasonable parameter to characterize the performance of CGM systems when keeping its limitations in mind. To support clinicians and patients in selecting which CGM system to use in a clinical setting, care should be taken to make MARD more comparable by employing a standardized evaluation procedure.

Time Delay of CGM Sensors: Relevance, Causes, and Countermeasures.

BACKGROUND: Continuous glucose monitoring (CGM) is a powerful tool to support the optimization of glucose control of patients with diabetes. However, CGM systems measure glucose in interstitial fluid but not in blood. Rapid changes in one compartment are not accompanied by similar changes in the other, but follow with some delay. Such time delays hamper detection of, for example, hypoglycemic events. Our aim is to discuss the causes and extent of time delays and approaches to compensate for these. METHODS: CGM data were obtained in a clinical study with 37 patients with a prototype glucose sensor. The study was divided into 5 phases over 2 years. In all, 8 patients participated in 2 phases separated by 8 months. A total number of 108 CGM data sets including raw signals were used for data analysis and were processed by statistical methods to obtain estimates of the time delay. RESULTS: Overall mean (SD) time delay of the raw signals with respect to blood glucose was 9.5 (3.7) min, median was 9 min (interquartile range 4 min). Analysis of time delays observed in the same patients separated by 8 months suggests a patient dependent delay. No significant correlation was observed between delay and anamnestic or anthropometric data. The use of a prediction algorithm reduced the delay by 4 minutes on average. CONCLUSIONS: Prediction algorithms should be used to provide real-time CGM readings more consistent with simultaneous measurements by SMBG. Patient specificity may play an important role in improving prediction quality.

LMI-Based Approaches for the Calibration of Continuous Glucose Measurement Sensors.

The problem of online calibration and recalibration of continuous glucose monitoring (CGM) devices is considered. Two different parametric relations between interstitial and blood glucose are investigated and constructive algorithms to adaptively estimate the parameters within those relations are proposed. One characteristic is the explicit consideration of measurement uncertainty of the device used to collect the calibration measurements. Another feature is the automatic detection of fingerstick measurements that are not suitable to be used for calibration. Since the methods rely on the solution of linear matrix inequalities resulting in convex optimization problems, the algorithms are of moderate computational complexity and could be implemented on a CGM device. The methods were assessed on clinical data from 17 diabetic patients and the improvements with respect to the current state of the art is shown.

A novel online recalibration strategy for continuous glucose measurement sensors employing LMI techniques.

This paper considers the problem of online calibration and recalibration of continuous glucose monitoring devices. A parametric relation between interstitial and blood glucose is investigated and a constructive algorithm to adaptively estimate the parameters within this relation is proposed. The algorithm explicitly considers measurement uncertainty of the device used to collect the calibration measurements and enables automatic detection of measurements which are not suitable to be used for calibration. The method was assessed on clinical data from 17 diabetic patients and the improvements with respect to the current state of the art is shown.

Performance evaluations of continuous glucose monitoring systems: precision absolute relative deviation is part of the assessment.

BACKGROUND: Even though a Clinical and Laboratory Standards Institute proposal exists on the design of studies and performance criteria for continuous glucose monitoring (CGM) systems, it has not yet led to a consistent evaluation of different systems, as no consensus has been reached on the reference method to evaluate them or on acceptance levels. As a consequence, performance assessment of CGM systems tends to be inconclusive, and a comparison of the outcome of different studies is difficult. MATERIALS AND METHODS: Published information and available data (as presented in this issue of Journal of Diabetes Science and Technology by Freckmann and coauthors) are used to assess the suitability of several frequently used methods [International Organization for Standardization, continuous glucose error grid analysis, mean absolute relative deviation (MARD), precision absolute relative deviation (PARD)] when assessing performance of CGM systems in terms of accuracy and precision. RESULTS: The combined use of MARD and PARD seems to allow for better characterization of sensor performance. The use of different quantities for calibration and evaluation, e.g., capillary blood using a blood glucose (BG) meter versus venous blood using a laboratory measurement, introduces an additional error source. Using BG values measured in more or less large intervals as the only reference leads to a significant loss of information in comparison with the continuous sensor signal and possibly to an erroneous estimation of sensor performance during swings. Both can be improved using data from two identical CGM sensors worn by the same patient in parallel. CONCLUSIONS: Evaluation of CGM performance studies should follow an identical study design, including sufficient swings in glycemia. At least a part of the study participants should wear two identical CGM sensors in parallel. All data available should be used for evaluation, both by MARD and PARD, a good PARD value being a precondition to trust a good MARD value. Results should be analyzed and presented separately for clinically different categories, e.g., hypoglycemia, exercise, or night and day.

A diabetes management system empowering patients to reach optimised glucose control: from monitor to advisor.

The DIAdvisor is an EC/FP7 funded project aiming at the development of a Blood Glucose prediction device which uses easily available information to optimise the therapy of patients with diabetes.