Objectives

New data have emerged from ambulatory and acute care settings about adverse patient events, including death, attributable to erroneous blood glucose meter measurements and leading to questions over their use in critically ill patients. The U.S. Food and Drug Administration published new, more stringent guidelines for glucose meter manufacturers to evaluate the performance of blood glucose meters in critically ill patient settings. The primary objective of this international, multicenter, multidisciplinary clinical study was to develop and apply a rigorous clinical accuracy assessment algorithm, using four distinct statistical tools, to evaluate the clinical accuracy of a blood glucose monitoring system in critically ill patients.

Design

Observational study.

Setting

Five international medical and surgical ICUs.

Patients

All patients admitted to critical care settings in the centers.

Interventions

None.

Measurements and main results

Glucose measurements were performed on 1,698 critically ill patients with 257 different clinical conditions and complex treatment regimens. The clinical accuracy assessment algorithm comprised four statistical tools to assess the performance of the study blood glucose monitoring system compared with laboratory reference methods traceable to a definitive standard. Based on POCT12-A3, the Clinical Laboratory Standards Institute standard for hospitals about hospital glucose meter procedures and performance, and Parkes error grid clinical accuracy performance criteria, no clinically significant differences were observed due to patient condition or therapy, with 96.1% and 99.3% glucose results meeting the respective criteria. Stratified sensitivity and specificity analysis (10 mg/dL glucose intervals, 50-150 mg/dL) demonstrated high sensitivity (mean = 95.2%, SD = ± 0.02) and specificity (mean = 95. 8%, SD = ± 0.03). Monte Carlo simulation modeling of the study blood glucose monitoring system showed low probability of category 2 and category 3 insulin dosing error, category 2 = 2.3% (41/1,815) and category 3 = 1.8% (32/1,815), respectively. Patient trend analysis demonstrated 99.1% (223/225) concordance in characterizing hypoglycemic patients.

Conclusions

The multicomponent, clinical accuracy assessment algorithm demonstrated that the blood glucose monitoring system was acceptable for use in critically ill patient settings when compared to the central laboratory reference method. This clinical accuracy assessment algorithm is an effective tool for comprehensively assessing the validity of whole blood glucose measurement in critically ill patient care settings.

Metrics derived from continuous glucose monitoring (CGM) systems are often discordant between systems. A major cause is that CGM systems are not standardized; they use various algorithms and calibration methods, leading to discordant CGM readings across systems. This discordance can be addressed by standardizing CGM performance assessments: If manufacturers aim their CGM systems at the same target, then CGM readings will align across systems. This standardization should include the comparator device, sample origin, and study procedures. With better aligned CGM readings, CGM-derived metrics will subsequently also align better between systems.

Comparing the performance of different continuous glucose monitoring (CGM) systems is challenging due to the lack of comprehensive guidelines for clinical study design. In particular, the absence of concise requirements for the distribution of comparator (reference) blood glucose (BG) concentrations and their rate of change (RoC) that are used to evaluate CGM performance, impairs comparability. For this article, several experts in the field of CGM performance testing have collaborated to propose characteristics of the distribution of comparator measurements that should be collected during CGM performance testing. Specifically, it is proposed that at least 7.5% of comparator BG concentrations are <70 mg/dL (3.9 mmol/L) and >300 mg/dL (16.7 mmol/L), respectively, and that at least 7.5% of BG-RoC combinations indicate fast BG changes with impending hypo- or hyperglycemia, respectively. These proposed characteristics of the comparator data can facilitate the harmonization of testing conditions across different studies and CGM systems and ensure that the most relevant scenarios representing real-life situations are established during performance testing. In addition, a study protocol and testing procedure for the manipulation of glucose levels are suggested that enable the collection of comparator data with these characteristics. This work is an important step toward establishing a future standard for the performance evaluation of CGM systems.