Accuracy of blood glucose measurements in the critical value range is important for properly treating patients with severe hypoglycemia and hyperglycemia.
To evaluate the performance and reliability of point-of-care glucose (POCG) results in the critical value range among multiple facilities.
Q-Probes participants retrospectively collected data from up to 50 POCG results in their critical value range including patient location, type of testing operator, repeat glucose results, and caregiver notification. A repeat measurement at 10 minutes or less that was within 15 mg/dL of initial critical low or 20% of initial critical high value was considered a confirmed result.
Fifty facilities submitted data. Of 2349 critical POCG measurements, 1386 (59.0%) were retested. The median institutional retest rate was 56%. The retest rate was significantly higher when initial results were in the critical low range, P < .001. Although 30 of 50 facilities (60%) had written procedures for retesting, this was not associated with higher retest rates (P = .34). Among 35 facilities that routinely retested critical POCG results, 23 (65.7%) had criteria defined for interpreting results. The median institutional confirmation rate for retested specimens was 81.7%. The median institutional rate for caregiver notification of critical POCG results was 85.7%. Five hundred eighty-six of 1488 critical POCG notifications (39.4%) were done on patients in whom specimens were not retested.
This study shows that POCG results in the critical range may be unreliable because of testing errors that are not recognized from lack of confirmatory testing. In addition, notification of critical POCG results is not consistently performed.
The concept for rapid communication of “life-threatening” laboratory results was first described in 1972 by Lundberg.1 Since then, notification of critical test results has been established as an important patient safety goal, integrated into medical practice, and required for hospital and laboratory accreditation. Although there is growing awareness that point-of-care glucose (POCG) measurements may be unreliable in critically ill patients,2–4 less attention has been directed at the accuracy of these methods when results for any patient are in the critical high or low range.
In the first large study of critical value practices among 623 clinical laboratories, Howanitz et al5 recognized that critical-value “policies and procedures that have been established for the central laboratory should be extended” to point-of-care testing. Among the few studies involving critical values with POCG methods, all have raised concerns about risks from erroneous measurements.6–9 In contrast, glucose or other test results in the critical value range from specimens tested by clinical laboratories are rarely inaccurate.10,11
These observations have important implications because testing conditions and reliability of point-of-care tests may require different operational and management strategies than those applied to clinical laboratory testing. For example, a large and variable group of nurses and patient care technicians who are generally responsible for POCG measurement rarely may encounter or have experience with handling critical results. In contrast, medical technologists working in clinical laboratories handle critical results on a daily basis. Furthermore, POCG testing is done under less uniform conditions using capillary blood, which tends to produce more inconsistent results than measurements on venous specimens. However, when it is properly used, some limitations to POCG testing are offset to some degree by the clinical benefit of having immediate access to results at the bedside.
The objective of this study was to describe the performance of POCG measurements under routine clinical conditions when results were in the critical value range as reported by participants in the College of American Pathologists (CAP) Q-Probes program. In addition, we evaluated various practices and policies that may affect performance and management of critical POCG results. Observations from this multi-institutional study along with previous reports were analyzed to provide evidence-based support to guide best practices for managing critical POCG results that would reduce errors and improve patient safety.
MATERIALS AND METHODS
This observational study was conducted in 2013 according to the Q-Probes format as previously described, which involved laboratories that subscribe and participate in the program.12 Instructions and data collection materials were provided and were returned to the CAP by a specified time. After data analysis, subscribers received an individual detailed report about their performance on quality indicators benchmarked against other participants. The range of performance among participants, summary analysis of significant associations with demographics and practice variables, and a critique report were sent to each participating institution.
Participants were instructed to collect retrospective data on the previous 50 cases in which POCG test results were within their critical range or for all critical results within the past 6 month if fewer than 50 cases were found. Participants submitted their data to the CAP for analysis. For each initial POCG measurement that was in the critical range as defined by the facility's criteria, participants checked for documentation that results were reported to the responsible medical caregiver as a comment accompanying test results in the patient's medical record or by other means. Documentation included the identity of the caregiver as well as the date and time of notification. Additional information collected for each critical POCG testing episode included patient location, whether the patient was a neonate, the operator who performed testing (nurse, patient care technician, or other), and critical POCG test result. If another specimen was tested or collected within 10 minutes, the specimen type (venous or capillary) and method used for retesting (point of care or clinical laboratory) were recorded. Results from tests performed in critical care, inpatient, and emergency department locations were included. Critical POCG results from non–emergency department outpatients were not collected by participants.
Any retested specimen with critical low value that was within 15 mg/dL of the initial critical low result or within 20% of critical high value was considered confirmed.13 Any repeat measurements that did not meet these criteria were considered invalid or nonconfirmed critical results.
Participants also provided information about relative test volume among inpatient, critical care, and emergency department locations as well as the number of operators trained and certified to perform POCG testing. Participants also reported various laboratory practices associated with POCG testing that included operator training about how to handle critical test results, policies and procedures related to retesting critical results, criteria used for interpreting repeat measurements, and POCG critical value notification policies and procedures. If any question was left unanswered, the participant was excluded from the analysis for that question. Finally, participants provided institutional demographic information that included occupied bed size, government affiliation, location, and type of institution.
Data were tabulated and analyzed by the biostatistics department of the CAP. Statistical analyses were performed to determine factors significantly associated with the 3 laboratory performance indicators—critical values repeat rate, result confirmation rate, and documentation of caregiver notification rate. Rates based on fewer than 6 cases were excluded from the analyses. Associations between the performance indicators and the demographic and practice variables were analyzed using Kruskal-Wallis tests for discrete-valued independent variables and regression analysis for continuous independent variables. Variables with significant associations (P < .10) were then included in a forward-selection multivariate regression model. A significance level of .05 was used for this final model. Aggregate study results were tested using χ2 and Fisher exact tests. A significance level of .05 was used for these tests. All analyses were performed with SAS 9.2 (SAS Institute, Cary, North Carolina).
RESULTS
Table 1 describes institutional demographics among participants. Table 2 shows the estimated 2012 volume of POCG testing. The highest testing rates occurred among inpatients, followed by critical care units and emergency departments. The relative proportion of critical values was highest in the emergency department in comparison with the relative distribution of annual POCG testing in other locations. For example, the proportion of critical POCG results from the emergency department was 23.2% whereas the median estimated annual volume of all POCG from this location was 2906 (of 74 947 POCG tests; 3.9%) from all locations. Of 2349 POCG measurements, 211 (9.0%) were reported as performed on neonates, 9 unknown, and the remaining 2129 were from nonneonatal patients. The median (central 80th percentile) of trained POCG testing personnel among 50 institutions was 731 (101–1636). The median number of annual POCG tests performed per operator was 119 in 44 reporting laboratories, with 80th central percentile range of 43 to 207 tests per operator. Measurements involving critical POCG test results were performed by nurses in 1530 of 2330 cases (65.7%), by patient care technicians or aides in 736 (31.6%), and by others in 64 (2.7%). Most participants (48 of 50; 96%) reported that POCG operators receive special instructions about handling or reporting critical values. Table 3 shows the high and low cutoff values for point-of-care and clinical laboratory glucose measurements used by each facility.
Repeat Testing
Among 49 participants, 35 (71.4%) reported that retesting another specimen was done for POCG results in their critical value range. Of these 35, 23 (65.7%) reported having criteria defined for confirmation of critical results based on repeat values. Among 50 facilities, 30 (60%) had a written procedure for routinely retesting critical POCG results using a venous blood specimen (n = 9), a new capillary specimen (n = 8), both venous and capillary specimens (n = 7), or either capillary or venous specimens (n = 6). Another 16 participants (32.0%) reported having other policies for handling critical values, but did not specify if these included repeat testing. There was no significant association found between participants who did or did not have a procedure in place for retesting critical POCG values and retesting rates, P = .34. A total of 27 of 49 participants (55.1%) reported that their clinical laboratory routinely retested specimens if results were in their critical value range.
Table 4 shows the distribution of retesting rates by all facilities. Among 1386 retested specimens, 973 (70.2%) were performed on capillary blood using the same meter, 330 (23.8%) on venous blood using core or satellite laboratory method, 72 (5.2%) on capillary blood with a different meter, and 11 (0.8%) on capillary blood with an unknown (same or different) meter. Table 5 shows aggregate results for retesting rates by location and staff among participants who reported this information. Critical low measurements were retested significantly more often than critical high values (χ2 test; P < .001). In addition, critical high values were retested significantly more often for inpatient and critical care locations when compared with the emergency department (χ2 test; P < .001) and when patient care technicians/aides and nurses performed testing compared with other staff (χ2 test; P < .001).
POCG Critical Value Confirmation
Table 4 shows the distribution of confirmation rates when retested specimens were within 15 mg/dL of initial critical low or within 20% of initial critical high values. Among the critical POCG tests not confirmed, the median repeat value was 101 mg/dL for critical low values and 418 mg/dL for critical high values.
Table 6 shows aggregate results stratified by repeat testing method, location, and staff performing tests in nonneonatal patients. The number of neonatal cases (n = 88) was too low to evaluate these factors. Overall, the percentage of confirmed critical low values was significantly less than that of critical high values (χ2 test; P < .001). In addition, critical high POCG confirmation rates were significantly greater when testing was performed in inpatient and emergency department locations (χ2 test; P = .04) or by patient care technicians (χ2 test; P = .04) or when retesting was performed on capillary blood compared with a venous specimen (χ2 test; P < .001). Confirmation of retested critical low POCG values was significantly higher when testing was performed by patient care technicians/aides (Fisher exact test; P = .04). The median confirmation rate was 62.5% (central 80th percentile range, 45.0%–100%) for the 21 lower–test-volume facilities (<75 000 tests/y), which was significantly lower when compared with a median confirmation rate of 86.8% (median 80th percentile range 67.6%–100%) in 21 higher–test-volume facilities (multiple regression; P = .002). There was no significant difference (Kruskal-Wallis; P = .81) between retest rates by low– and high–test-volume groups; the median rates were 56.0% and 56.2%, respectively.
POCG Critical Value Results Notification
Table 4 shows the frequency of critical POCG cases for which there was documentation that a primary caregiver was notified of results. A total of 48 of 50 participants (96%) reported that there was a policy for notifying responsible clinical personnel for critical POCG results. Of 1488 cases that involved caregiver notifications, 586 (39.4%) were not retested.
COMMENT
Blood glucose results in the critical range can have a major impact on the immediate care and management of patients. Recognition and treatment of severe hypoglycemia is especially important, as its incidence has risen with the use of tight glycemic control protocols for inpatient care.14–16 Because of the high volume of POCG testing and clinical impact in hospital and emergency department settings, it is important to understand factors that affect test performance in the critical value range.
When POCG devices are used as a waived test, confirmation of test performance such as linearity, accuracy, and precision, which is mandatory for moderately complex clinical laboratory testing, is not necessary as long as properly trained and qualified operators follow prescribed and approved procedures. Even when analytical assessments are done on POCG devices, confirming test performance on clinical samples in the extreme high and low ranges is difficult and may require contrived specimens using spiked whole blood.17 Consequently, analytical performance or preanalytical factors associated with sample collection and processing that could affect accuracy of results in the critical range may go unrecognized.6,18
A key observation in this study was that only about half of the POCG measurements in the critical range were retested, and of these, about 19% of repeat measurements did not meet criteria to confirm results. Previous studies conducted in single institutions using similar criteria have shown comparable discrepancy rates when critical POCG measurements were repeated.7–9 In comparison, significant error rates for critical glucose measurements performed in 86 clinical laboratories on venous blood specimens was only 0.5% (13 of 2517).11
Interestingly, rates of repeat testing were not significantly higher among participants with retesting procedures, suggesting that this factor did not affect practice. In addition, interpretative criteria to verify critical POCG results were defined in only about two-thirds of facilities. In contrast, repeat testing to confirm critical results from core laboratory measurements is usually unnecessary.13,14 This may account for why procedures for repeating critical glucose results in participants' clinical laboratories were less frequent than for repeating critical POCG measurements. Together, these observations show that current practices for POCG testing lack uniformity and do not reliably detect erroneous critical results. Guidelines for standardizing procedures and interpreting critical POCG results after retesting are needed. This would reduce the potential for adverse consequences and potential harm from treatment based on unconfirmed or incorrect information.
International Organization for Standardization standards have been used as criteria for determining accuracy of POCG measurements.13 However, these standards were intended to be used to validate home use glucose meters against a reference standard.18 Surveillance error grid analysis,19 which was designed to take into account clinical significance of POCG results, might be a better alternative for assessing accuracy of POCG measurements, although this system was also designed to be used with a reference method. We used 10 minutes as the time limit for retesting because other studies have used 5 to 10 minutes and, in practice, most repeat measurements are completed within the first few minutes.7–9 Additional work is needed to standardize confirmation methods that include optimal time for retesting and criteria used to interpret results.
The type of specimen (capillary, venous, or either) specified for retesting varied widely among those with retesting procedures, although in practice, capillary blood was used about 76% of the time. Surprisingly, repeat testing on capillary blood demonstrated higher confirmation rates for critical high POCG measurements than did testing venous specimens, which is considered the reference method. This may have been due to variation in testing practices among facilities or preanalytical factors associated with specimen collection. This study was not designed to determine if capillary blood is an acceptable specimen to use for confirming critical POCG results. Additional studies should be done to resolve this question because capillary blood appears to be the preferred specimen for retesting. Although obtaining a venous specimen is less convenient and results are less timely, it might nevertheless increase the margin of safety for managing hypoglycemic or hyperglycemic states, especially if POCG results do not correlate with the patient's clinical condition.
One limitation to this study was that information about specific POCG or clinical laboratory methods was not obtained. Although the type of device or reagent strip may have accounted for some of the variability in error rates, POCG measurements have poor precision15 and high error rates6,20 at low and high glucose concentrations with all methods. However, newer point-of-care devices may be more reliable.21,22
The number of POCG tests performed per operator in the median laboratory was relatively low—about 10 per month. A previous study showed that only about 25% of operators encountered a critical POCG result during a 1-year period.7 Thus, the likelihood of POCG operators measuring a critical result is relatively infrequent compared with that of core laboratory technologists, who manage critical results every day. Thus, it would be expected that most POCG operators would have little experience with critical POCG results. This makes visual aids and training especially important to help operators maintain competency with handling critical POCG measurements and reduce risk of harm from inaccurate results.
Finally, this study showed that notifying the patient's caregiver about critical POCG results did not achieve the same level of performance as has been demonstrated in clinical laboratories. In spite of most participants (96%) having policies to notify caregivers about critical POCG results, the median rate of undocumented notifications was 14.3%. In comparison, Valenstein et al23 reported only 10 of 3545 notification failures (0.3%) among 121 clinical laboratories. This difference might partly be explained by the clinical setting (remote laboratory versus bedside) where testing is performed. In addition, as discussed above, operator inexperience with handing POCG testing may be another contributing factor.24 Some notifications may also have occurred without documentation. For example, the most recent CAP checklist question states that the person notified need not be documented when the individual performing the test is the same person who treats the patient.
In summary, these findings have practical implications for improving point-of-care testing practices. It is important to recognize that critical POCG results are sometimes unreliable. Treatment based on erroneous results could harm patients. Because of the risk to patient safety, every critical POCG measurement, even if performed as a waived test, should be confirmed by retesting unless the method has been carefully validated for accuracy at the critical high and low range of results. Interpretive criteria for determining confirmation of initial results should also be defined, as should the type of specimen to use. Because POCG operators rarely encounter critical POCG results, strong training, visual aids, and audits are recommended to ensure compliance with policies and procedures for reliable testing and notification of critical results. Performance standards for laboratories and point-of-care programs should include specifications for competently handling critical POCG results equivalent to what is required for clinical laboratories to reduce risk to patient safety. Finally, this study and others provide evidence to support development of guidelines to assist point-of-care programs in standardizing procedures for reliably handing critical POCG results.