OBJECTIVE

Medication errors involving intravenous medications continue to be a significant problem, particularly in the pediatric population due to the high rate of point-of-care and weight-adjusted dosing. The pharmaceutical algorithm computerized calculator (pac2) assists in converting physician medication orders to correct volumes and rates of administration for intravenous medications. This study was designed to assess the efficacy of the pac2 in simulated clinical scenarios of point-of-care dosing.

Methods

The study design was a within-subject controlled study in which 33 nurses from pediatrics, pediatric critical care, or critical care (mean nursing experience of 10.9 years) carried out various point-of-care medication-dosing scenarios with and without the aid of the pac2.

RESULTS

Use of the pac2 resulted in a significantly higher percentage (mean [95% CI]) of medication volumes calculated and drawn accurately (91% [87–95%] versus 61% [52–70%], p<0.0001), a higher percentage of correct recall of essential medication information (97% [95–99%] versus 45% [36–53%], p<0.0001), and better recognition of unsafe doses (93% [87–99%] versus 19% [12–27%], p<0.0001) as compared to usual practice. The pac2 also significantly reduced average medication calculation times (1.5 minutes [1.3–1.7 minutes] versus 1.9 minutes [1.6–2.2 minutes], p=0.0028) as compared to usual practice.

CONCLUSIONS

The pac2 significantly improved the performance of drug calculations by pediatric and critical care nurses during simulated clinical scenarios designed to mimic point-of-care dosing. These results suggest that the pac2 addresses an area of safety vulnerability for point-of-care dosing practices and could be a useful addition to a hospital's overall program to minimize medication errors.

Medication errors are a serious safety problem in hospitalized patients. The American Academy of Pediatrics Committee on Drugs and Committee on Hospital Care defined a medication error as any preventable event that occurs in the process of ordering or delivering a medication, regardless of whether an injury occurred or the potential for injury was present.1 The reported incidence of medication errors in hospitalized adults ranges from less than 1% to as high as 56% of all medications administered with the estimation that medication errors occur in 6% of all adult patients.2 The reported rates of medication errors in pediatric patients are often higher with error rates as high as 1 per 6.8 (14.7%) of children admitted to a Neonatal or Pediatric In tensive Care Unit (PICU).3 Takata et al. reported an adverse medication event rate of 11.1 per 100 patients in 12 children's hospitals in the United States.4 The potential impact of such medication errors in the pediatric population is especially worrisome since medication errors in pediatric patients are three times more likely to result in a clinically important adverse event.5 Incorrect dosing is the most commonly reported medication error in pediatrics including errors in the computation of dosage and dosing interval.4 ,6–10 Pediatric studies have noted that the majority of medication errors occur either during the prescribing11,12 or administration stage.3,9,13,14 

The unstable infant or child in an emergency department or ICU requires immediate pointof-care medications. Figure 1 is an outline of a medication administration sequence, with interdisciplinary responsibilities, for point-of-care medication dosing in hospitalized patients. The figure demonstrates the critical and complex role of the bedside nurse in this process. In addition to medication administration, the nurse is often responsible for traditional pharmacy activities including preparation, dispensing, and recognition of incorrect doses.

Figure 1

Typical Process and Individual Responsibilities for Point-of-Care Medication Dosing of Hospitalized Patients

Figure 1

Typical Process and Individual Responsibilities for Point-of-Care Medication Dosing of Hospitalized Patients

Close modal

An example of a point-of-care administration sequence involves the acute resuscitation of child with shock. The physician orders a medication in terms of dose/kg/time (dopamine 10 mcg/kg/ min by continuous infusion or epinephrine 0.01 mg/kg IV push). The bedside nurse must then quickly and correctly translate the physician's order into an intravenous flow rate (volume/ time) taking into account the patient's weight, the concentration of the available medication in the solution, and the ordered dose. It is estimated that nearly 20% of all hospital medications are dispensed at the point-of-care without first being measured in the hospital pharmacy.15 Nursing staff must be proficient in performing these mathematical calculations. There have been multiple studies over the past three decades which have documented an unacceptably high calculation error rate by practicing nurses.16–19 

The Pharmaceutical Algorithm Computerized Calculator (pac2) was conceptualized to assist with calculations for point-of-care dosing. The pac2 automatically performs all unit conversions (e.g., lb to kg or mg to mcg) while calculating the correct infusion rate (volume per time) based upon the concentration of the medication solution, the child's weight, and the physician's prescribed dose (Figure 2). This handheld computer captures common entry errors such as misplaced decimal points and improper expression factors. The pac2 can be custom programmed to provide an immediate visual warning if the input medication order falls outside the hospital's established limits for the prescribed medication, a given age range (e.g., adult versus pediatric), or care unit. The current study prospectively evaluates the efficacy of the pac2 in reducing medication-dosing errors when practicing pediatric and critical care nurses prepared and administered intravenous medications during simulated resuscitations of critically ill children.

Figure 2

Screen of the Pharmaceutical Algorithm Computerized Calculator.

Figure 2

Screen of the Pharmaceutical Algorithm Computerized Calculator.

Close modal

Study Participants

Nurses from three different institutions volunteered to participate in the study. Participants needed to be a registered nurse, involved in direct patient care, and currently working in a pediatric, pediatric critical care, or critical care setting. Those nurses participating were willing to comply with the requirements of the protocol. The study was approved by the local institutional review board.

Two separate groups of nurses were enrolled for participation in the study: the full study group and the learning effect group. The full study group included all nurses that were enrolled into the study with the intent of performing both sets of scenarios, first without and then with the pac2. The learning effect group consisted of a subset of different nurses who were selected to reflect the demographic/experience profile of the full study group. This group only performed one set of scenarios using the pac2. The intent of this group was to assess the potential learning bias in the full study group from the consistent usage of the pac2 in the second set of scenarios. It was reasonable to expect the full study group's pac2 results to outperform the learning effect group's pac2 results if a learning bias existed. The possibility that prior experience using the pac2 might impact performance existed since some nurses in the full study group had evaluated the pac2 prior to this study and had some limited experience operating the device. Analyses of the full study group minus the nurses with prior experience (i.e., naïve subgroup) were therefore performed.

Study Design

The study design was a within-subject controlled study. The nurses were randomized to perform one of two simulation scenario sets (see below) without the pac2 during the first phase of the study. The calculation of the medications was performed using their usual practice. Usual practice allowed the use of handheld calculators, pre-calculated pediatric emergency drug cards, and other supportive materials typically available at point-of-care dosing. Usual practice was assessed first to control for a learning effect from using the pac2 initially. The nurses then watched a 20-minute instructional tutorial describing the function and operation of the pac2 after completing the first set of scenarios without the pac2. The nurses were allowed to ask questions during and after the tutorial. The nurses were then crossed over to perform the alternative scenario set using the pac2 during the second phase of the study after watching the tutorial. Those nurses who performed scenario set A during the first phase of the study performed scenario set B during the second phase of the study and vice-versa. The performance of each nurse during the two sets of simulated dosing scenarios was evaluated.

Clinical Scenarios

Scenario set A included the case of a pediatric patient with anaphylaxis and impending respiratory arrest and a second pediatric patient with seizures who required a series of three medication doses including one continuous intravenous medication infusion. Scenario set B included a different emergency situation with a pediatric patient with status epilepticus who progressed to respiratory arrest requiring rapid sequence intubation with resulting hypotension and a second pediatric patient in severe pain who required a series of three medication doses including one continuous intravenous medication infusion. Each set of scenarios required the nurses to perform 12 dose volume calculations (i.e., make eight mcg to mg conversions, compute three continuous IV rates and one non-weight based order), recognize five unsafe doses (ten or more times above or below acceptable ranges), and recall seven critical pieces of medication administration information. The scenarios were designed to be of equal degrees of difficulty. Three trained nursing school faculty observers followed a written script to oversee and score the nurse's performance that was recorded on a case report form and measure the time required to complete the calculation task.

Study Endpoints

The primary efficacy endpoint was predefined as the percent of medication volumes calculated and drawn accurately (i.e., ± 10% correct medication volume) into syringes. Some results are presented as the percent error rate (e.g., percent error rate equals 100% minus the percent of medication volumes calculated and drawn accurately). The secondary efficacy endpoints included: percent of medication volumes calculated and stated accurately, percent of unsafe medication doses identified, percent of essential medication administration information recalled, average medication calculation time, and average time to complete the scenario. A composite score was calculated as the unweighted average of percent of volumes calculated and drawn accurately, percent of unsafe doses identified and percent of essential information recalled.

Statistical Analysis

Results are reported as mean (95% CI) percentages. The percentage of accurately drawn volumes with and without the pac2 was determined for each nurse. A paired t-test was used to compare the average percentage of correct volumes without the pac2 to the average percentage of correct volumes with the pac2. A paired t-test was used to compare results with the pac2 (average percentage of accurately stated volumes, percentage of unsafe doses recognized, percentage of essential information recalled, average medication calculation time, average overall scenario time and a composite score) to results without the pac2. The significance of the average paired difference was tested at a=0.05 (SAS 9.1.3, SAS Institute, Inc, Cary, NC).

The study cohort included 39 nurses, 33 of whom participated in the full study group and six in the learning effect group. The two groups were comparable with respect to age, sex, total nursing experience, and specialty nursing experience (Table).

Table 1

Summary of Subject Demographics

Summary of Subject Demographics
Summary of Subject Demographics

Full Study Group

There was no difference in any of the efficacy endpoints for the naïve subgroup relative to the full study group and therefore the following data includes the 33 nurses who comprised the full study group. Use of the pac2 resulted in a significantly higher rate of medication volumes that were drawn up accurately (91% [87–95%] versus 61% [52–70%]; p < 0.0001), a significantly higher percentage of essential information recalled (97% [95–99%] versus 45% [36–53%]; p < 0.0001), and a significantly higher percentage of recognition of unsafe doses (93% [87–99%] versus 19% [12–27%]; p < 0.0001) as compared to usual practice (Figure 3). These findings were consistent irrespective of the order of the scenario sets (A or B) and were consistent across all nursing subgroups (pediatric, critical care or pediatric critical care). Similar results were observed for the endpoint of medication volumes stated accurately.

Figure 3

Comparison of Outcome Variables for Simulated Scenario Sets with and without the Pharmaceutical Algorithm Computerized Calculator.

Without PAC2, With PAC2, Percent correct response with 95% confidence intervals, * = Significance of paired difference, p<0.0001

Figure 3

Comparison of Outcome Variables for Simulated Scenario Sets with and without the Pharmaceutical Algorithm Computerized Calculator.

Without PAC2, With PAC2, Percent correct response with 95% confidence intervals, * = Significance of paired difference, p<0.0001

Close modal

Use of the pac2 resulted in a significantly higher composite score (94% [91–97%] versus 42% [37–46%]; p < 0.0001) as compared to usual practice (Figure 3). The average composite error rate without the pac2 was 58% compared to 6% with the pac2. All three of the setting subgroups (pediatric, critical care, or pediatric critical care) demonstrated a significant increase in the correct composite score with the pac2.

Use of the pac2 resulted in faster medication calculation time (1.5 minutes [1.3–1.7 minutes] versus 1.9 minutes [1.6–2.2 minutes], p = 0.0028) as compared to usual practice (Figure 4). Although a faster calculation time was observed in all nursing subgroups, it was not consistent across both randomized groups. Those that had the A scenarios first showed no difference while those that had the B scenarios first did. This finding may be reflective of the difference in the three medications that were selected for timing from each scenario set.

Figure 4

Average Drug Calculation Times (minutes) with and without the Pharmaceutical Algorithm Computerized Calculator.

Without PAC2, With PAC2, Average drug calculation time (minutes) with 95% confidence intervals, * = Significance of paired difference

Figure 4

Average Drug Calculation Times (minutes) with and without the Pharmaceutical Algorithm Computerized Calculator.

Without PAC2, With PAC2, Average drug calculation time (minutes) with 95% confidence intervals, * = Significance of paired difference

Close modal

The pediatric critical care subgroup of nurses performed the medication calculations most quickly with no significant difference between using the pac2 or not. However, both of the other nursing subgroups performed comparably and demonstrated a significant decrease in medication calculation time for the scenarios with pac2. All three nursing subgroups were comparable in medication calculation time with the pac2, but the pediatric critical care subgroup tended to be quicker without the pac2. The nurses also completed the scenario sets significantly faster with the pac2 as compared to usual practice (19.4 minutes [17.9–21.0 minutes] versus 22.9 minutes [20.8–25.0 minutes]; p = 0.0012). This finding was consistent irrespective of the order of scenario sets (A or B) and was consistent across all nursing subgroups.

Learning Effect Group

There was no evidence that participation in the simulations alone during the first phase of the study resulted in a learning effect that would carry over to improved performance when using the pac2. The learning effect group, which only used the pac2 without prior completion of a simulation, demonstrated higher percentages of accurately drawn volumes (93% [86–100%]), accurately stated volumes (93% [86–100%]), essential information recalled (100% recalled; no variation among subgroup), and recognition of unsafe doses (97% [88–100%]) as compared to the nurses evaluated without and with the pac2 in the full study group. The learning effect group also completed the scenarios more quickly (15.0 minutes [12.2–17.8 minutes]) than the full study group (19.4 minutes [17.9–21.0 minutes]).

Many process changes and newer technologies have been reported to reduce medication errors rates. Ward-based clinical pharmacists,6 individual unit-dosing, computerized physician order entry,20,21 preprinted order sheets,22 standard infusion concentrations plus “smart-pump” technology,23 and a computer-based medication and equipment calculator24 have been reported to reduce medication errors. However, the computer-based medication and equipment calculator was the only safety measure/device that attempted to address administration errors that occur during point-of-care dosing during resuscitations. Shannon and colleagues24 tested a computerized calculator for accuracy and quickness compared to conventional paper-based calculation methods. Subjects made significantly fewer errors and were three times faster in performing theoretical resuscitation calculations for pediatric and adult patients with this resuscitation calculator. Similarly, the pac2 also improved accuracy and reduced the average medication calculation time and overall time to complete the scenario as compared to usual practice. Some of the potential advantages of the pac2 versus the computerized calculator include the portability of the handheld pac2, the large number of medications stored (285 vs. 21) including non-resuscitation medication infusions, the availability of essential medication information, and customized warnings for doses outside the institution's typical ranges. Advantages of the computerized resuscitation calculator include the feature that all 21 medication doses are calculated simultaneously, with increased ease of use, and likely a lower cost.

The only other process/device demonstrated to reduce medication administration errors at pointof-care was a combination of standard medication concentrations, “smart-pump” technology, and user-friendly labels. Larsen and colleagues23 demonstrated that the combination of these features significantly reduced continuous medication infusion errors. The pac2 is complementary to the “smart-pump” because it has distinctive features not shared with this technology, including dose calculation of manual intravenous, intramuscular, and subcutaneous injections and dose volume calculations for point-of-care medication liquids, elixirs, and suspensions. The error rate in medication volumes observed in our study was seemingly high at 39%, but is supported by other published investigations. Bayne and Bindler tested 67 practicing nurses with a 20-item medication calculation test and found that the average correct score was 75.5%.18 In a study of 546 nurses working at a university hospital in Finland, only 56% answered the “volume of solution” dosage calculation correctly.19 Bindler and Bayne documented a 48% error rate for intravenous calculation questions among a sample of 110 nurses.25 Kozer and colleagues found a 20% error rate in 16% (9/58) of syringes examined after eight mock resuscitations.26 They concluded that many administration errors could only be detected by analyzing the syringe content. Combining the added “stress” of a simulated resuscitation plus a predefined error rate of ± 10% likely contributed to our seemingly high medication volume error rate of 39%.

The major limitation of this study is that it was conducted under simulated conditions rather than actual clinical conditions where other factors may influence performance and outcomes. The increased mortality reported by Han and colleagues after the implementation of a computerized physician order entry system serves as a good example.27 Some factors attributed to this negative outcome included significant changes in clinical workflow, communication among providers, and distribution of responsibilities.28 Effectiveness of the pac2 may not be susceptible to these factors in actual clinical practice but other factors that may influence the impact of the pac2 include greater time pressure and stress during acute resuscitations, increased susceptibility of severely ill children to medication errors,6,11 and unpredictable logistical problems that arise during emergency situations. Simulation studies are commonly used in medicine for educational and training purposes,29 and the controlled environment of a simulation laboratory was appropriate for this initial study to assess the efficacy of the pac2 by nursing staff. Our simulation study specifically did not include the presence of a clinical pharmacist as we wished to mimic emergency resuscitations in which nurses need to function without this benefit. Another limitation of the study is that it was not possible to blind the observers regarding use of the pac2. However, the only outcome measure directly controlled by the observers was timing of the exercise. There are certain types of medication administration errors that were not and cannot be addressed by the pac2. These errors include missed doses, prescribing the wrong medication, and administration at the wrong time. Finally, calculating medication doses based on a patient's weight is not usual practice for adult critical care nurses. Interestingly, the medication calculation times were not significantly different across the three nursing subgroups.

This study provides the first controlled evaluation of a novel device that addresses an area of safety vulnerability in point-of-care dosing. Our results demonstrated the reduction in medication calculation errors that can be obtained by nurses using a pac2 in a simulated pediatric patient care setting. Use of the pac2 device reduced the volume calculation error rate by nurses from 39% to 9%, allowed the nurses to recognize unsafe doses more frequently, and recall essential medication information more effectively than during performance without the pac2. These improvements were consistently observed from nurses across pediatric, critical care and pediatric critical care settings (with the exception of medication calculation time). Future research to collect information under conditions of actual use in clinical practice would be helpful to define the full range and precise role of this device in improving the accuracy of dosing within the hospital.

This study was supported by InformMed, Inc., and the Department of Pediatrics, University of Illinois College of Medicine at Peoria.

DISCLOSURE The authors declare no conflicts or financial interest in any product or service mentioned in the manuscript, including grants, equipment, medications, employment, gifts, and honoraria.

pac2

Pharmaceutical Algorithm Computerized Calculator

PICU

Pediatric Intensive Care Unit

1
American Academy of Pediatrics
Committee on Drugs and Committee on Hospital Care. Prevention of medication errors in the pediatric inpatient setting.
Pediatrics
2003
.
112
:
431
436
.
2
Krahenbuhl-Melcher
,
A.
,
R.
Schleinnger
,
M.
Lampert
, et al
.
Drug-related problems in hospitals. A review of the literature.
Drug Safety
2007
.
30
:
379
407
.
3
Raju
,
T. N. K.
,
S.
Kecskes
,
J. P.
Thornton
, et al
.
Medication errors in neonatal and paediatric intensive care units.
Lancet
1989
.
334
:
374
376
.
4
Takata
,
G. S.
,
W.
Mason
,
C.
Taketomo
, et al
.
Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children's hospitals.
Pediatrics
2008
.
121
. .
5
Kaushal
,
R.
,
D. W.
Bates
,
C.
Landrigan
, et al
.
Medication errors and adverse drug events in pediatric inpatients.
JAMA
2001
.
285
:
2114
2120
.
6
Folli
,
H. L.
,
R. L.
Poole
,
W. E.
Benitz
, and
J. C.
Russo
.
Medication error prevention by clinical pharmacists in two children's hospitals.
Pediatrics
1987
.
79
:
718
722
.
7
Chedoe
,
I.
,
H. A.
Molendijk
,
S. T. A. M.
Dittrich
, et al
.
Incidence and nature of medication errors in neonatal intensive care with strategies to improve safety. A review of the current literature.
Drug Safety
2007
.
30
:
503
513
.
8
Walsh
,
K. E.
,
C. P.
Landrigan
,
W. G.
Adams
, et al
.
Effect of computer order entry on prevention of serious medication errors in hospitalized children.
Pediatrics
2008
.
121
.
Available at: www.pediatrics.org/cgi/doi/10.1542/peds.2007-0220. Accessed June 30, 2010
.
9
Cowley
,
E.
,
R.
Williams
, and
D.
Cousins
.
Medications errors in children: a descriptive summary of medication error reports submitted to the United States Pharmacopeia.
Curr Ther Res Clin Exp
2001
.
62
:
627
640
.
10
Leape
,
L. L.
,
D. W.
Bates
,
D. J.
Cullen
, et al
.
Systems analysis of adverse drug events.
JAMA
1995
.
274
:
35
43
.
11
Kozer
,
E.
,
D.
Scolnik
,
A.
Macpherson
, et al
.
Variables associated with medication errors in pediatric emergency medicine.
Pediatrics
2002
.
110
:
737
742
.
12
Fortescue
,
E. B.
,
R.
Kaushal
,
C. P.
Landrigan
, et al
.
Prioritizing strategies for preventing medication errors and adverse drug events in pediatric inpatients.
Pediatrics
2003
.
111
:
722
729
.
13
Vincer
,
M. J.
,
J. M.
Murray
,
A.
Yuill
, et al
.
Drug errors and incidents in a neonatal intensive care unit. A quality assurance activity.
AJDC
1989
.
143
:
737
740
.
14
Suresh
,
G.
,
J. D.
Horbar
,
P.
Plsek
, et al
.
Voluntary anonymous reporting of medical errors for neonatal intensive care.
Pediatrics
2004
.
113
:
1609
1618
.
15
Pederson
,
C. A.
,
P. J.
Schneider
, and
D. J.
Scheckelhoff
.
ASHP national survey of hospital pharmacy practice in hospital settings: dispensing and administration – 2002.
Am J Health Syst Pharm
2003
.
60
:
52
68
.
16
Conti
,
A.
and
P. G.
Beare
.
Performance on a mathematics/drug calculation test: relationship to subsequent reported medication errors.
J Nurs Staff Dev
1988
.
4
:
54
58
.
17
Bliss-Holtz
,
J.
Discriminating types of medication calculation errors in nursing practice.
Nurs Res
1994
.
43
:
373
375
.
18
Bayne
,
T.
and
R.
Bindler
.
Effectiveness of medication calculation enhancement methods with nurses.
J Nurs Staff Dev
1997
.
13
:
293
301
.
19
Grandell-Niemi
,
H.
,
M.
Hupli
,
H.
Leino-Kilpi
, and
P.
Puuka
.
Medication calculation skills of nurses in Finland.
J Clin Nurs
2003
.
12
:
519
528
.
20
King
,
W. J.
,
N.
Paice
,
J.
Rangrej
, et al
.
The effect of computerized physician order entry on medication errors and adverse drug events in pediatric inpatients.
Pediatrics
2003
.
112
:
506
509
.
21
Potts
,
Al
,
F. E.
Barr
,
D. F.
Gregory
, et al
.
Computerized physician order entry and medication errors in a pediatric critical care unit.
Pediatrics
2004
.
113
:
59
63
.
22
Kozer
,
E.
,
D.
Scolnik
,
A.
Macpherson
, et al
.
Using a preprinted order sheet to reduce prescription errors in a pediatric emergency department: a randomized controlled trial.
Pediatrics
2005
.
116
:
1299
1302
.
23
Larsen
,
G. Y.
,
H. B.
Parker
,
J.
Cash
, et al
.
Standard drug concentrations and smart-pump technology reduce continuous-medicationinfusion errors in pediatric patients.
Pediatrics
2005
.
116
.
Available at: www.pediatrics.org/cgi/doi/10.1542/peds.2004-2452. Accessed June 30, 2010
.
24
Shannon
,
T.
,
A.
Ratchford
,
D.
Southward
, and
A.
Hildreth
.
The development of a computerized equipment and drug calculator for use in resuscitation.
Emerg Med J
2002
.
19
:
215
218
.
25
Bindler
,
R.
and
T.
Bayne
.
Medication calculation ability of registered nurses.
IMAGE: J Nurs Scholarship
1991
.
23
:
221
224
.
26
Kozer
,
E.
,
W.
Seto
,
Z.
Verjee
, et al
.
Prospective observational study on the incidence of medication errors during simulated resuscitation in a paediatric emergency department.
BMJ
doi:10.1136/bmj.38244.607083.55 (published September 28, 2004). Accessed June 30, 2010
.
27
Han
,
Y. Y.
,
J. A.
Carcillo
,
S. T.
Venkataraman
, et al
.
Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system.
Pediatrics
2005
.
116
:
1506
1512
.
28
Rosenbloom
,
S. T.
,
F. E.
Harrell
,
C. U.
Lehmann
, et al
.
Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system (letter to the editor).
Pediatrics
2006
.
117
:
1452
1454
.
29
Shapiro
,
M. J.
,
J. C.
Morey
,
S. D.
Small
, et al
.
Simulation based teamwork training for emergency department staff: does it improve clinical team performance when added to an existing didactic teamwork curriculum?
Qual Saf Health Care
2004
.
13
:
417
421
.