Lessons from a Successful Implementation of a Computerized Provider Order Entry System

Electronic health record systems are attractive to many institutions as they offer the potential to improve patient safety, clinician and patient care efficiency, cost effectiveness, education, and regulatory compliance.1 Computerized provider order entry (CPOE) represents a significant component of the in-patient electronic health record. Nevertheless, a 2003 report suggested that less than 5% of United States hospitals have adopted CPOE.2 Converting an institution from a paper base to CPOE represents a significant culture change and many associated challenges. Reasons for implementation failure of new informatics systems include inadequacies of existing software, financial burden on the institution, lack of user satisfaction, and unanticipated systematic and cultural problems.3 Attempts to sustain newly implemented CPOE systems have failed altogether at some institutions.4 

Cincinnati Children's Hospital Medical Center (CCHMC) has successfully implemented an inpatient Integrating Clinical Information System (ICIS) in which CPOE was a major component. Prior to success, the implementation encountered both anticipated and unanticipated challenges. Lack of preparation for unanticipated issues has been associated with significant challenges to success experienced at other institutions.5,6 In this article, we describe some of the major unanticipated challenges associated with implementing ICIS at CCHMC, discuss how these challenges were overcome, and suggest appropriate design and implementation strategies to address these issues prior to implementing such systems in other organizations.

CCHMC is a freestanding 475-bed tertiary care teaching hospital with over 760,000 patient encounters each year. CCHMC employs nearly 8,500 individuals including 1,113 active medical staff. In 2002, CCHMC customized and implemented a commercially available CPOE platform in order to optimize patient safety and consistency in care, improve clinician and patient care efficiency, maximize regulatory compliance and enhance provider, patient and family satisfaction. The primary components of ICIS include CPOE, Clinical Documentation (CD), an electronic Medication Administration Record (MAR), a Lifetime Clinical Record (LCR) for data storage, and a web-based portal that integrates these modules with existing applications. All primary components were enhanced and redesigned from the Invision software platform (Siemens Medical Solutions, Malvern, PA). The hospital board of directors, Chief Executive Officer, and senior executives established the expectation that all clinicians would transition to the use of the ICIS in the care of inpatients. By April of 2002, ICIS was piloted in two medical-surgical inpatient units within the hospital. Less than eight months later the system was successfully implemented in all inpatient care areas with the exception of the Blood and Marrow Transplant and Hematology/Oncology units (implemented in December of 2003). Physicians, advanced practice nurses, and registered nurses enter an average of 30,000 orders each week into ICIS, of which 40% are pharmacy and 60% non-pharmacy orders. Physicians and advanced practice nurses enter 90% of these orders directly into the computer, while 10% of orders are entered by nurses who receive them verbally. Twenty-four hour ICIS onsite support is available for clinicians, and clinician feedback forms the basis of system design and enhancement.

During the implementation period we experienced several unanticipated challenges. We categorized these challenges in nine discrete areas: 1) Deterioration in communication; 2) Excessive number of system alerts to ICIS users; 3) Unrecognized discontinuation of medications; 4) Unintended loss of orders; 5) Loss of orders during implementation of a new unit; 6) Amplification of errors; 7) Unintentional generation of actual patient care orders by system analysts; 8) Persistence of specific patient care order instructions; and 9) Verbal patient care orders entered under the incorrect clinician.

Data for this paper were derived from user-generated feedback within ICIS. Feedback can be generated by any individual working within ICIS. Feedback reports were reviewed by the investigators and placed into categories based on root cause of the issue. Recurring issues or issues which led to potential or actual patient injury are included in this review. In addition to the authors, the ICIS Leadership Team and Physician Design Group (a supervisory board of physicians tasked with approving ICIS design for the institution) were involved in addressing the unanticipated challenges.

Excellent medical care requires a high level of communication between care team members. Notably, the clinician must converse frequently with nurses and other bedside caregivers. However, there were unexpected breakdowns in communication after the ICIS was implemented. Prior to ICIS implementation, clinicians were physically present at the bedside to write orders, and communicated with nursing staff immediately when new orders were written. With ICIS, clinicians were able to enter new orders or revise orders from anywhere in the hospital. Once an order is generated it appears instantly in the patient's electronic MAR, avoiding the written transcription process (Figure 1). A new order flag was created to alert caregivers of new orders. However, the flag still requires the nurse to log onto the computer on a regular basis to see the new order flag (a minimum of every 2 hours at CCHMC). When a clinician changes an order from another location and does not verbally notify nursing staff of the change, the old treatment plan can continue until the nursing staff checks the system, possibly up to two hours later. CCHMC experienced a number of occurrences where orders were not carried out in a timely manner due to dependence on the ICIS as sole means of communication, particularly orders that were entered shortly after the care team had met. In addition, orders for STAT testing such as bedside radiographs, electrocardiograms and respiratory therapy treatments were initially routed to these technicians by printing in their respective work areas. These printed orders were often augmented by a phone call or page to avoid delays.

To ensure prompt communication regarding order changes, a widespread education effort was put in place for new and existing ICIS users. The focus of the effort was to emphasize that entering an order into ICIS was not a substitute for oral communication with the clinicians caring for that patient. In addition, new programming was added to ICIS to allow STAT radiograph, electrocardiogram and respiratory therapy orders to be routed directly from CPOE to the technician's pager. As a result, communication between prescribers, nurses, respiratory therapists, and radiology and electrocardiogram technicians has greatly improved.

A primary goal of ICIS at CCHMC is to improve patient safety. The system alerts clinicians of problems related to potential medication errors including: 1) Medication-allergy interactions; 2) Orders which duplicate a generic drug previously ordered; 3) Orders which duplicate the same therapeutic classification of a previously ordered medication; 4) Drug-drug interactions; and 5) Medication dosages outside recommended ranges for the patient's weight. All alert categories were fully activated when ICIS was implemented. However, we discovered that orders with no errors could generate one or more unwarranted alerts. For example, the computer alerted a clinician ordering potassium in intravenous fluid that the potassium could have a potential interaction with a diuretic medication. Another order for a normal saline fluid bolus generated a generic duplicate alert due to the presence of sodium in the patient's existing maintenance intravenous fluid. Excessive alerting was most prevalent in the intensive care units where use of multiple sedation agents, vasoactive medicines or antibiotics is common. During the first 1.5 million orders generated in the system, two significant errors occurred. One was the prescribing and administering of a ten-fold, followed by a twenty-fold overdose of intravenous adenosine. The patient was unharmed, though the potential for serious injury with other emergent medications was illustrated by this incident. The resident who generated the order was following the instructions of a clinical fellow, and stated he overrode the alert because the fellow had verbally stated the desired dose. He also noted that alerts were so common, even for orders which were correct, that he did not regularly read alerts. The second error involved a physician entering an order for acetaminophen 325 tablets (instead of selecting milligrams) every six hours. The system alerted the physician of the possibility of an overdose. However the physician overrode this alert and proceeded with the order that could have led to a patient receiving over 100 grams of acetaminophen every six hours- easily a toxic dose. This error was intercepted by a pharmacist. While the patient did not receive an overdose, this mistake again demonstrated the seriousness of ignoring alerts. It has been recognized that physicians will ignore alerts if they are too frequent and generally viewed as irrelevant.7 In one study of physician practice, over 90% of drug allergy alerts and over 89% of high-severity drug interaction alerts were overridden.8 It was clear from the experiences at CCHMC that the ICIS system had too many unhelpful alerts.

A survey was placed within the ICIS alerting system. Over a two-week period, each alert was tagged with mandatory buttons asking the clinician to click whether the alert was “helpful” or “not helpful” (Figure 2). The number of alerts labeled “not helpful” during the initial query was 73%. At the recommendation of the Alerts Task Force, the alerts generated in ICIS were made more restrictive. Many drug interaction and generic duplication alerts which were minimally helpful were removed from the system and some alerts were tailored to be location specific. For example, the continued warning of therapeutic duplication of sedative drugs was removed for orders entered in the intensive care units, but retained for orders on the general floors. The medication-allergy checking and medication dose-range checking alerts were not changed. Many potentially dangerous medicines such as digoxin, acetaminophen or morphine currently have absolute blocks so that a clinician cannot override an alert for an excessively high dose of a medicine, but must revise the dose (Figure 3). Additionally, the system was modified to default to the appropriate form of medication. For example, when acetaminophen 325 mg is selected, the dose defaults to mg, while the dosage form defaults to TABLET. The user no longer has the option to change the dose from mg to Tabs (Figure 4). After implementing these changes, the alerts labeled “not helpful” were decreased to 55%.

CCHMC restricts the duration of orders for narcotics and antibiotics (2 days and 7 days, respectively). Prior to implementation of ICIS, if a reason for a non-standard duration was not included with the order the pharmacist would call the clinician to clarify the reason. All CCHMC policies restricting duration of treatment were incorporated into ICIS. When a clinician orders an antibiotic or narcotic, the computer auto-populates the standard duration of therapy consistent with policy. All active medication orders are displayed together; however, 24 hours prior to expiration, antibiotic and narcotic orders are re-displayed under a separate banner titled “Orders Approaching Expiration.” When orders appear under this banner, the clinician has a 24-hour period to renew the medication before it automatically expires in the system. A clinician desiring a change in the duration of an order can refuse the default duration and indicate a non-standard duration. However, immediately after ICIS implementation the pharmacy provided a patient the standard antibiotic duration on any order which specified ‘until discontinued’ and a call was not made to the prescribing clinician to notify him or her of the change. This miscommunication was discovered when clinicians realized that some antibiotic orders were discontinued prematurely.

Clinicians were educated on CCHMC policy regarding default durations on antibiotics and narcotics. This information was conveyed by ICIS support staff through individual feedback to clinicians who generated complaints about the automatic discontinuation of their orders. It was also highlighted in new clinician ICIS training. Pharmacists were educated that a non-standard duration of an antibiotic or narcotic represented an active selection by a clinician, and that a call to the ordering clinician was required if pharmacy is performing a duration change. ICIS was reprogrammed so that any order which was to expire within 24 hours appears at the top of the ICIS rounds report for each patient (Figure 5), clearly reminding clinicians to renew or update orders approaching expiration. The rounds report is heavily relied upon during clinician rounds, and having the orders approaching expiration at the top of this screen has been effective in minimizing the number of times that orders expire without clinician awareness. Additionally the standard antibiotic duration was changed from 7 days to 14 days and the schedule II controlled substance standard duration was extended from 2 days to 5 days.

With paper order systems, a clinician writes an order with an associated date and time. When that order is transcribed to the medication administration record, decisions regarding the medication administration times are made by the nurse. Similarly, in ICIS, the medication order has an associated start date and time which auto-populates the electronic medication administration record. A potential challenge was noted when nursing staff needed to change the timing of a dose due to insufficient intravenous access, conflicting diagnostic testing, delays in delivery of the medications, etc. ICIS allows nursing staff the option of altering the administration times of the medication. However, canceling the first intended administration time to replace it with the desired administration time occasionally resulted in the removal of one administration time occurrence (one dose) for that medication. This resulted in the medication being discontinued one dose earlier than ordered. Until this problem was corrected, it was common for antibiotics to stop one dose early. Another challenge occurred with doses that were entered into ICIS to be administered a certain number of times each day (i.e., BID orders). This issue was related to institutionally-designated standardized administration times (i.e. BID is 0900 and 2100). The system defaults to the next standardized administration time for the frequency ordered. For example, if a patient was being admitted to the hospital and was on a regular BID medication, the order needed to be entered in the ICIS before 2100 or the computer would not generate an occurrence of this medication until the following morning at 0900.

For the first issue, ICIS was reprogrammed to allow nursing staff to adjust medicine dosing more easily and not allow a dose to be inadvertently canceled. The education team concentrated on this issue during new nursing staff ICIS classroom sessions and made an online teaching module to re-educate other members of the nursing staff. Additionally, because this is a frequently used function, this training module was added to the education competencies which are periodically required for all nursing staff. The second issue was addressed by educating prescribers on how ICIS interpreted the institutionally-designated standardized administration times. In addition, upon selection of a frequency like BID or TID, the system now displays for the prescriber the next administration time for the medication. If the first administration time is not acceptable, the prescriber can select a priority of “1st dose stat” to schedule the medication for immediate administration and subsequent doses timed from that point forward.

Computerized systems will have times when the platform is not functional. Most commonly this can be due to scheduled software updates, but it can also occur during the implementation phase, when some units or departments are “live” and some are not. When a new unit was in the process of going “live,” the current active written orders were entered into the ICIS for the first time. This process included disabling the routing of newly generated electronic orders to receiving departments in order to avoid duplication of previously filled written orders. As a result, orders entered electronically were available for online review without active routing to receiving departments (e.g., laboratory, pharmacy, radiology, etc.). A challenge was recognized when a patient's orders were “lost” during a unit's transition. In this unique case, the patient pre-operatively resided in the unit in transition, and as stated above, all order routing had been turned off during the back loading of active orders for that unit. According to the computer, the patient continued to reside on the transition unit while physically being in the operating room. The clinician in the operating room entered heparin infusion orders into the computer for the patient for post-operative care. However, because the order routing was disabled, the pharmacy never received the order and heparin therapy was delayed a number of hours. The patient developed vascular thrombosis and had to return to the operating room for thrombectomy.

When new units were going “live,” a block was put into the system that would prevent clinicians from placing orders into ICIS during the unit transition period. Even after full implementation, electronic health records will always have “down time,” and orders may be lost during this time. As a safety measure, the pharmacy now matches “down time” paper orders with the ICIS generated orders to avoid any duplication or loss of orders.

ICIS has comprehensive disease, diagnosis and procedure-based order sets built into the system. Order sets save time by allowing the clinician to input groups of orders from a list of diagnoses or procedures. Additionally, the order sets provide a guide so that complex orders, such as post-operative transplant patient orders, are standardized and not overlooked. A challenge arose when an error was included in the design of an order set. The standard admission order set for the neonatal intensive care unit contained an erroneous fluid order. The intended order was for D10W 1/4 NS + 10 mEq/L KCL to be delivered in a 250 mL bag. However, the order was designed for ICIS as D10W 1/4 NS + 10 mEq KCL in a 250 mL bag – four fold as much potassium as intended. Any error built into an order set has the potential to be repeated on every patient for which the order set is used.

All order sets must now pass a systematic review process. First, the division or department submits a set of paper orders to the Forms Committee who reviews the order set for content and format. After approval, the order set is eligible for construction into the ICIS test system. The requesting division or department owner evaluates the order set within the test system and gives the final written approval prior to the order set being moved into the active ICIS system.

Information Services (IS) personnel must have full access to the system, as they are responsible for implementing design decisions, troubleshooting problems, and testing the functionality of the system. However, an error can occur while IS personnel are assisting a clinician and real, though unintended, orders are entered for actual patients. This challenge was recognized when an IS team member believing she was in the test system placed a medication order for a real patient. Pharmacy did not recognize that the order was not signed by a prescriber and sent the medication to the nursing unit for administration. When an alert nurse questioned the order, she recognized that the signature on the order was one of a systems analyst rather than a licensed prescriber. While this error did not result in the patient receiving the medication, the potential for harm was significant.

Access to the system by IS personnel was reassessed and the security file was modified to put a flag on all IS staff sign-on. Any user with this flag is unable to sign orders on real patients. The system will now allow the IS personnel to go through the order process up to the point of signing the orders, which is the step that is necessary to make the order active and communicate it to others. Also, patient names in the test database were changed to include only descriptors and numbers. For example, names like “Mary Poppins” which could be mistaken for a real name were replaced with “Test015.”

Each ICIS ordering screen contains a free-text box for adding additional patient clinical information and a free-text box for additional instructions to be communicated to the department receiving the order (nursing, pharmacy, radiology, etc.). The clinical information box was intended for information which will not necessarily change over the course of admission (e.g. “2 year old boy with relapsed leukemia”). Once this information was entered for any single order, by design it appeared with all subsequent orders. The additional instructions box, on the other hand, was intended for instructions or information relevant to a specific order. This additional information is seminal to providing the right timing of tests or medication administration or getting the best information from consulting services; as such, this box is renewed for each new order. A challenge arose when it was discovered that comments in the clinical information box may not be helpful if the comment stayed with the patient the entire visit. This occurred frequently when orders for radiology included indications for tests in the clinical information section, rather than in the additional instructions section. The most common example of this was the “Please check endotracheal tube placement” comment entered in the clinical information box in an order for a chest radiograph (Figure 6). This same request appeared in the patient history even after the endotracheal tube was removed. A similar challenge arose when a comment in the additional information box was helpful when the order was first written but did not remain relevant. While this information was not attached to other orders, it remained on the order to which it was initially attached for the duration of the order. For example, the additional information of “Dose given at 0700 this AM” was added to a BID antiepileptic medication order. The first day of the order, this was helpful to nursing staff so that an additional dose of the medicine was not given at 0900 (standard BID time). However, the instruction remained with the order the following mornings, and the patient's morning dose of antiepileptic medication was repeatedly omitted.

For the first challenge, we changed ICIS to automatically erase the clinical information box contents once every 24 hours, so the patient's history was updated by the care team once each day. For the second challenge we educated clinicians to generate orders to make these kinds of instructions unnecessary. In this instance, even though the clinician is entering the order at 0830, he/she can time the order to start some time after 0900. Clinicians are now taught to time orders appropriately so that they start when the next dose of a medication is due. To facilitate this change in practice, ICIS was modified to display the appropriate scheduled times beneath the start time field. Clinicians were also educated that the additional information they provide on an order would stay with that order.

Unsigned verbal orders or verbal orders not signed in a timely fashion were a significant problem at CCHMC prior to ICIS implementation. A benefit of ICIS is that clinicians can sign orders anywhere in the hospital without physically touching the chart. A challenge arose when we discovered that nursing staff had inadvertently selected the incorrect prescribing clinician (from the drop-down list containing thousands of names) in 3.5% of all verbal orders, making it difficult to track which clinicians are negligent in signing their verbal orders. Additionally, clinicians found verbal orders in their names on patients for whom they never provided care.

Nurses were educated on the importance of selecting the correct clinician. However, the education effort had limited effect. Subsequently, a new electronic design was implemented. Now, after the nurse selects the clinician who made the verbal order, a screen pops up indicating the selected clinician and asking the nurse if this is the correct clinician (Figure 7). Another educational campaign has been instituted to remind nurses to ask clinicians who are giving verbal orders to spell their full name and to refrain from accepting verbal orders unless it is an emergent situation. Today, incorrect clinicians are selected in only 0.8% of verbal orders and the incidence of verbal orders has decreased from 22% of total orders prior to ICIS, to 10% of total orders.

The purpose of this review was to illustrate some of the unanticipated challenges associated with the successful implementation of an inpatient electronic health record in a major hospital. The lessons learned from this experience should assist other institutions in anticipating challenges and taking appropriate preventive action when implementing their own electronic health record system. The introduction of any major system change, such as an electronic health record, will inevitably lead to workflow adjustments until the culture has adapted to these changes. During such a significant undertaking, unanticipated challenges in the system are likely to be unavoidable. Considerable efforts should be placed in preventing problems prior to system implementation; however, the goal should not be preventing all possible problems. Requiring an electronic platform to be problem-free prior to implementation is an unreasonable goal. More appropriately, new systems should be implemented with the expectation that unanticipated challenges will occur. The implementation plan should include methods in place to identify, evaluate and repair unanticipated challenges as quickly as possible after identification. One of the key reasons for the successful implementation of ICIS at CCHMC was the existence of active support by IS staff and invested system users after the system was put in place. While implementation was initially successful, it is recognized that ICIS is not a static system. The ICIS multidisciplinary design teams still meet on a weekly basis to address new concerns.

It is important to note that few of the unanticipated challenges were deficiencies in the computer software or hardware and none required significant changes to the system. The majority of the challenges related to changes in the culture of the organization and underlying process deficiencies. For example, the challenge of unrecognized discontinuation of medication was not an ICIS issue, but one of educating the clinicians on how policy was represented in ICIS. Previously implied instructions and “It's always been done this way” practices must be explicitly communicated as the computer cannot transmit implications. By educating users on how the ICIS acts on information entered by users, the majority of the unanticipated challenges of implementation were resolved.

The medical system's failings have been brought into the open in response to the study on adverse drug events conducted by the Institute of Medicine.9 The added complexity of ordering care for the pediatric population potentially makes the risk of adverse events in children's hospitals even greater than at adult facilities.10 The benefits of CPOE have been enumerated in a number of studies, and similar benefits have been recognized at CCHMC.11,12 The medication error rate has significantly decreased, clinical care efficiency has improved dramatically, and the rate of verbal orders and unsigned verbal orders is lower. While continued challenges are expected with this complex system, we believe that the ICIS will continue to facilitate improved patient care and safety. Unanticipated challenges are to be expected with the implementation of an electronic health record. Hopefully, the lessons learned at CCHMC will allow other institutions to avoid some of these challenges and to put in place a robust system to detect and respond to these issues in a timely fashion to ensure implementation success.

The Clinical Informatics Outcomes Research Group includes the following contributing members: Patricia Abboud, Andrew Bazemore, Mary Bellman, Angela Booth-Jones, Richard Brilli, Edward Donovan, Floyd Eisenberg, Marty Goodfriend, Craig Hallstrom, Kim Ward Hart, John Hingl, Brian Jacobs, Neil Johnson, Angela Jones, Jennifer Kaplan, Gayle Lykowski, Daniela Mahoney, Michael McKibben, Jennifer Moore, Nora Paulford-Lecher, Pat Pilcher, Terri Price, Jenny Raake, Don Rucker, Pat Schaffer, Dan Wells, Mark Wess, and Debra Wolf.

CCHMC

Cincinnati Children's Hospital Medical Center

CD

Clinical Documentation

CPOE

Computerized provider order entry

ICIS

Integrating Clinical Information System

LCR

Lifetime Clinical Record

MAR

Medication Administration Record