ANALYSIS: Protocol for a New Method to Measure Physiologic Monitor Alarm Responsiveness

Evaluating the sociotechnical impact and limitations of newly adopted biomedical technology is challenging and complex. The Joint Commission National Patient Safety Goal1 and a growing body of literature recognize alarm fatigue as a healthcare hazard,2,,3 thereby drawing attention to the need to evaluate the effectiveness of available physiologic monitoring technology.4,,5 

Alarm fatigue is defined broadly as the lack of or delay in response to alarms due to excessive numbers of alarms, resulting in sensory overload and desensitization of hospital workers.1,6,7 In children's hospitals, patients generate between 42 and 155 alarms per monitored day8 and only about 0.5% of alarms on pediatric wards are actionable.3 Up to 67% of alarms heard outside a patient room are not investigated,9 and response time varies widely.3 Although many commercially available alarm management products promise improved alarm communication and patient safety, hospitals often lack the resources to measure the benefits and unintended adverse consequences of implementing these systems.

Analysis of the benefits (and hazards) of an alarm management system is challenging because alarm response is the result of complex interactions among physiologic monitors, the patient, the acuity of illness, the clinician's competing tasks, and the dynamic hospital environment. Despite advances in characterizing alarm frequency and decision making surrounding alarm response, evaluation of clinical response to truly actionable and life-threatening alarms has remained elusive.

One method for assessing clinician response to alarms is through the use of video observation.10 Video observation of clinician alarm response is highly accurate but is complicated and expensive to implement.11 In addition, given the rarity of life-threatening alarms, video recording thousands of alarms is likely to reveal only a handful of truly critical actionable alarms.10 Novel approaches are needed to measure responsiveness to rare, life-threatening alarms and identify potential latent hazards in our current alarm system.

Simulation has been used to evaluate clinician and system performance, particularly in emergent or infrequent clinical scenarios.12–14 Kobayashi et al.15 evaluated alarm responsiveness in the emergency department (ED) through simulation of a life-threatening arrhythmia. A simulator was connected to the in-room bedside monitor in an empty ED room, a life-threatening electrocardiogram tracing was generated, and clinical response time was measured by an observer. The simulation ended when any clinical provider responded to the life-threatening event or after three minutes. In the researchers' preintervention arm, only one of 20 simulation alarms (5%) garnered clinical response.

In the current work, an in situ simulation approach was developed to evaluate critical alarm response in a pediatric, non–intensive care unit (ICU) setting, adapting the simulation methodology by Kobayashi et al.15 To our knowledge, this form of in situ simulation, in which simulated alarms appearing to come from an actual admitted patient are injected into a clinical environment to measure alarm response, has not been previously described. Our aim was to develop a method for efficient, rapid, repeated measurement of unit-level response to life-threatening events that trigger physiologic alarms and require an immediate response. Using response time and response rates to quantify alarm fatigue/responsiveness on a unit level, we sought to evaluate our current alarm management system and establish a baseline from which to measure the impact of future changes to the alarm system. The intent of the intervention was not to evaluate individual nurse performance, and individual-level response time data were not available to or shared with unit leadership, in order to avoid the possibility of punitive actions for slow responses.

This article describes reproducible methodology that may be useful to other institutions seeking to evaluate hospital alarm systems in non-ICU, nontelemetry inpatient units.

This work was part of a portfolio of quality improvement work undertaken by the Patient Safety Learning Laboratory (PSLL) at Children's Hospital of Philadelphia. The PSLL is funded by the Agency for Healthcare Research and Quality, and one of its primary aims is to reengineer the system of monitoring hospitalized children on acute care units, with a focus on reducing noninformative alarms and accelerating nurse responses to critical events. In the PSLL, our transdisciplinary team uses systems engineering approaches to proceed through a five-step process for each aim: problem analysis, design, development, implementation, and evaluation. Evaluation of the relationship between alarm burden and nurse workload was part of the problem analysis phase of this improvement initiative.

The Committees for the Protection of Human Subjects (i.e., institutional review board) at Children's Hospital of Philadelphia determined that the problem analysis, design, and development phases of this project were consistent with quality improvement activities and did not meet criteria for human subjects' research.

Children's Hospital of Philadelphia is an urban, tertiary care children's hospital with a combination of private and semiprivate rooms. Each inpatient bed is connected to a physiologic monitor (Dash 3000, 4000, or 5000 Patient Monitors; GE, Boston, MA) and are used for patients on general non-ICU floors who require continuous physiologic monitoring, which requires a physician order. The monitors are capable of simultaneously displaying data from the following channels: electrocardiogram, respiratory rate, noninvasive blood pressure, pulse oximetry, and carbon dioxide. These monitors can physically be connected to a private hospital network (Carescape Network; GE) through a standard ethernet connection (RJ45 connection) for every patient bed. The network is monitored via a network gateway (Mobile Monitoring Gateway; Ascom, Research Triangle Park, NC) that has the ability to route alarm notifications to hand-held mobile devices (d62 phone; Ascom). The network gateway has a user interface (Unite Assign; Ascom) that allows users to interact with the server and assign specific patient physiologic monitors detected on the hospital network and route alarm messages to hand-held mobile devices carried by clinical staff.

Nurses are primarily responsible for responding to physiologic monitor alarms. Nurses on a general care unit typically care for three to five patients. Nurse/patient assignments are completed by unit charge nurses, who attempt to balance patient acuity, workload, and patient flow. Currently, continuous monitoring is not considered when evaluating workload for nursing assignment purposes.

Alarm “primary notification” occurs when the bedside monitor alarm sounds. Alarm “secondary notification” occurs when a notification appears on the hand-held mobile device and on the centralized information center (CIC) monitor at the front desk. Each CIC displays a remote live view of up to 16 bedside monitors. When an alarm event occurs, the CIC also alarms and highlights which bed monitor is alarming. No individual is assigned to monitor the CIC. Each unit also has one to two remote view monitors of the main CIC screen in other locations. The remote monitors do not audibly alarm, but they do provide a live feed of the patient monitor visual display.

Currently, when a patient is connected to a physiologic monitor and a physiologic alarm is triggered, those data are transmitted to the hospital network, which is monitored at all times by the network gateway. Based on the filters set in place in the network gateway, the gateway routes alarm messages to the mobile device of the nurse who is assigned to that particular hospital bed during his/her shift (Figure 1). The nurse's phone alarms with a ring tone accompanied by a text message that includes the room location and alarm details (e.g., “Bed 5S061 Warning SpO₂ LO 76”; Figure 2). The nurse can “accept” the message and respond accordingly. When the alarm condition resolves, the message is automatically removed from the phone.

We sought to transmit a high-fidelity, simulated critical alarm that appeared to come from an existing, admitted, and actively monitored patient in order to observe and measure response times on three units. The critical alarm was sent to the patient's primary nurse's mobile phone and displayed on all unit CICs, enabling staff who heard or observed the alarm at the central nurse's station or on the other unit CIC monitors to respond. To avoid disruption to patient care or patient-clinician trust, we simulated only “secondary notification” (to mobile phone and CIC display), avoided entering patient rooms, and took steps to ensure that no delay in or interference of patient care occurred. Prior to beginning the in situ simulations, all unit nurses were informed that alarm simulations would be taking place over the coming months as part of a quality improvement effort to better understand and combat alarm fatigue.

We simulated a critical hypoxemic event with a peripheral oxygen saturation (SpO₂) level between 60% and 70%. SpO₂ alarms were chosen because they are the most frequent in pediatric hospitals. Although less than 2% of SpO₂ alarms are actionable,3 true and sustained low SpO₂ will result in cardiac arrest if unresolved.16 Simulations ended when either a clinician responded to the alarm or 10 minutes passed without a response. We selected the 10-minute maximum based on data from animal models approximating toddler cardiac arrest, which demonstrated that sustained hypoxia progresses to pulseless electrical activity and cardiac arrest in approximately half of patients in less than 10 minutes.16 

The primary outcome of each simulation was time to response by any clinical staff to the simulated critical alarm, starting from receipt of critical alarm message on mobile phone devices (t = 0 min). An observer was discreetly positioned in the hallway with a line of sight to the door of the patient's room to ensure no bedside alarms were silenced, confirm that alarm systems were functioning, and verify whether a clinician entered the room during the simulation. The simulation was discontinued with any clinician entry into the patient room, even if the clinician entered to perform routine care (e.g., administer medication, check an intravenous line) and was not responding to the simulated critical alarm. This accounts for the fact that in true patient decompensation, the bedside monitor alarm (“primary notification”) would alarm and staff inside the room would presumably respond accordingly. Observers recorded simulation end time and qualitative field notes, including whether anyone interacted with the CIC, whether the CIC was making alarm noises, or whether they observed anyone silencing a CIC or phone alarm (Table 1).

In collaboration with hospital leadership, inclusion criteria were developed to establish an accurate and realistic representation of threats within the current alarm system. We sought to simulate alarms among patients who are particularly vulnerable in our current alarm system (rather than adopting a random sampling approach). Therefore, we defined inclusion criteria for patient room selection, including patient in the room must be on a continuous monitor at the time of the simulation, should be cared for by a nurse who works regularly on the unit, must not have experienced a code event or rapid response in the previous 24 hours, and was not designated by the unit as a “watcher” (i.e., receiving additional surveillance due to heightened risk for clinical deterioration). These criteria were selected to evaluate response among patients at risk of harm from alarm fatigue in our current alarm system (as opposed to patients who already receive heightened surveillance within our unit). Although such criteria by themselves may be subject to nurse response bias, the level of desaturation (SpO₂ <70%) simulated was intentionally set at a signal threshold that should have superseded existing bias. To avoid priming responses, we did not repeat simulations on the same day within the same unit.

We sought to realistically simulate what should be “never events” within a high-reliability organization, in order to evaluate potential vulnerability within our alarm system. To determine the optimal number of simulations to conduct, we discussed with hospital and unit leadership the number of simulations required to demonstrate, with sufficient statistical confidence, the safety of our alarm system. We elected to conduct 20 simulations because it provided an appropriate level of precision in reporting response rates with a feasible completion time frame of three months. For instance, in the case of perfect response, For instance, in a perfect response, nurses would respond to 20 of 20 of simulations (100%; 95% CI 83–100%). If response was observed in 10 instances (50%), the 95% CI of response would be 27% to 73%.

We formed an interprofessional team consisting of biomedical engineers, human factors engineers, information technology (IT) specialists, clinical nurses, physicians, facilitators from the hospital simulation center, clinical informaticians, and hospital administrative leadership to create and implement the simulation. The clinical engineering and IT departments facilitated in (1) acquiring simulation and monitoring technology to replicate critical alarms, (2) reconfiguring the gateway server to recognize the simulation equipment, and (3) routing simulated alarm messages to the appropriate CIC and mobile devices.

The simulation team, human factors engineers, and clinical nursing leadership at the unit and hospital level sought to ensure the integrity and acceptability of in situ simulation design. Aware that this simulation could expose system vulnerabilities (e.g., delayed or absent response to life-threatening alarms), we sought broad-based input on how to maintain psychological safety (defined as “the degree to which team members feel that their environment is supportive of asking for help, trying new ways of doing things, and learning from mistakes”17). To this end, nurses and simulation facilitators developed presimulation language and postsimulation debriefing scripts to explicitly frame this simulation as systems evaluation (rather than comparative assessment of individual- or unit-level performance).

Our engineering and IT departments created a virtual simulation unit within the hospital's network gateway to deliver simulated alarm notifications directly to nurses' mobile devices and CICs. The virtual hospital unit (named “simulation unit”) in the gateway was created using the gateway user interface and populated with unique hospital bed names that appeared visually similar to existing hospital beds (e.g., replacing the numeral “0” with the letter “O”). When a physiologic monitor is programmed with the bed name of one of the simulation unit beds and is connected to the private hospital network, it can be detected by the network gateway. The monitor then appears in the gateway user interface, and a user (e.g., a charge nurse) can assign the simulation monitor to an active mobile phone device so that it receives simulation monitor alarms. Because this monitor's bed name is unique and not programmed into any other downstream system, it is not recognized by systems outside of the network gateway, preventing downstream consequences (e.g., abnormal data being abstracted into the electronic health record) (Figure 2).

On the day of a simulation, unit nursing leadership identified eligible patient rooms and selected one using a random number generator. Unit leaders were notified that a simulation would take place but had no knowledge of the specific room number selected.

A team that included a nursing leader, biomedical engineer, simulation observer, and simulation center facilitator/debriefer assembled with necessary equipment just outside the unit (Table 2). The biomedical engineer and nursing leader programmed the simulator to mimic the selected patient's true vital signs and connected the simulator to the simulation bed monitor. The monitor was connected to the hospital network via the RJ45 connection in the data closet on the unit. The engineer pulled the simulation monitor into view on the central workstation display so that the critical alarm would be displayed at the central nursing station and other unit central displays. The nursing leader then logged into the network gateway to assign the simulation monitor to the nurse caring for the selected patient room and to a control phone held by a team member. To ensure patient safety during the simulation, all of the participating nurse's true patient monitor alarm messages and calls were routed to the unit charge nurse.

At the start of the simulation, the engineer programmed the simulator to a pulse oximeter value between 60% and 70%, which triggered an alarm when detected by the simulation monitor. The gateway identified the alarm coming from the simulation monitor and routed the message to the nurse's mobile device and to the team's control mobile device, signaling the start of the simulation (t = 0 min).

An observer stood within view of the selected patient room to intercept the clinician responding to an alarm. The simulation ended when a clinician arrived at the patient bed or after 10 minutes. Observations and timing were recorded by the observer into a Research Electronic Data Capture survey.18,,19 The observer recorded interactions with the CIC and monitor, how the nurse interacted with the phone while it was alarming, and how the simulation ended (Table 1). The observer also collected information about the patient and nursing assignment. Following the simulation, the debriefer reviewed goals of the simulation with the bedside nurse and solicited suggestions on improvements to the unit alarm system.

This methodology must be considered in light of its limitations. First, we did not design this simulation to shed light on the mechanisms that explain delayed responses to alarms. Rather, we sought an overall indicator of the clinical unit's performance within the sociotechnical environment as a whole in response to critical alarms. Second, we created the simulations only on general medical-surgical units, thus limiting the applicability of this method to ICU or telemetry patients and other types of units with different staffing models and alarm systems. Third, we developed and tested this method in a pediatric hospital. Although we believe that alarm response time is a relevant outcome measure across patient populations, some of the aspects of the simulation might need to be modified for use in adult settings. Last, our method gives information about specific alarm notification delivered by one mechanism; therefore, it cannot be extrapolated to different types of alarms (e.g., primary notification, arrhythmia).

As healthcare systems adapt to the rapidly changing landscape of communications technology, understanding the effects and latent threats of these technologies is important. We describe a reproducible approach using existing hospital technology to test and quantify alarm response times and rates without interfering with routine patient care. We intend to use simulation observations and outcomes to design and evaluate quality improvement efforts to address alarm fatigue. In the future, such simulations could provide a framework to benchmark performance with different communication and alarm systems.

Grace Good (Simulation Center, Children's Hospital of Philadelphia) provided simulation debriefing support. Charlie Kovacs (Information Services, Children's Hospital of Philadelphia), Larry Villecco (Biomedical Engineering, Children's Hospital of Philadelphia), and Michael Hamid (Biomedical Engineering, Children's Hospital of Philadelphia) provided technical support in simulation implementation.