Quality Indicators in Adult Critical Care Medicine

Attaining the desired outcomes of patients in any healthcare system is the direct result of the quality of provided care.^[1] The Institute of Medicine describes quality care in healthcare as a care that is safe, timely, effective, efficient, equitable, and patient-centered.^[1] Intensive care medicine is a core component of modern healthcare systems. Because of the complexity of patients, invasiveness of interventions, the interdependence of several providers, and reliance on teamwork, medical errors and adverse outcomes are common in intensive care units (ICUs).^[2] In addition, variations in available resources and training backgrounds lead to variations in the provided health services and practice patterns among different healthcare providers, institutions, and countries.^[3,4] As the demand for intensive care services and the associated costs have increased in the past few decades, several healthcare authorities and institutions have focused on improving the quality of care in the ICU and mandated the monitoring of quality indicators to compare their performance with national standards or international benchmarks.

Quality improvement in healthcare depends on the measurement of relevant quality measures and indicators. Quality indicators are tools that are used to identify potential areas for improvement in the quality of care. In the ICU as well as in other settings, a quality measure can be a measure of structure, process, or outcome and can be selected based on the best available evidence or expert consensus if the evidence is lacking. A good quality measure should meet the following criteria.

1. Important, thus addressing an essential structure (such as ICU staff qualifications and staffing patterns), a common process with a large effect on outcome (such as the care processes related to mechanical ventilation) or a clinically meaningful outcome (such as mortality or development of a complication that is related to a common ICU process, i.e., ventilator-associated pneumonia [VAP]).^[5]

2. Relevant to the patients and/or their families, thus being a patient-centered measure (such as pain control).^[5]

3. Feasible, such that data collection is not too burdensome.^[5]

4. Valid, thus the measure is supported by robust evidence linking it to improved outcomes.^[5–7] In addition, data collection tools are standardized so that the results reflect the true problem, able to capture what providers do rather than patients' characteristics and are comparable across all ICUs.^[5–7]

5. Interpretable, such that there are estimates, expressed in common metrics with sufficient variability to differentiate poor performers from good ones, and there are care processes associated with the indicator that are measurable.^[8] This will make the quality measure more actionable.^[5]

The selection process of a quality indicator usually goes through multiple steps.^[7] It usually starts with a thorough literature review of the interventions/processes that improve one or more outcomes in the ICU.^[7] This is followed by the selection of a pilot indicator in order to investigate the feasibility of data collection and to determine the required changes that would positively influence the indicator.^[7] The data collection process should be defined (who, what, when, how) and the validity and reliability of data should be based on field studies.^[7] Finally, the pilot indictor should be tested to evaluate how it performs in the ICU.^[7] An indicator that does not provide opportunities for improvement can be dropped and other indicators that may potentially improve care can be adopted. If an indicator does not show any more variability, the indicator can be exchanged for a new one. As there are numerous quality measures, each ICU should select the indicators that are more likely to impact the outcomes of its patients.

More than 20 years ago, Berenholtz and coworkers^[9] performed a systematic review of all studies that were published between 1965 and 2000 and provided a potential measure of the quality of intensive care and identified six outcome measures (ICU mortality rate; ICU length of stay >7 days; average ICU length of stay; average days on mechanical ventilation; optimal pain management; and patient/family satisfaction). They tested 18 items that were considered to be core quality indicators for the ICU and demonstrated the feasibility of implementing them in 13 adult medical and surgical ICUs in urban community teaching and community hospitals.^[10] National institutions in several countries followed and developed quality indicators based on the best evidence and with the goal of improving the outcome of ICU patients.^[11–14] In 2005, the Joint Commission on Accreditation of Healthcare Organizations instituted a reporting requirement for a set of four ICU core measures (patient positioning, stress ulcer prophylaxis, thromboprophylaxis, and central line–associated bloodstream infection [CLABSI]) that were based on the National Quality Forum report, with an additional two test measures (risk-adjusted ICU length of stay and hospital mortality). A task force of the European Society of Intensive Care Medicine published a list of quality indicators in 2012.^[15] The structure indicators included compliance with national standards and an “adverse event” reporting system.^[15] The process indicators included routine multidisciplinary ICU visits and a standardized transfer protocol.^[15] The outcome indicators included standardized mortality rate, 48-hour readmission rate, rate of CLABSI, and rate of unplanned extubation.^[15] Quality improvement programs in the ICU setting frequently use the following concepts.

A stretch goal is a goal that may be difficult to attain. It usually involves setting a target for a change/improvement in a certain period. An example is the reduction of the rate of VAP by more than 50% within a 6-month period in an ICU with a high baseline rate. This usually inspires growth, counters complacency in teams, and communicates immediately and clearly that maintaining the current status is not an option.^[16]

Targeting zero events for certain quality indicators in the ICU reflects zero tolerance policy for adverse events and may be a realistic goal.^[17,18]

In a care bundle, related care processes, which are usually performed separately, are bundled to ensure that they are given together and to reduce the chance of important aspects of care being missed. This concept has been used to reduce VAP, CLABSI, and catheter-associated urinary tract infection (CAUTI). The all-or-none approach to measuring performance, in which all elements of a bundle must be done to be considered compliant, offers several important advantages over either individual or composite item measurement.^[19]

This is a multifaceted approach to patient safety that has been proven to improve and sustain quality of care.^[20] Its framework is composed of five steps: (1) train staff in the science of safety; (2) engage staff to identify defects; (3) senior executive partnership and patient safety rounds; (4) continue to learn from defects; and (5) implement tools to improve teamwork and communication.^[20,21] The CUSP model provides a strategy for healthcare organizations to improve culture and learn from mistakes through the integration of safety practices into daily work.^[21]

There are many measures that are candidates to be quality indicators for the ICU.^[8,13] These indicators usually address the most relevant core processes of intensive care such as mechanical ventilation, analgesia, sedation and delirium treatment, anti-infective therapy, nutrition, hygiene, and communication with patients and their relatives. An important factor is the number of indicators in the ICU at any given time and 10 indicators are thought to be a manageable number. It is also believed that some ICU quality indicators in one setting might not be transferred unchanged to another setting. In addition, quality indicators need to be reviewed periodically, as new evidence may refute the implementation of certain interventions. Table 1 describes important ICU quality indicators. The core indicators are the ones that should be implemented in all or most ICUs. We also present additional details on pertinent indicators and selected quality improvement projects that were related to them.

Occupancy is a measure of capacity strain such that ICUs with high occupancy rates may be less capable of providing the same high-quality care on a given day as an ICU with more available beds.^[22] There is no commonly accepted method for calculating ICU occupancy, but dividing the number of patient bed hours by the total number of available bed hours is probably the most accurate method.^[23] This will allow the calculation of daily occupancy, which is a better benchmark of care quality than mean annual occupancy.^[24] There is a clear association between higher daily ICU occupancy and early discharge/nonclinical transfer,^[24] but whether this may lead to worse outcomes is unknown. It should be noted that some ICUs may be able to function as high-reliability organizations, where outcomes are maintained during low and higher occupancy rates.^[25] In general, the optimal ICU occupancy rate is approximately 70–75%.^[23]

Mortality is an outcome measure that is important to healthcare providers as well as patients. The actual mortality rate can be misleading if it does not consider changes in patient mix (demographics, comorbidities, and diagnoses), and severity of illness. The standardized mortality ratio is the observed mortality divided by the mortality rate that is predicted by a prognostic score such as the various versions of Acute Physiology and Chronic Health Evaluation, Simplified Acute Physiology Score, and the Mortality Probability Model score.^[26,27] The reliability of these scores depends on the completeness and accuracy of the abstracted data. In addition, they have been shown to overestimate the risk of death in the studied populations, as older scoring systems are used in newer data sets.^[28] Standardized mortality ratios are useful in examining overall performance among general and specific ICU populations on a retrospective basis and in benchmarking across ICUs and institutions. They, however, cannot be used to prognosticate individual cases or to determine medical futility.

Readmission rate is frequently used as a quality indicator because it is related to both patient outcome (increased mortality, cost, and length of stay) and organizational efficiency.^[29] Currently, available studies are not clear about modifiable factors as tools to reduce readmission rate. In a retrospective study of 19,750 ICU admissions, the readmission rate was 7% and the independent variables associated with readmission were age, severity of disease, type of admission, infection, immunodeficiency, and last-day noradrenaline use.^[30] In this study, the latter factor was the only one that could be modified.^[30] Another study found that ICU readmission was associated with greater severity and complexity of illness.^[29] Hence, readmission rates require case-mix adjustment before they can be a useful quality indicator.^[29,30] Another study found that post-ICU admission hospital mortality and ICU readmission were poorly correlated and that ICU readmission performed poorly as a performance metric.^[31]

ICU length of stay is highly associated with the costs of care in the ICU. It is an easily quantifiable outcome indicator for the efficiency of ICU care. ICU discharge is often based on subjective criteria and could be influenced by the availability of bed and staff resources.^[32] It is greatly influenced by the post-ICU setting and bed availability in the wards.

Hand hygiene is an important indicator of safety and quality of care in any ICU. There is substantial evidence demonstrating that effective hand hygiene reduces ICU-acquired infection. Several quality measures, related to structure, process, and outcome, may be used in relation to hand hygiene.^[33] In general, hand hygiene compliance is lower than the target for most ICUs worldwide. A systematic review that included 61 studies found a mean hand hygiene compliance of 59.6% (high-income countries 64.5%, adult ICU 58.2%, nursing staff 43.4%, physicians 32.6%).^[34] To emphasize the importance of hand hygiene in preventing hospital-acquired infections, the World Health Organization started the “Clean Care is Safer Care” campaign in 2005, and then adopted the “SAVE LIVES: Clean Your Hands” initiative in 2009. It also provided tools and protocols on how to measure and monitor hand hygiene compliance. Optimal monitoring requires manual and continuous observation of hand hygiene practices, which is time-consuming and not feasible. Periodic audits are frequently performed but might be inadequate to measure the real compliance rate. A less complicated alternative is a monthly measurement of the volume of alcohol used for handrub. Published studies reported an average use of 68–73 mL of alcohol per patient per day^[35]; however, this measurement does not provide information about the quality of hand hygiene or the compliance rates by specific healthcare providers.^[35]

Strategies to improve hand hygiene compliance consist of the following: increasing the availability of alcohol‐based hand hygiene products, staff education, reminders (written, electronic, and verbal), performance feedback, administrative support, and staff involvement.^[36] Studies have reported variable improvement results with the different strategies.^[36] Strict compliance with hand hygiene is time-consuming (a nurse may spend 58–70 minutes on hand hygiene per ICU patient per 12-hour shift),^[37] and should be considered in staff planning.

Historically, the outcome measure that is related to ventilator care is VAP (Table 1). The Centers for Disease Control and Prevention in the United States replaced their longstanding VAP definitions with ventilator-associated event definitions in 2013 to capture harm not only from pneumonia but also from pulmonary edema, atelectasis, and acute respiratory distress syndrome.^[38] Whether ventilator-associated event definitions are suitable to serve as quality indicators for the ICU is a controversial issue.^[39] Interventional data demonstrating the preventability of these events are limited and conflicting.^[40–42]

Process measures related to ventilator care are also used as quality indicators in the ICU. In 2004, the Institute for Healthcare Improvement recommended a ventilator bundle as part of the 100,000 Lives Campaign.^[43] This bundle had four components: (1) elevation of the head of bed to 30–45°, (2) daily “sedation vacation” and assessment of readiness to extubate, (3) stress ulcer prophylaxis, and (4) thromboprophylaxis.^[43,44] In 2010, the Institute for Healthcare Improvement added a fifth intervention: (5) daily oral care with chlorhexidine. Other elements were added with time as the related evidence evolved and included the use of subglottic secretion drainage, avoidance of scheduled ventilator circuit changes, and oral hygiene without chlorhexidine as chlorhexidine oral care was associated with higher risk of ventilator-associated events.^[45,46] Improving ventilator care was also part of the ICU Liberation Collaborative, which focused on implementing pain, agitation, and delirium guidelines in ventilated patients.^[47] This collaborative adopted the ABCDEF (A, assess, prevent, and manage pain; B, both spontaneous awakening and spontaneous breathing trials; C, choice of analgesic and sedation; D, delirium: assess, prevent, and manage; E, early mobility and exercise; and F, family engagement and empowerment) bundle.

In general, the evidence supports the use of the ventilator care bundle in the ICU. In a systematic review of studies that evaluated the implementation of the Institute for Healthcare Improvement ventilator bundle, 22 of 38 studies showed more than 36% decrease in VAP and 10 studies showed more than 65% decrease.^[48] Another systematic review of 45 randomized controlled trials (5493 patients) showed that daily sedation interruption significantly reduced mechanical ventilation duration, ICU stay length, sedation duration, and tracheostomy and VAP.^[49] In another improvement collaborative, the ABCDEF bundle was associated with a lower risk of hospital death within 7 days.^[50] The multicenter prospective quasi-experimental National Approach to Standardize and Improve Mechanical Ventilation collaborative assessed the impact of evidence-based practices (subglottic suctioning, daily assessment for spontaneous awakening trial, spontaneous breathing trial, head-of-bed elevation, and avoidance of neuromuscular blockers) in 42 ICUs from 26 hospitals in Saudi Arabia, using the CUSP model and interventions that included online educational activities and real-time benchmarking of daily care process measures to drive improvement.^[51] The collaborative was associated with improvements in daily care processes and with a reduction in mortality.^[51]

CLABSI represents an important quality indicator as it is associated with significant morbidity, mortality, and cost and there is a good understanding of the related evidence-based prevention measures. Rates of CLABSI are tracked, reported, and tied to reimbursement by the Centers for Medicare and Medicaid Services. Studies show that the mean rate is 7.5 per 1000 catheter days, but vary widely between centers and countries (1.1 to 12.1 per 1000 catheter days in studies from the United States and 1.4 to 45.9 per 1000 catheter days in studies from other countries).^[52] The CUSP model has been used for CLABSI prevention in the ICU and its effectiveness has been demonstrated in the implementation of the CLABSI bundle (hand hygiene, use of full barrier precautions, avoidance of femoral lines, skin antisepsis, and removal of unnecessary lines) in several institutions, which led to a sustained reduction in CLABSI rates by > 60%.^[17,18]

CAUTI is probably the most common infection in the ICU. The mean rate is 12.5 per 1000 catheter days (1.4 to 15.8 per 1000 catheter days in studies from the United States; 0.8 to 90.1 CAUTIs per 1000 catheter days in studies from other countries).^[52] The principles of CLABSI prevention (aseptic insertion, maintenance care, and prompting removal) and the CUSP model have been applied to CAUTI prevention. There has been a slower adoption of CAUTI bundle than CLABSI likely due to less awareness about the clinical significance of CAUTI as well as lower success in reducing CAUTI rates.^[53]

Pressure injury is a recognized metric of quality of care by the Centers for Medicare and Medicaid Services, which restricts reimbursement for hospital-acquired pressure injuries. Unavoidable pressure injuries do occur and may represent a form of acute organ failure.^[54] Silicone foam dressing has been shown to reduce sacrum and heel pressure injuries in the ICU.^[55] However, the evidence on other interventions, such as nutrition, skin-care regimen, positioning and repositioning schedule, support surfaces, and the role of education, is limited.^[55] Pressure injury prevention and treatment in the ICU should be a multidisciplinary quality improvement initiative.

The perception of patients and their relatives about the care received is important. Their feedback on care provided in the ICU may provide information about a hospital's ability to provide good service and ultimately will support healthcare professionals in their continuing efforts to improve care.^[56] Patient satisfaction is a complex measure that may not be consistent with patient outcomes,^[57] nor with other markers of quality of care.^[58] However, studies suggested that patient or family ratings correlated with other quality domains (organization and safety) in some settings.^[59] In general, high ratings may reflect low expectations, and rising expectations might lower satisfaction. As the public increasingly adopts patient- and family-centered care, satisfaction ratings may drop further.

Tools to measure satisfaction include Critical Care Family Needs Inventory, the Society of Critical Care Medicine Family Needs Assessment, the Critical Care Family Satisfaction Survey, and the Family Satisfaction in the Intensive Care Unit.^[60]

It is intuitive that the ICU staffing level impacts the quality of care, patient outcomes, and staff well-being. A British study found no influence of registered nurse staffing on mortality rates, but a positive impact for doctor staffing and mortality.^[61] By contrast, a Finnish study demonstrated that high nursing workload is associated with increased hospital mortality.^[62] The optimal nurse-to-patient ratio in the ICU depends on the complexity of provided care but is usually 1:2 or fewer in high-level ICUs.^[63] In academic medical ICUs, an intensivist-to-patient ratio less than 1:14 may negatively impact education, staff well-being, and patient care.^[64] A taskforce of the Society of Critical Care Medicine suggested that high staff turnover or decreases in quality indicators in an ICU may be markers of overload.^[64]

The Infection Prevention and Control Department performed an audit in a busy 15-bed medical ICU of a tertiary-care hospital and reported to the medical director that 10 patients developed VAP in the preceding 3 months (12 cases per 1000 ventilator days). This was higher than the two cases of VAP diagnosed during a previous audit that was performed 9 months earlier. The ICU medical director embarked on a quality improvement project for VAP prevention.

In the following, we describe the quality improvement project as performed in the ICU focusing on the main steps and referring to the tools, concepts, and strategies (Six Sigma, Lean, and the Model for Improvement)^[65–67] that were used during the process.

A team was formed and consisted of involved stakeholders and had a champion (a senior respiratory therapist who was responsible for the day-to-day management of the project), a leader (an intensivist who understood the implications of changes on other parts of the system), improvement advisor (a specialist with expertise in quality improvement methods), and three other team members (one ICU fellow and two ICU nurses). This team had the right balance of leadership, management, expertise, and power so that the project could succeed. The members received training in the Science of Improvement and a project charter was completed to document and outline the main aims.

The team decided that the target was a reduction in the VAP rate by 50% in the next 6 months. The team reviewed evidence-based practices to reduce VAP and decided that ventilator care bundle was the best process measure that would lead to VAP prevention.

The team performed audits daily at unannounced and variable times using a standardized and tested data collection form (a paper checklist). The average compliance rates for the elements of the ventilator care bundle were 91% for head-of-bed elevation, 95% for stress ulcer prophylaxis, 91% for thromboprophylaxis, 65% for sedation vacation, 70% for oral care, and 50% for subglottic suctioning. The total bundle compliance was 70%.

The team performed root cause analysis^[68] to evaluate the causes of the increasing rate of VAP taking into consideration the measured data. A fishbone diagram was used to identify causes categorized into physical (equipment/environment), human (patient/provider), and system factors. The team decided that the low ventilator care bundle compliance was the root cause with a need to focus on elements with compliance less than 90%.

The team produced an action plan that consisted of staff education on ventilator care bundle through posters and PowerPoint presentations, revision of the oral care protocol, and having only endotracheal tubes with subglottic suction lines in the ICU. The team used the PDSA (Plan, Do, Study, and Act) framework to test changes. One of the PDSA cycles was testing the validity of an electronic ventilator bundle checklist that was built in the electronic health record. Another PDSA cycle was related to improving daily sedation vacation. The team tested implementing a protocol to lighten sedation daily at 8 AM for eligible patients. The team also evaluated the precautions to prevent self-extubation (a balancing measure), such as increased monitoring and vigilance during the trial. Multiple cycles were performed to achieve the desired goals. Common tools used during the PDSA cycles were brainstorming, cause-and-effect diagram, prioritization of “vital few” causes (Pareto principle), corrective action, and monitoring of impact.

This involved standardizing the changes so that they became part of daily routine work to facilitate sustainability.

Compliance with the ventilator care bundle improved to 85%. The rate of VAP went down to 4.5 per 1000 ventilator days in 6 months. A control chart that depicted VAP rates at 3-month intervals was periodically reviewed by the team.

Sustaining quality of care requires continuous or periodic auditing of quality indicators, and so significant resources, which may not be available for many ICUs. Electronic health records may have the feature of automated extraction of data including quality indicators,^[51,69] as well as the real-/near-real-time display and share of data in different formats including control charts and dashboards.^[12,51] This may also allow for detecting variation in process or outcome measures and identifying data points (i.e., a significant decline in a certain measure) that should be investigated,^[12] thus possibly contributing to the performance and sustainability of quality projects. In practice, automated data extraction is currently limited by the variable capabilities of the different electronic health records and by the validity of entered data. These and other limitations should be addressed in the design and implementation of electronic health records in the future.

Quality indicators are crucial for the measurement of the quality of care in the ICU and are essential to inform quality improvement efforts. Each ICU should determine which indicators to measure and monitor taking into consideration national regulations and their specific needs. ICU staff knowledge of the essentials of quality improvement methods will help quality improvement efforts. Such efforts depend on the monitoring of quality indicators, which requires significant resources to perform. Automated data extraction from electronic health records may contribute to the sustainability of quality improvement efforts but is currently limited by multiple factors.