My clinical and leadership career from 1982 to 2014 as a diagnostic radiologist in general and pediatric radiology and executive medical director of an acute National Health Services (NHS) hospital coincided with huge changes in the design and delivery of clinical care for patients with cancer. However, involvement from 2009 to 2014 in a regional program to significantly improve patient safety in general medical care showed that using quality improvement methodologies and organizational leadership approaches had a swifter and more sustained impact than policy and top-down directives in any aspect of clinical practice. If we wish research to impact positively on every patient, then we need to think about how we measure and embed changes and deliver better care more reliably.
Our region was challenged with hospital-acquired infections, poor outcomes in intensive care and general ward, and prescribing errors. However, by using systems thinking and frontline change approaches, such as the Model for Improvement, Human Factors Analysis, and Safe System Design, we made a significant impact over 16 large hospitals and reduced standardized hospital mortality by 14% in 4 years. Sustaining high-quality reliable services in the arena of advancing medicine with new treatments, new capacity, and increasing demand is best achieved by using such approaches. Just promoting best practices can fall on challenged systems and resistant cultures and thus fail to deliver best practice in spite of best intentions.
Complex systems such as health care may struggle to embrace and deliver at pace new knowledge and science. They often overlook the social and professional interrelations that influence the rate of change. The context of demographics, long-term survivorship expectations, and political and societal pressures each adds a powerful opportunity but also many barriers to effective and efficient adoption of best practices.
We increasingly understand through experience and observation that improvement needs to be a continuous mind-set, not a time-limited action. In cancer care, the triple dynamics of increasing access to earliest diagnosis, enabling the best evidence-based health care, and enabling health and ongoing well-being and survivorship must drive our system to evolve and adapt to the country we work in and the demographics and clinical challenges we face most. The first step is understanding the current work, the second step is inspiring and enabling staff to innovate and change their system of work, so it is safer and improves outcomes for all patients.
We are reminded by the observation of Aristotle that “excellence is a habit.” It is supported and enabled by a system that is designed to deliver excellence. It is not a single person's attribute, it is the result of a complex system working to achieve improvement and thus excellence at all times. We are also wise to remember that “every system is perfectly designed to get the results it does.” This observation is attributed to W. Edwards Deming, a founder of Improvement Science, and discussed in detail by Professor Don Berwick in 1996. If we want different results, we need to make changes.
We can learn from academic work across many different types of organizations seeking high levels of reliability and safety. The works undertaken by Vincent et al. and Vincent and Amalberti[5,6] over the last decade have evolved to enable us to understand how quality improvement approaches can address the variation we see in health care. When care envisaged by the standards set as best practice is in place but unreliably delivered, we can deliver excellent care in some cases but fall short with minimal or no harm in other cases. In such systems, there is the ever-present risk that significant harm will result because the system itself contains the risk. When the team performs a routine check for renal function and ensures hydration as part of renal protection in a chemotherapy regimen, usually minor or no renal deterioration is observed. When a system is delivering 70%–90% reliability at each step, it rapidly becomes unreliable with significant impacts for patients. In such a system, where each step is happening just four of five times or less, there is an ongoing 60% chance something could go wrong, with potential for major permanent impact on the patient. Understanding the reliability of our system and working on the processes and structures to make it easy and efficient to do exactly what is needed at each step can swiftly move a system from 40% overall reliability to 70% or more [Figure 1]. When each part of a process determines the chances of success of the next step (First Pass Yield), then every step needs attention.
Reflecting on these ideas enables teams to challenge themselves: Are we reliable enough? What approaches can we use, or is our system so out of line with what is needed that serious and fatal harm is occurring? The answer tells the team if major changes are needed. This also helps clinicians learn where to focus and where they can make early gains but not lose sight of the significance of the whole system for delivering care.
Measurement is something we apply to sick people by repeatedly, at short intervals, checking the measures to see the impact of our intervention. Unlike research, where data are collected and analyzed for a set period, quality improvement teams think of sick systems as we all think of sick patients. Our starting point in any methodology is to be clear about what we are trying to achieve, to be able to articulate our purpose, and to always remember the patient is at the center. Our second action is to consider how we would know we had made an improvement by observing a variety of measures we have agreed on, which show evidence that the sick system is changing in the way we intend and in line with our purpose.
Research is crucial in cancer care. However, one cannot map over to quality improvement environments, the controlled environment of the randomized control trial, the blinded period of knowledge, and the large volumes of aggregated data to gain sensitivity and specificity through enumerative statistics. In quality improvement projects, we aim to understand what is happening as it happens, to learn and adapt our approach, and to innovate within the real world that is full of variation in presentation, environment, and patients' needs. Our basis for a measurement strategy is also grounded in what was first described 50 or more years ago by Avedis Donabedian, a US physician who clearly articulated that quality required processes to change but also structures to support and embed those processes and their outputs in the team/organization. Without the structural support, enablement and leadership progress is hard to sustain and outcomes shift slowly if at all.
Thus, measurement to develop and sustain excellent quality of care requires frontline measurement that rapidly gives feedback to staff that they are doing their work more effectively, reliably, and efficiently, as well as measures and observed structural changes that embed learning, behaviors, and accountability, to sustain excellence over time. This combination drives the cultural change toward highest levels of continuous improvement because everyone, every day is doing their work and improving their work.
What to Measure and How
How do we measure across the various processes and structures that we want to change? How do we ensure people learn as we go and how do we avoid adding to harm in the process?
Define a clear aim with a complete picture of the health-care situation and a range of measures that can give feedback and show improvement (outcome measures).
Use those measures to adapt and refine better processes, as without these, as reliability math tells us, outcomes will not shift sustainably toward the stated aim (process measures).
Measure processes, outcomes, and most importantly, the unintended consequences of focusing our effort where we have chosen (balancing measures).
Balancing measures are key to spotting unintended harm before or as it happens for the first time. This is crucial in quality improvement projects for patients but also to sustain the positive psychology that people need as they change the way they work.
The process measures are key to learning and empowering staff and patients. Measurement of progress and change at the front line will ensure best efforts are made to collect data as it happens and easily within the work flow. This ensures fast feedback loops and positive learning experiences.
Focusing on process improvement is key to initiating system improvement and sustaining it over the long term. To this end, having a clear operational definition of what is being measured and exactly how that measurement is to be taken or calculated is vital. It is too easy in the busyness of work to guess, estimate, do it differently, and thus muddle ourselves. Our approach is to test the act of measurement. When we are clear our new way of delivering care meets the new and higher standards we have set ourselves, then we embed the approach in our work and do more of it. In addition, sampling is clearly of value in our work. The actual clinical world is like a river, an ever-changing flow of events, people with unique features to their illness, and environments that differ within a service and between services. By testing our changes in different situations and environments, we gain clear understanding of what works and what does not work. This also provides the opportunity to amend a plan and for the team to gain confidence in its impact. To support this learning, we do not aggregate data, we want to see it as it happens in time-ordered displays with which we can then see the variation, narrow the variation, and see if we are close to an upper or lower control limit that we have set. In this way, we seek to shift performance to a new and improved norm.
In such circumstances, it is useful to consider people and their response to data as well as their reaction to change. We use the Kubler–Ross change curve to understand grief, and such grief often comes early on in improvement work. We cannot accept that we are that unreliable or bad. We need to use these data, and how we display these data to lead people along this journey of initial shock then denial, frustration, depression, experimentation, and eventually the decision to act. For innovators and early adopters, change, uncertainty, and the excitement of exploring a new approach are core to our personality. For those who are more cautious about change, the narrative of a change that impacts patients and staff as well as meaningful real-time data showing how the change is making care safer, more effective, more efficient, equitable, and more person-centered maintains the momentum. Different people require different levels of confidence to change their own work and support a new approach.
Sustained improvement in a system comes when the whole team gets the message, does the work, and works to improve that work. Excellence becomes the habit of the system and all who work in it, not just the habit of one or two individuals. Key messages for those seeking to build measurement systems that will drive continuous improvement are:
Measurement for quality improvement is continuous and focused on processes, outcomes, and the unintended consequences.
It is easily understood and can be replicated by everyone involved.
It should ideally be undertaken in the flow of work, not an “add-on,” wherever possible.
The measurements made and their impact on output and outcomes of the work are regularly fed back to the team so they see the immediate impact of the changes they are making rather than data being aggregated and reported back weeks or months later.
Financial support and sponsorship
The author disclosed no funding related to this article.
Conflicts of interest
The author disclosed no conflicts of interest related to this article.
For reprints contact:firstname.lastname@example.org