The objective of this study was to assess the effectiveness of the Saudi national accreditation program on patient safety culture in a secondary-tertiary public hospital in Saudi Arabia.
Three hundred health professionals were randomly selected to participate in a survey. The survey was used in three phases: baseline, before accreditation, and after accreditation. Primary and secondary outcome measures were teamwork within hospital units, feedback and communication about errors, hospital handoffs and transitions, overall perceptions of safety, frequency of event reporting, and perception of patient safety grade.
The survey response rate was 100%. A statistically significant impact of accreditation was found for teamwork within hospital units, feedback and communication about errors, and hospital handoffs and transitions (p = 0.002, 0.009, and 0.010, respectively). Ordinal logistic regression confirmed that the accreditation program had a significant effect on overall perceptions of safety (odds ratio [OR] [1.42–13.56], p = 0.010), frequency of event reporting (OR [0.91–7.96], p = 0.073), and staff awareness of grading safety culture (OR [0.02–0.70]) and reporting behavior (OR 0.10 [0.03–0.37]).
The Saudi national accreditation program had a significant positive impact on some patient safety culture dimensions and outcomes. These findings provide local empirical evidence on the benefits of implementing national accreditation programs. Further research on a larger scale is highly recommended.
Improving quality and patient safety is gaining more national and international attention due to the undergoing worldwide transformation of health systems. The Patient Safety Network has defined patient safety as the “freedom from accidental or preventable injuries produced by medical care.” The ongoing growing complexity of healthcare systems emphasizes the critical need to protect patients from avoidable harm and build safer healthcare system for patients, providers, and society. Conversely, providing poor-quality health services and having poor patient safety increase the length of hospitalization, resulting in the loss of the patients' productive days, and can even result in patient disability and death, among other negative outcomes. The Institute of Medicine report from 2 decades ago revealed that 48,000–98,000 patients die per year due to medical error in the United States. Likewise, several studies estimated the direct medical costs due to adverse events in the United States, United Kingdom, and Australia range from $1.2 billion to $19.5 billion annually.[5–7]
Among the initiatives to enhance quality of care is promoting patient safety culture. There is growing evidence in the literature for the importance of assessing the culture of patient safety to understand the values, beliefs, attitudes, behaviors, and norms that are related to patient safety behavior among health providers.
Several international accreditation programs have been initiated lately to evaluate hospitals' compliance with healthcare quality standards and to ensure quality and patient safety. Accreditation is a self-assessment and external peer review process used by healthcare organizations to accurately assess their level of performance in relation to established standards and to implement ways to continuously improve their health care systems. The accreditation process is a comprehensive approach for continued quality improvement and learning.
As far as the relationship between accreditation and safety culture is concerned, the assessment of patient safety culture has been included in accreditation since 2007 in the patient safety goals of the Joint Commission for Accreditation of Healthcare Organizations. Different accreditation bodies have been established worldwide to accredit hospitals, such as the Joint Commission International, Canada's national accreditation program, United Kingdom Care Quality Commission, German Medical Association, and others. However, research regarding the impact of accreditation programs on quality and safety in general and patient safety culture has been noted as scarce. The systematic review of Brubakk et al reported that accreditation continues to grow internationally, but organizational outcomes related to accreditation remain vague. This current gap in the literature raises the question of whether accreditation, considering all the costs associated with it, improves the process and outcomes of care.[3–8]
Most previous literature has focused on the impact of accreditation on patient satisfaction or safety outcomes only.[10,12,13] Accreditation is most effective if it succeeds in improving the sustainability of quality of healthcare, patient safety, and patient safety culture. Influencing attitudes, increasing safety perceptions, and focusing on patient needs in any health intervention are more likely to positively alter the patient safety culture and patient safety outcomes.
In Kingdom of Saudi Arabia (KSA), the Ministry of Health (MOH) is the main provider of healthcare, providing about 60% of health services in the country. The remaining health services are provided either by other government agencies or the private sector. The 2030 Saudi Vision, developed in 2016, highlights new health reforms, including enhancing the quality and safety of healthcare services as well as reducing costs across all stages of service delivery.
The Saudi national accreditation body, called the Central Board of Accreditation for Healthcare Institutions (CBAHI), was established in 2006, aiming at improving compliance of healthcare organizations with quality and safety standards. Despite rapid developments and quality interventions in the Saudi healthcare system, local literature reveals a shortage of assessments of the impact of the national accreditation program on organizational outcomes, including patient safety culture. In this study, we hypothesized that patient safety culture would be more prevalent among healthcare providers postaccreditation rather than preaccreditation. Thus, this study aims to assess the effectiveness of the CBAHI accreditation on patient safety culture in a large public hospital.
Study Design and Setting
A pre- and post evaluation design was used to achieve the aim of the study. Patient safety culture was measured before and after accreditation by the CBAHI in a secondary-tertiary general hospital managed by the KSA MOH: King Fahad Hospital Al-Hofuf. The hospital is located in the eastern province of Saudi Arabia in Alhasa. It has 502 beds and 29 specialized healthcare departments. It covers healthcare institutions in the Hofuf area and its adjoining villages and covers all areas of Alhasa through referrals from primary and secondary care facilities including private, governmental, and nongovernmental hospitals. King Fahad Hospital was accredited by CBAHI in 2010.
To measure the effectiveness of accreditation on patient safety culture, the Agency for Healthcare Research and Quality's hospital survey on patient safety culture (HSOPSC) was used. The HSOPSC consists of 42 items. Items are scored on a five-point Likert scale ranging from strongly disagree to strongly agree and from never to always. Additional background demographic information was included. The HSOPSC is a validated and reliable tool that is widely used in the United States, Europe, and developing countries to evaluate patient safety culture at hospitals. Safety culture dimensions and outcome measures of HSOPSC have been explained previously.
Study Population, Sample Size, and Data Collection
For the current study, we targeted all healthcare providers serving patients directly at the hospital, including physicians, nurses, pharmacists, laboratory specialists/technicians, radiology specialists/technicians, anesthesiology specialists/technicians, and others. We excluded any staff on administrative or extended sick leave and staff that appeared in more than one staffing category or hospital area/unit. The sample size was calculated using a sample calculation formula from Raosoft (www.raosoft.com). Assuming an expected outcome level regarding safety culture domains as 50%, a 95% CI, a margin of error of ±10%, and an estimated population size of 2000, the estimated sample size came to 92; for contingencies, the sample size was inflated to 100 participants. At each phase (three phases) of the study, 100 questionnaires were distributed and collected, for a total of 300 responses all together. The survey was a self-administered questionnaire. In each of the hospital departments, lists of healthcare providers were obtained from the department manager. The sample was selected using a randomizer from Research Randomizer (www.randomizer.org).
Data Management and Statistical Analysis
The collected data were analyzed using the statistical package SPSS Version 22. The HSOPSC is composed of 42 items that measure 12 components (composites), along with the participant characteristics section. Each item was scored using a 5-point scale representing either the level of agreement or frequency of event. Some of these items are negatively worded, and they were reversed when calculating the percentage of positive responses for both individual items and composites. For example, the responses “Strongly agree/Agree” or “Always /Most of the time” are positive responses for positively worded items, whereas for negatively worded items, disagreement or less frequency are indicators of a positive response, so the responses “Strongly disagree/Disagree” or “Never/Rarely” were considered as positive responses. There are four outcome variables: two were measured using a single item for each, namely, the patient safety grade and the number of events reported. The other two were measured using a set of items: frequency of events reported and overall perception of safety. Mean and standard deviation as well as projection pursuit regression were used to measure the level of agreement and the frequency of events. χ2 tests were used to assess the participant characteristics by the phase (baseline assessment, before and after accreditation) of the study. Baseline before accreditation and after accreditation are the overall time periods between the surveys relative to when the hospital was accredited in 2010. ANOVA was used to compare the safety culture composites across the study phases with Tukey's multiple pairwise comparison when applicable. Participant characteristics were compared across study phases using the χ2 test. Each of the four outcome variables were recoded into three categories: the patient safety grade was recoded as “Poor or Failing,” “Acceptable,” or “Excellent/Good,” and the number of events reported was recoded as “No events reported,” “1 to 2 events reported,” or “3 or more events reported.” The mean score of each frequency of event reporting and overall perceptions of safety were recoded as “needs improvement” if the mean was less than 3, “average” if mean was between 3 and 4, and “meeting expectations” if mean was greater than 4. These four outcomes were compared across the study phases using χ2 tests. Four ordinal logistic regression models were carried out to assess the association between each outcome and participant characteristics and the 10 safety composites.
Ethical approval for the study was obtained from the Institutional Review Board at King Abdullah International Medical Research Center (KAIMRC), as well as the participating hospital. The collected data was managed by the research team from King Saud bin Abdulaziz University for Health Sciences. To ensure confidentiality of participants, questionnaires were coded by a third party who removed participant names and replaced with a code. Only completed questionnaires were included in the final analysis. Participants were informed of the purpose of the study, the tool, and that their participation was anonymous and voluntary. After getting their written informed consent, one point of contact was appointed in the hospital so that the participants had one central source of assistance in case they had questions or concerns about the survey. To encourage participation and confidentiality, respondents were instructed to return their questionnaires directly to survey drop-off locations within the hospital.
Table 1 presents participant characteristics. The study included participants from different departments with (22%) from medicine (nonsurgical) and (19%) from the surgery department. The largest proportion of participants (48%) had been working in the target hospital from 1 to 5 years, and (13%) had been working for less than 1 year, (21%) had been working 6 to 10 years, and (18%) had been working in the same hospital for more than 10 years. The majority (64.7%) of participants was registered nurses, 9% were physicians, 9% were technicians (EKG, lab, or radiologists), and 4% were pharmacists. A total of 39% of participants been working in their current specialty or profession from 1 to 5 years, 25% from 6 to 10 years, 29% for more than 10 years, and 7% had been working for less than 1 year. Approximately four in five (80%) participants regularly had direct interaction or contact with patients.
Impact of Accreditation on Safety Culture Composites
Table 2 summarizes the effect of accreditation on safety culture dimensions (composites). The “supervisor/manager expectations and actions promoting safety” and “nonpunitive response to error” had mean scores higher at baseline compared to before accreditation, with p-values of 0.027 and 0.009, respectively. Each of the domains “organizational learning-continuous improvement,” “hospital management support for patient safety,” and “teamwork across hospital units” had mean scores higher at baseline compared to before (p = 0.044, 0.020, < 0.001, respectively) and after accreditation (p = 0.020, < 0.001, < 0.001, respectively). On the other hand, differences exist only between baseline and after accreditation in three composites: “teamwork within hospital units,” “feedback and communication about error,” and “hospital handoffs and transitions,” with p-values of 0.002, 0.009, and 0.010, respectively. It has been found that accreditation had no effect on two composites: “communication openness” and “staffing,” with p-values of 0.32 and 0.849, respectively.
Impact of Accreditation on Safety Culture Outcomes
Table 3 presents the association between study phase and patient safety culture outcomes. The study phase had a significant effect on the four-patient safety culture outcomes with all p-values less than .05. The “frequency of event reporting” had the highest percentage (57%) of meeting expectations before accreditation compared to only 37 and 36% at baseline and after accreditation, respectively. However, only 13% of participants were rated in “overall perceptions of safety” as needing improvement at the baseline, which then increased to 38% and 31% for before and after accreditation, respectively. The percent of participants who reported three or more events increased by phase of study from 8% at baseline to around 14% before accreditation and to around 24% after accreditation. There was a large proportion of participants (74%) who rated the patient safety grade as excellent or very good at the baseline compared to before and after accreditation, with around 46 and 44%, respectively.
Four ordinal logistic regressions were run to estimate the effect of all study variables on the safety culture outcomes, which are presented in Tables 4 and 5. Table 4 presents two outcomes, namely frequency of event reporting and overall perceptions of safety, rated in ascending order on three ordinal levels: needs improvement, within average, and meeting expectations. Accreditation increased the odds of the frequency of event reporting before and after accreditation compared to the baseline (reference) by 7.74 (95% CI = 1.68–35.64) and 2.69 (95% CI = 0.9–7.96), respectively. Almost none of the participant characteristics influenced the frequency of event reporting except the length of experience in current specialty or profession where those with less than 1 year or 6 to 10 years rated much higher compared to the reference group, 16 years or more. A one-unit increase on “feedback and communication about error” increased odds for frequency of event reporting by 3.31 (95% CI = 1.69–6.51). However, none of the remaining safety composites influenced the frequency of event reporting. Similarly, accreditation increased the odds of the overall perceptions of safety before and after the accreditation compared to the baseline by 2.91 (95% CI = 0.63–13.53) and 4.39 (95% CI = 1.42–13.56), respectively. The overall perceptions of safety were different across the work area/unit; intensive care unit workers had the highest OR, with 21.33 (95% CI = 0.63–13.53) compared to “others” followed by “medicine (nonsurgical)” and then “emergency department.” Only two safety composites had an effect of the overall perceptions of safety: “supervisor/manager expectations and actions promoting safety” and “teamwork across hospital units” with an OR of 5.15 (95% CI = 2.19–12.11) and 3.44 (95% CI = 1.54–7.69), respectively.
Table 5 presents the distribution of the parameters, “number of events reported” and “patient safety grade.” Even though there was no significant effect of accreditation on number of events reported, it was much lower before accreditation compared to baseline and higher after accreditation, with an OR of 0.25 (95% CI = 0.05–1.19) and 1.41 (95% CI = 0.45–4.38), respectively. As mentioned before, participant characteristics had a minor effect on safety culture, which also applies to the number of events reported. Only the “staffing” composite had an effect on the number of events reported, with an OR of 0.41 (95% CI = 0.20–0.82). Accreditation had an inverse effect on patient safety grade, where before and after accreditation had lower ORs compared to the baseline, with 0.13 (95% CI = 0.02–0.70) and 0.10 (95% CI = 0.03–0.37), respectively. Only the “feedback and communication about error” composite had an effect on the patient safety grade, with an OR of 2.87 (95% CI = 1.31–6.26).
This is, to our knowledge, the first study addressing the impact of a national accreditation intervention on patient safety culture dimensions and outcomes. The study findings are of utmost importance for any national accreditation programs, such as CBAHI, or others worldwide, as it provides valuable information on how implementing accreditation can have an impact on the patient safety culture at public hospitals.
With regard to accreditation, our analysis revealed no effect of accreditation on communication openness and staffing composite scores. For all other composites, the baseline evaluation of patient safety culture was higher compared to before and after accreditation. This reflects how our perceptions on the level of patient safety culture might be higher than it is in actuality. The external accreditation process assesses the level of performance in relation to established standards that might not be clear before being engaged in the accreditation process. Overall, the study phase (before and after accreditation) had a significant effect on the four safety culture outcomes. Frequency of event reporting and frequency of events reported, particularly three or more event reports, had a higher percentage of participants that needed improvement after accreditation. Accreditation increased the odds of the frequency of event reporting before and after the accreditation by 7.74 and 2.69 (95% CI), respectively. This may be because accreditation is a quality improvement process that encourages staff to report errors. Also, staff who had been working less than 10 years in their profession reported more events compared to staff who had been working for 16 years or more in the current profession. A possible explanation is that long-term staff might be becoming more accepting of normalization of deviance, possibly because nothing gets done or changed. That is an indicator of more troubling concerns about the safety culture. More studies are needed to investigate the reasons.
The perception of patient safety grade had a smaller proportion of participants in the “excellent or very good” category after accreditation. This may be because being exposed to the required safety level makes staff aware of the real safety grade. Moreover, a one-unit increase on “feedback and communication about error” increased the odds of increased frequency of event reporting by 3.31 (95% CI). Open communication and feedback encourage people to report more events.[2,17]
Findings show that accreditation increased the likelihood of having positive overall perceptions of safety before and after the accreditation compared to the baseline by 2.91 and 4.39 (95% CI), respectively. Overall perceptions of safety had lower percentage of participants needing improvement after accreditation. Being part of the accreditation assessment and knowing the procedures and systems that are needed to ensure the prevention of errors improves the healthcare provider's overall perception of safety. This overall perception of safety was shown to have been affected by “supervisor/manager expectations and actions promoting safety” and “teamwork across hospital units.” These two composites are directly connected by the procedures and systems to prevent errors from happening. Therefore, we found an impact from these two domains on overall perception of safety.
Similar to other studies, working in the more critical units have an effect on the overall perception of safety.[18,19] The intensive care unit, followed by nonsurgical medicine and the emergency department had the highest ORs on overall perceptions of safety. These critical units have a high impact on the safety level, safety perception, and safety culture.
With regard to number of events reported, we found no significant effect of accreditation on the number of events reported. These findings are consistent with results from Lam et al. who showed that “accreditation is not associated with better patient outcomes as the focus of organizations has been on improving structural factors and clinical processes rather than actually improving patient outcomes.” However, departments with staffing problems were 59% less likely to report events in our study. It may be speculated that this is so because having staffing problems will not allow staff to focus on patient safety challenges, where reporting of events is one of the important issues. Staffing on its own may not be the direct reason; also having a nonpunitive environment where employees do not fear reporting events might contribute to these results. A punitive work environment was reported to be an area for improvement in Lebanon, Palestine, and Saudi Arabia.[8,18,21] Moreover, accreditation on its own required additional work from staff to do extra administration standards work such as quality interventions and improvement projects, frequent meetings, etc. This put staff under additional pressure that may have highlighted staffing as an issue in hospital patient safety culture.
The correlational data found that accreditation had an opposite effect on perception of the patient safety grade, where as hospital staff before accreditation were 87% less likely to be classified as “excellent or very good” on the perception of patient safety grade. Hospital staff after accreditation were 90% less likely to be deemed “excellent or very good” on perception of patient safety grade. This is inconsistent with results of other research that states that accreditation has a positive effect on patient safety. Of note is that one-unit increase on “feedback and communication about error” had a three times higher OR of reporting a better patient safety grade. Having a positive perception about feedback and communication about errors increased the likelihood of getting a better participant's perception grade, as it encourages staff to talk, discuss, and learn from mistakes.
A few limitations were encountered in this study. A certain degree of recall bias and social desirability is expected in self-reported questionnaires. We think that including more public hospitals from other geographic regions might have boosted the generalizability of the study findings and representativeness of the sample.
To the best of our knowledge, this is the first study to conduct an impact assessment of a national accreditation program on patient safety culture dimensions and outcomes in Saudi Arabia. Results are mixed on the impact of accreditation on safety culture. Accreditation has significant impact on some patient safety culture dimensions and a more significant effect on safety culture outcomes. Thus, this study is a good step in providing good empirical evidence on the Saudi national accreditation program and valuable insights on accreditation impact worldwide regarding patient safety culture in hospitals.
AVAILABILITY OF DATA AND MATERIALS
The data that support the findings of this study are available from King Saud bin Abdulaziz University for Health Sciences, but restrictions apply to the availability of these data, which were used under license for the current study and thus are not publicly available. Data are, however, available from the authors upon reasonable request and with permission of King Saud bin Abdulaziz University for Health Sciences.
We particularly thank KAIMRC and the Ministry of National Guard–Health Affairs for their support and funding of this project. KAIMRC did not have a role in the study design, data collection, analysis, or interpretation, or in writing the manuscript. Moreover, we thank all healthcare providers serving patients directly at the hospital that participated in our study.
Source of Support: King Abdullah International Medical Research Center. Conflict of Interest: None.