Assessing Patient Safety Culture of Internal Medicine House Staff in an Academic Teaching Hospital

The Institute of Medicine (IOM) suggests that the biggest challenge to moving toward a safer health care system is changing the patient safety culture (PSC) from one in which people are blamed for errors to one in which errors are treated as opportunities to improve the system and prevent harm.1 Patient safety culture examines how the perceptions, behaviors, and competencies of individuals and groups determine an organization's commitment, style, and proficiency in health and safety management.2 PSC assessments have been used by organizations to determine targets for interventions to improve patient safety, evaluate the success of patient safety interventions, fulfill regulatory requirements, and conduct benchmarking.3,4 Hospitals with well-developed PSC have been shown to reduce lengths of stay, reduce medication reconciliation errors, and improve nursing staff retention.5 

The IOM also recommends that health care organizations assess their PSC, redesign systems to reduce opportunities for error, and establish comprehensive patient safety programs to increase detection of adverse events.6,7 Toward this end, our internal medicine residency program submitted a successful application in 2006 to the Accreditation Council for Graduate Medical Education (ACGME) to become an Educational Innovation Project (EIP)-recognized training program.8 The overall goal of the EIP project is to facilitate competency-based education and outcomes assessment in those internal medicine programs that are well suited and ready for innovation. The reporting of house staff PSC fulfills the EIP requirement for assessing and reporting outcomes. More recently, the ACGME Program Requirements for Resident Education in Internal Medicine, state that all programs demonstrate that there is a culture of patient safety and continuous quality improvement.9 

The purpose of this article is to describe the initial set of steps taken by our internal medicine residency program to assess patient safety in the context of our EIP proposal. We assessed hospital PSC from the perspective of internal medicine house staff using a standardized and previously validated instrument in order to raise awareness of patient safety issues, identify targets for interventions to improve patient safety, fulfill the EIP requirement for assessing and reporting outcomes, and establish our own program-specific benchmark data. The primary objective of this study was to assess PSC in internal medicine house staff at our institution, and to compare the results across postgraduate years (PGYs) of training and to national hospital benchmark data. The secondary objective was to determine a list of key patient safety topics to be included in a patient safety curriculum.

We used the Agency for Healthcare Research and Quality (AHRQ) Hospital Survey on Patient Safety Culture (HSOPSC), a validated instrument that has 12 safety culture dimensions and 2 outcome measures (table 1). This survey is primarily useful for assessing the safety culture of a hospital as a whole, or for specific units within hospitals, and not for assessing individual patient safety perceptions or skills. The 12 safety culture dimensions measure the perception of the respondent with respect to the safety of patients in their patient care unit (9 dimensions) and also their overall view of the safety of patients in the hospital in its entirety (3 dimensions). Each dimension has 3 to 5 questions and uses a 5-point Likert scale of agreement (“Strongly disagree” to “Strongly agree”) or frequency (“Never” to “Always”).

The outcome measures use single-item responses about the number of events reported (defined as errors of any type, regardless of whether they result in patient harm) and the overall patient safety grade (“Excellent” to “Failing”). Previous and current analyses have shown that all 12 dimensions had acceptable levels of internal consistency (Cronbach's alpha  =  .63 to .84 and .31 to .83).10 

The AHRQ established the HSOPSC Comparative Database as a central repository for survey data from hospitals that have administered the AHRQ patient safety culture survey instrument. The database serves as a resource for organizations wishing to compare their patient safety culture survey results with those of other hospitals. To create publicly accessible benchmark data, the survey was administered to 108 621 hospital employees from 382 hospitals containing 8 279 internal medicine units across the United States between October 2004 and July 2006.11 

The HSOPSC was pilot tested for use by 4 internal medicine chief residents. Based on their suggestions, the following changes were made to create the House Staff Patient Safety Culture (H-PSC) survey: (1) an additional definition for “event reporting” was added to orient participants; (2) the following phrases were modified–“staff” was replaced with “house staff,” “hospital work area” or “unit” was replaced with “hospital,” and “agency/temporary staff” was clarified to mean moonlighters or cross-covering house staff; and (3) the demographics section was expanded to include information about gender, age, medical school training, and future career plans. No other changes were made to the HSOPSC. The H-PSC uses the same question format, question order, and response options as the HSOPSC (see Appendix online).

An additional section of the survey was added listing 11 potential patient safety topics to be included in the future development of a patient safety curriculum. The topics for this section were based primarily on a list of patient safety practices published by the AHRQ.12 Additional topics were included based on prior work or expertise of the study coauthors.13,14 In order to prioritize the development of the initial safety curriculum, each respondent was asked to select 5 safety topics.

After receiving approval from the institutional review board, the survey was distributed to a convenience sample comprising all (N  =  68) PGY-2 and PGY-3 internal medicine house staff at the University of Pittsburgh Medical Center Presbyterian Hospital training program between December 2006 and February 2007. The PGY-1 house staff were excluded from the study because they had been at the training program for as little as 6 months, and the literature suggests that it may take at least 1 year to appreciate and assess institutional PSC.15 Each survey packet contained a cover letter, a copy of the survey, and a ballot to enter into a drawing for one of two $100 gift certificates. A second packet was placed in the mailboxes of nonrespondents if the survey was not received within 4 weeks.

Guidelines for computing patient safety dimensions for the HSOPSC have already been published, and we used the same scoring methods so that the results of this study could be compared with their benchmark data.10,16 The scoring consists of several steps. First, to calculate response rates, the number of respondents per PGY was divided by the total number of potential respondents per PGY. Next, individual responses to each survey question were classified as a “positive” response if the actual response was “Agree/Strongly agree” or “Most of the time/Always” in positively worded questions, and “Disagree/Strongly disagree” or “Rarely/Never” in negatively worded questions. For each respondent, domain scores were computed by taking the number of positive responses for each domain, dividing it by the number of questions in the same domain, and multiplying by 100. The PSC domain scores could range from 0 to 100, where lower scores represented worse (ie, less well-developed) PSC.

The 12 domain scores were summarized using appropriate descriptive statistics for all respondents and by level of training. We used one and two sample t tests to compare domain score means between PGYs and against the published national benchmarks. The safety curriculum topics were calculated using appropriate descriptive statistics. For all statistical analyses, we used SAS version 9 for Windows (SAS Institute Inc, Cary, NC).

The overall response rate was 85.3% (58 of 68), with PGY-2 and PGY-3 response rates of 85.7% and 84.8%, respectively. Most respondents (70%) were between 26 and 30 years of age and had graduated from a US medical school (74.1%). Most of the house staff (72.2%) planned on entering into a fellowship program upon completion of their residency training. Additional demographic information is summarized in table 2.

The mean PSC composite scores, reflecting the perceptions of the house staff about their hospital, are shown in table 3. The PGY-3 staff scored lower than PGY-2 staff in 2 of the 12 PSC dimensions: supervisor expectations and actions promoting patient safety, and teamwork across units (P < .05).

The mean PSC composite scores of internal medicine house staff compared with all-hospital and internal medicine unit benchmarks are shown in table 4. House staff had lower PSC in 6 dimensions when compared to the all-hospital benchmark data, namely communication openness, feedback and communication about error, frequency of events reported, handoffs and transitions, overall perceptions of safety, and teamwork within units (P < .05). House staff had lower PSC in 4 dimensions when compared with medicine unit benchmarks, namely communication openness, feedback and communication about error, frequency of events reported, and handoffs and transitions (P < .05).

Additionally, house staff reported fewer adverse events when compared with the national benchmark data. Sixty-four percent of house staff did not report an adverse event during the 12-month period prior to survey administration, as compared with 53% of the benchmark respondents. Overall, 70% of the house staff and benchmark respondents rated their institutional patient safety grade as excellent or very good. However, only 6% of the house staff rated the hospital as having an excellent patient safety grade, compared with 22% of the benchmark respondents.

The 3 safety topics most frequently selected by internal medicine house staff to structure their initial patient safety curriculum included (1) adverse drug events (79.3%), (2) adverse events related to transition of care (72.4%), and (3) disclosing medical errors to patients and family (55.2%) (table 5).

This study systematically assessed PSC in an internal medicine residency program and has 2 important findings. First, PGY-2 and PGY-3 internal medicine house staff were in general agreement in 10 of the 12 PSC dimensions measured in the survey. Second, in half of the PSC dimensions, the PSC composite scores for the house staff were significantly lower than national benchmark data.

The PGY-2 internal medicine house staff rated both the dimensions for supervisor expectations and actions promoting patient safety and teamwork across units significantly higher than their PGY-3 counterparts. These findings are not entirely surprising as the differences can be potentially attributed to the increased hospital ward experiences and changing roles and responsibilities as house staff progress in their training. For example, PGY-2 house staff usually have more direct supervision by attending physicians, compared with PGY-3 residents. The increased supervision provides additional opportunities for feedback about performance on patient management and safety issues.

Similarly, the difference in the teamwork across units dimension could be because in our institution PGY-3 staff have more experience with other rotations that require significantly more cooperation among hospital units for patient care. The PGY-3 staff may have a lower score in this dimension because of the challenges that they have faced in their interactions with other specialties in the hospital (eg, transferring a patient to or from the intensive care unit or to a surgical floor). These challenges, combined during a greater than 2-year period, may have led to the perception that there was decreased cooperation among the different hospital units.

Difficulty with communication and teamwork was a common theme associated with the majority of PSC dimensions that were significantly lower in internal medicine house staff as compared with hospital benchmarks. In particular, our study suggests that house staff perceive that there are significant problems with handoffs and transitions of care both within and across units. Communication failures can compromise optimal patient care and are one of the most common root causes of medical error and adverse events.17–19 For example, Singh et al20 studied medical errors involving trainees and found that a major cause involved teamwork-related factors, the most prevalent ones being lack of supervision and handoffs. Moreover, many studies have demonstrated how a lack of clear communication during transitional care and sign-out are critical causes of error and adverse events.21–26 

House staff play a pivotal role and have a responsibility to communicate information about both near-misses and adverse events.27,28 However, it has been shown that trainee physicians are sometimes reluctant to communicate errors and adverse events. Barriers to incident reporting include the fear of blame and retribution (ie, the “culture of blame”), the uncertainty about reporting needs and mechanisms, concern about the time required, and lack of feedback once an incident is reported.29–34 Therefore, it is critical to improve the climate of “speaking up” and break free from the “blame cycle” and promote a “reporting culture.”19,29,30 

House staff selected adverse drug events, adverse events related to transitions in care, and disclosing medical errors to patients as the topics to be included in a patient safety curriculum, likely because they commonly occur and are often associated with negative consequences. For example, a meta-analysis of fatal adverse drug events suggest that these events are between the fourth and sixth leading causes of death in the United States, are associated with prolonged lengths of stay, and excess health care expenditures.35,36 Adverse events related to transitions in care may be a result of processes or factors at the level of the health system, patient, or clinician.37,38 Finally, despite the frequency and potential impact, physicians are rarely provided with the skills necessary to disclose them appropriately and prevent medical errors.39,40 

Our study has several strengths. First, it systematically assessed PSC in an internal medicine residency program. Second, the instrument used to measure house staff PSC is based on the same items and dimensions as the previously validated and reliable HSOPSC survey instrument developed by the AHRQ. In addition, minimal changes were made to the original survey, thereby preserving the instrument's psychometric properties. Last, the response rate to this survey was 85.3%, which is higher than most surveys that have assessed PSC in a variety of clinical settings that range from 18% to 71%.41–43 

Our study has several potential limitations that should be acknowledged. The sample size is small and represents only a single institution. This may have limited the statistical power to detect significant differences across PGYs. There are also limitations inherent to conducting survey-based research. These limitations include response and nonresponse bias as well as recall bias. Overall, these factors may reduce the generalizability of our results to other institutions. Another limitation could have been the inclusion of a nonstandardized list of general (ie, not house staff-specific) patient safety topics previously published by the AHRQ. It is possible that having open-ended questions about which patient safety topics were most important may have provided more valid results.

Our house staff training program and institution understand the complexities of the IOM reports on safety and quality, and have taken significant steps toward improving patient safety culture. Communication and transmission of these interventions to the frontline health care professionals are continuing to be refined and implemented. To improve PSC, we have developed a multifaceted patient safety curriculum, part of which has been successfully implemented in the house staff program. These interventions include grand rounds presentations; morbidity and mortality conferences incorporated into morning reports; patient safety noon conferences; academic detailing by faculty and resident champions; simulation-based education on procedures with real-time feedback; direct 24-hour supervision on performing procedures from designated specialists; multidisciplinary teams composed of house staff, faculty, pharmacists, nurses, and case managers who work together to enhance patient care and safety; and point-of-care reminders and computerized physician order entry with computerized decision support. We are aware that curricular changes and educational strategies may not improve all issues surrounding PSC but will require resources and systems beyond the residency program.

We believe that the H-PSC survey can help meet the new Residency Review Committee for Internal Medicine standard that requires programs to demonstrate that there is a culture of patient safety. Additional studies are also needed to assess PSC in other types of residency training programs (eg, general surgery, pediatrics, family medicine, geriatric medicine) and clinical environments (eg, ambulatory care, home-based primary care, nursing home care) to develop benchmarking data and identify targets for interventions to improve PSC. In addition, research should also be conducted to further investigate potential differences in PSC between house staff and attending physicians.

The PGY-2 and PGY-3 internal medicine house staff at our institution were in agreement with each other on most of the PSC dimensions. Overall, house staff perceptions of hospital PSC was significantly lower than national hospital benchmark data for half of the dimensions. The results of this study will be used to establish internal PSC benchmarks to track temporal trends. It is anticipated that greater exposure to patient safety programs will improve PSC scores and promote the education of resident physicians invested in improving the safety and efficiency of patient care.