Appendix A1. General Characteristics of Sponsoring Institutions (SIs)

A1.1. Sponsoring Institution Distribution by Region and Type

A1.2. Sponsoring Institution Distribution by Number of ACGME-Accredited Residency and Fellowship Programs, and Participating Sites

Note. aLimited to three SIs in Puerto Rico.

A1.3. Number and Distribution of Core Faculty Members at Sponsoring Institutions by Specialty Grouping

A1.4. Number and Distribution of Residents and Fellows at Sponsoring Institutions by Gender, Level of Training, and Specialty Grouping

Appendix A2. General Characteristics of Clinical Learning Environmentsa

A2.1. Clinical Learning Environment Distribution by Type of Ownership and Services

A2.2. Clinical Learning Environment Distribution by Beds and Staffing

Note. aBased on the 2013 American Hospital Association Annual Survey.

bMember of Council of Teaching Hospital of the Association of American Medical Colleges.

cIncludes psychiatric, rehabilitation, acute long-term care hospital, children's general medical and surgical, and other types of services.

dPhysicians, residents, interns, and other trainees omitted from staff count.

eFull-time and part-time personnel only; excludes full-time and part-time equivalent personnel.

Appendix A3. Clinical Learning Environments Visited: Number of Programs at Sitea

Appendix A4. CLER Visits: Characteristics of Groups Intervieweda

A4.1. Selected Characteristics of Residents and Fellows in the Group Interviews

A4.2. Selected Characteristics of Faculty Members and Program Directors in the Group Interviews

Appendix B. Selected Results from Resident and Fellow Group Interviewsa

B1. Percentage of residents and fellows who reported knowing the clinical site's priorities in the area of patient safety

Percent of Total Surveyedb (n=8,663)

Percent of Total Surveyedb (n=8,663)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,663)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,663)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,663)

Note. aAll percentages are rounded to whole numbers (values of .50 or above are rounded to the next highest whole number).

bMissing data=92.

cResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

dDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.01.

**Statistically significant at p<.0001.

B2. Percentage of residents and fellows who reported knowing the clinical site's process for reporting an adverse event, near miss/close call, or unsafe condition in patient care

Percent of Total Surveyeda (n=8,616)

Percent of Total Surveyeda (n=8,616)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,616)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,616)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,616)

Note. aMissing data=139.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

**Statistically significant at p<.0001.

B3. Percentage of residents and fellows who reported experiencing an adverse event, near miss/close call, or unsafe condition

Percent of Total Surveyeda (n=8,629)

Percent of Total Surveyeda (n=8,629)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,629)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,629)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,629)

Note. aMissing data=126.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.01.

**Statistically significant at p<.0001.

B4. Percentage of residents and fellows who reported experiencing an adverse event, near miss/close call, or unsafe condition and submitted a report through the clinical site's reporting system

Percent of Total Surveyeda (n=5,768)

Percent of Total Surveyeda (n=5,768)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=5,768)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=5,768)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=5,768)

Note. aMissing data=98.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.0001.

B5. Percentage of residents and fellows who reported a near miss/close call event

Percent of Total Surveyeda (n=8,665)

Percent of Total Surveyeda (n=8,665)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,665)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,665)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,665)

Note. aMissing data=90.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.01.

**Statistically significant at p<.0001.

B6. Percentage of residents and fellows who reported receiving feedback on the outcome of a report submitteda through the clinical site's reporting system

Percent of Total Surveyedb (n=4,828)

Percent of Total Surveyedb (n=4,828)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=4,828)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=4,828)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=4,828)

Note. aReport submitted by resident or fellow or through a nurse or supervisor.

bMissing data=426.

cResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

dDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.01.

**Statistically significant at p<.0001.

B7. Percentage of residents and fellows (PGY-3 and above) who reported participating in an interprofessional (physicians, nurses, administrators, others) investigation of a patient event (e.g., root cause analysis)

Percent of Total Surveyeda (n=6,369)

Percent of Total Surveyeda (n=6,369)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=6,369)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=6,369)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=6,369)

Note. aMissing data=120.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.01.

**Statistically significant at p<.0001.

B8. Percentage of residents and fellows (PGY-2 and above) who reported knowing the clinical site's priorities in the area of quality improvement

Percent of Total Surveyeda (n=8,315)

Percent of Total Surveyeda (n=8,315)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,315)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,315)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,315)

Note. aMissing data=112.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

**Statistically significant at p<.01.

***Statistically significant at p<.0001.

B9. Percentage of residents and fellows (PGY-2 and above) who reported participating in a quality improvement project of their own design or one designed by their program or department

Percent of Total Surveyeda (n=8,298)

Percent of Total Surveyeda (n=8,298)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,298)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,298)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,298)

Note. aMissing data=129.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

**Statistically significant at p<.01.

***Statistically significant at p<.0001.

B10. Percentage of residents and fellows who reported participating in a quality improvement project linked to one or more of the clinical site's quality improvement goals

Percent of Total Surveyeda (n=6,154)

Percent of Total Surveyeda (n=6,154)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=6,154)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=6,154)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=6,154)

Note. aMissing data=135.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

**Statistically significant at p<.0001.

B11. Percentage of residents and fellows who reported being engaged in interprofessional quality improvement teams (e.g., nurses, administrators, pharmacists, etc.) while participating in a quality improvement project directly linked to one or more of the clinical site's quality improvement goals

Percent of Total Surveyeda (n=3,133)

Percent of Total Surveyeda (n=3,133)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=3,133)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=3,133)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=3,133)

Note. aMissing data=88.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 88% or more of the 297 CLEs.

*Statistically significant at p<.01.

**Statistically significant at p<.0001.

B12. Percentage of residents and fellows who reported having ready access to organized systems for collecting and analyzing data for the purpose of quality improvement

Percent of Total Surveyeda (n=8,549)

Percent of Total Surveyeda (n=8,549)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,549)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,549)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,549)

Note. aMissing data=206.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

**Statistically significant at p<.01.

B13. Percentage of residents and fellows who reported knowing the clinical site's priorities with regard to addressing health care disparities

Percent of Total Surveyeda (n=8,607)

Percent of Total Surveyeda (n=8,607)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,607)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,607)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,607)

Note. aMissing data=148.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

**Statistically significant at p<.0001.

B14. Percentage of residents and fellows who reported following a standardized process for handling transitions of care during hand-offs between shifts

Percent of Total Surveyeda (n=8,471)

Percent of Total Surveyeda (n=8,471)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,471)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,471)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,471)

Note. aMissing data=284.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

**Statistically significant at p<.01.

***Statistically significant at p<.0001.

B15. Percentage of residents and fellows who reported following a standardized process that included a standardized written template for communication for handling transitions of care during hand-offs between shifts

Percent of Total Surveyeda (n=7,262)

Percent of Total Surveyeda (n=7,262)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=7,262)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=7,262)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=7,262)

Note. aMissing data=163.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.0001.

B16. Percentage of residents and fellows who reported having been placed, or witnessing one of their peers placed, in a situation where they believed there was inadequate supervision at the clinical site (e.g., the attending wasn't available)

Percent of Total Surveyeda (n=8,542)

Percent of Total Surveyeda (n=8,542)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,542)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,542)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,542)

Note. aMissing data=213.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.01.

**Statistically significant at p<.0001.

B17. Percentage of residents and fellows who reported knowing what they were allowed to do with and without direct supervision

Percent of Total Surveyeda (n=8,621)

Percent of Total Surveyeda (n=8,621)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,621)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,621)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,621)

Note. aMissing data=134.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

**Statistically significant at p<.01.

***Statistically significant at p<.0001.

B18. Percentage of residents and fellows who reported having an objective way to know what procedures residents and fellows from other services were allowed to do with and without direct supervision when they consulted on patients

Percent of Total Surveyeda (n=8,080)

Percent of Total Surveyeda (n=8,080)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,080)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,080)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,080)

Note. aMissing data=675.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.01.

**Statistically significant at p<.0001.

B19. Percentage of residents and fellows who reported that they would power through to hand-off if placed in a situation in which they were impaired by fatigue, maxed out on caffeine, and a 15 minute nap had no effect

Percent of Total Surveyeda (n=8,387)

Percent of Total Surveyeda (n=8,387)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,387)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,387)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,387)

Note. aMissing data=368.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

**Statistically significant at p<.0001.

B20. Percentage of residents and fellows who reported having attended institutional training on topics of professionalism and ethics

Percent of Total Surveyeda (n=8,631)

Percent of Total Surveyeda (n=8,631)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,631)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,631)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,631)

Note. aMissing data=124.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs. *Statistically significant at p<.01.

B21. Percentage of residents and fellows who reported that their clinical site provided a supportive, non-punitive environment for coming forward with concerns regarding honesty in reporting (e.g., patient data, duty hours)

Percent of Total Surveyeda (n=8,592)

Percent of Total Surveyeda (n=8,592)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,592)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,592)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,592)

Note. aMissing data=163.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

B22. Percentage of residents and fellows who reported having felt pressured to compromise their honesty or integrity to satisfy an authority figure during their training at the clinical site

Percent of Total Surveyeda (n=8,609)

Percent of Total Surveyeda (n=8,609)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,609)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,609)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,609)

Note. aMissing data=146.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

B23. Percentage of residents and fellows who reported they had documented a history or physical finding in a patient chart that they did not personally elicit at the clinical site (e.g., copying and pasting from another note)

Percent of Total Surveyeda (n=8,488)

Percent of Total Surveyeda (n=8,488)

Close modal

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,488)

Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,488)
Percent by Resident and Fellow Characteristics and CLE Characteristics (n=8,488)

Note. aMissing data=267.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

cDistribution includes 90% or more of the 297 CLEs.

*Statistically significant at p<.05.

**Statistically significant at p<.0001.

Appendix C. Selected Qualitative Results from Site Visit Reportsa

C1. Percentage of CLEs by proportion of resident and fellow knowledge of basic patient safety terminology and principlesb

Percent by CLE Region, Bed Size, and Ownership (% Most)

Percent by CLE Region, Bed Size, and Ownership (% Most)
Percent by CLE Region, Bed Size, and Ownership (% Most)

Note. aResults based on 297 site visit reports; all percentages are rounded to whole numbers (values of .50 or above are rounded to the next highest whole number).

bMissing data=73. It should be noted that missing data is largely due to the development and refinement of a formal template of the written site visit report in the early stages of program implementation.

cResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

C2. Percentage of CLEs that tracked the number of patient safety event reports submitted by residents and fellowsa

Percent by CLE Region, Bed Size, and Ownership (% Tracks Reporting)

Percent by CLE Region, Bed Size, and Ownership (% Tracks Reporting)
Percent by CLE Region, Bed Size, and Ownership (% Tracks Reporting)

Note. aMissing data=53. It should be noted that missing data is largely due to the development and refinement of a formal template of the written site visit report in the early stages of program implementation.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

C3. Percentage of CLEs where residents and fellows frequently submitted patient safety event reportsa

Percent by CLE Region, Bed Size, and Ownership (% Infrequently Submit Reports)

Percent by CLE Region, Bed Size, and Ownership (% Infrequently Submit Reports)
Percent by CLE Region, Bed Size, and Ownership (% Infrequently Submit Reports)

Note. aMissing data=29. It should be noted that missing data is largely due to the development and refinement of a formal template of the written site visit report in the early stages of program implementation.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

C4. Percentage of CLEs by proportion of resident and fellow knowledge of basic quality improvement conceptsa

Percent by CLE Region, Bed Size, and Ownership (% Most)

Percent by CLE Region, Bed Size, and Ownership (% Most)
Percent by CLE Region, Bed Size, and Ownership (% Most)

Note. aMissing data=45. It should be noted that missing data is largely due to the development and refinement of a formal template of the written site visit report in the early stages of program implementation.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

C5. Percentage of CLEs with cultural competency training for residents and fellowsa

Percent by CLE Region, Bed Size, and Ownership (% Tailored Training)

Percent by CLE Region, Bed Size, and Ownership (% Tailored Training)
Percent by CLE Region, Bed Size, and Ownership (% Tailored Training)

Note. aMissing data=32. It should be noted that missing data is largely due to the development and refinement of a formal template of the written site visit report in the early stages of program implementation.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

C6. Percentage of CLEs with hand-off processes that were standardized across programs, based on direct observationsa

Percent by CLE Region, Bed Size, and Ownership (% All Standardized)

Percent by CLE Region, Bed Size, and Ownership (% All Standardized)
Percent by CLE Region, Bed Size, and Ownership (% All Standardized)

Note. aMissing data=18. It should be noted that missing data is largely due to the development and refinement of a formal template of the written site visit report in the early stages of program implementation.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

C7. Percentage of CLEs by mechanism used for identification of resident and fellow competency to perform clinical procedures, as reported by nursesa

Percent by CLE Region, Bed Size, and Ownership (% Trust Primarily)

Percent by CLE Region, Bed Size, and Ownership (% Trust Primarily)
Percent by CLE Region, Bed Size, and Ownership (% Trust Primarily)

Note. aMissing data=9. It should be noted that missing data is largely due to the development and refinement of a formal template of the written site visit report in the early stages of program implementation.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

C8. Percentage of CLEs where patient safety and quality leaders recalled patient safety events related to resident and fellow fatiguea

Percent by CLE Region, Bed Size, and Ownership (% Recalled Events)

Percent by CLE Region, Bed Size, and Ownership (% Recalled Events)
Percent by CLE Region, Bed Size, and Ownership (% Recalled Events)

Note. aMissing data=67. It should be noted that missing data is largely due to the development and refinement of a formal template of the written site visit report in the early stages of program implementation.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

C9. Percentage of CLEs where executive leadership recalled incidents concerning professionalisma

Percent by CLE Region, Bed Size, and Ownership (% Recalled Events)

Percent by CLE Region, Bed Size, and Ownership (% Recalled Events)
Percent by CLE Region, Bed Size, and Ownership (% Recalled Events)

Note. aMissing data=23. It should be noted that missing data is largely due to the development and refinement of a formal template of the written site visit report in the early stages of program implementation.

bResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.

C10. Percentage of CLEs where disruptive or disrespectful behavior was reported across more than one clinical unit

Percent by CLE Region, Bed Size, and Ownership (% Reported Disruptive Behavior)

Percent by CLE Region, Bed Size, and Ownership (% Reported Disruptive Behavior)
Percent by CLE Region, Bed Size, and Ownership (% Reported Disruptive Behavior)

Note. aResults from CLEs in Puerto Rico (n=3) have been omitted to protect their anonymity.