Redesigning the Clinical Learning Environment to Improve Interprofessional Care and Education: Multi-Method Program Evaluation of the iPACE Pilot Unit

The responsibility of graduate medical education (GME) is to prepare residents to meet the challenges of working in a rapidly evolving health care environment. The Accreditation Council for Graduate Medical Education (ACGME) Common Program Requirements mandate that “Residents must care for patients in an environment that maximizes communication [and] includes the opportunity to work as a member of effective interprofessional (IP) teams.”¹ 

Identifying methods to train high-functioning teams remains a challenge. While integration of IP care into traditional inpatient teaching services has been described as a “utopian” ideal,²  practice-based interventions such as interdisciplinary rounds have not been shown to improve collaborative behavior, clinician well-being/burnout, clinical processes, care efficiency, or patient outcomes.^3–6 

The Interprofessional Partnership to Advance Care and Education (iPACE) model is an exploratory educational pilot attempting to meet this need. In 2016, Maine Medical Center (MMC) was awarded an ACGME Pursuing Excellence in Innovation grant to redesign the clinical learning environment (CLE) to promote IP care and education. MMC approached this project as a quality improvement (QI) project or “learning laboratory.” The iPACE model is the result of an iterative design process that incorporated recommendations from the literature on how to optimize IP collaborative practice in health care.^7–12  Elements selected for inclusion from other published IP care models included geographic co-location of clinicians and patients; structured, scheduled IP rounds; a physician–nurse leadership dyad that uses unit-level quality and safety data to inform care; and IP educational sessions to promote cross-discipline learning and collaboration efforts.^9,13–15  The iPACE model was further refined with input from a formal systems engineering analysis of MMC GME processes conducted by Northeastern University's Healthcare Systems Engineering Institute.

Many key elements of IP team care have been characterized in the literature, including shared goals, clear roles, mutual trust, effective communication, measurable processes and outcomes, and organizational support, but there is no single, universally accepted definition or measure of the construct of IP team care.^16,17  The evaluation plan of the iPACE pilot contained measures of team characteristics, including an adaptation of the Mini-Z (table 1) to assess team well-being. It has been reported that such metrics may be more sensitive to IP interventions than traditional clinical outcome measures (ie, length of stay or readmission rates).^18,19  Identification of a tool to evaluate the construct of teaming was more difficult.

Relational coordination (RC), developed by Jody Hoffer Gittell, PhD, is a well-established theoretical framework and assessment strategy. RC states that the coordination necessary for ideal IP team care requires “the management of interdependencies between the people who perform those tasks.”²⁰  In this framework, good teamwork is reliant on a “mutually reinforcing process of communicating and relating for the purpose of task integration,” and captures key elements of existing conceptions of IP care, including aspects of interpersonal relationships (shared knowledge, shared goals, and mutual respect), and interpersonal communication (frequency, timeliness, accuracy, and problem-solving).²¹  RC has been measured empirically in a variety of organizational settings, including health care. High levels of RC have been found to predict team performance outcomes, including health care quality, improved clinical outcomes, improved patient and clinician satisfaction and engagement, and shorter length of stay.^22–28  RC has been previously used as a pre-/post-evaluation of a successful IP curriculum designed to improve teaming skills.²⁹  For these reasons, RC was chosen as a valid and useful conceptual and measurement framework for evaluating the impact of iPACE on teaming.

The overall objective of the iPACE pilot was to obtain preliminary evidence on the feasibility and effectiveness of a multipronged intervention to promote IP care and education on an inpatient internal medicine (IM) teaching unit. The specific objectives were to determine the impact the intervention had on teaming, team member experience (including well-being and perceived quality of care and education), and patient experience.

The iPACE model was implemented in June 2017 in a new IM inpatient teaching unit at MMC, a 637-bed tertiary care, independent academic medical center in Portland, Maine. ACGME grant funding was used to provide infrastructure for development and analysis of the intervention, but the iPACE unit received no additional operational support. During the period of evaluation, the unit consisted of 11 telemetry-capable general medical beds. Patients are assigned to the iPACE unit based on bed availability with no specific patient inclusion or exclusion criteria. All iPACE team patients are co-located on the unit. Attending physicians may have additional patients off unit.

Structured bedside rounds are a cornerstone of iPACE and promote IP care and education on the unit. Each appointment aims to include all members of the care team and the patient/family. The rounding schedule is advertised daily to maximize patient and family participation. Patients may decline participation in bedside team rounds.

During rounds, clinical care and order entry occur simultaneously with patient interview and assessment. One common team progress note documents the assessment and plan for the day. IM residents typically initiate the note during round preparation, but any member of the team may scribe to allow learners to vary their role in rounds and ensure timely completion of the note.

The timing and duration of rounds preclude the IM team from attending traditional departmental lectures (ie, morning report). Therefore, educational opportunities designed to expose learners to best practices in IP collaborative care are purposefully integrated into the schedule. These include dedicated time for team-based bedside teaching, IP lunch and learn sessions, and in-depth IP case discussions (daily schedule provided as online supplemental material).

A pragmatic observational strategy was used. The mixed-methods approach included quantitative, qualitative, and clinical data to examine various key domains: Teaming, Care Team Experience, IP Education, and Patient Experience.

Pre- and post-iPACE implementation surveys measured the impact of the iPACE model on team functionality, well-being, and perceived care team experience. These constructs were measured using 3 different tools: the RC tool, the adapted Mini-Z, and a perceived quality of education and care measure (all combined into a single survey). The pre-implementation sample consisted of representative members of the anticipated iPACE team on comparable inpatient medical units. The post-implementation sample consisted of iPACE unit team members, which included a mixture of members reassigned from the pre-implementation sample units and new staff. Purposeful sampling of IP team members was employed to promote representation of all groups. Responses were deidentified, analyzed, and are presented in aggregate.

The RC tool is a self-administered survey consisting of 7 questions: 4 on communication (frequency, timeliness, accuracy, and problem-solving) and 3 on relationships (shared goals, shared knowledge, and mutual respect). Participants were asked to consider performance in these areas for team members within their workgroup (eg, nurses and nurses) and between workgroups (eg, nurses and residents). Cardiologists were included in the pre-survey in anticipation of including cardiology patients in the model at a future date and were purposefully left out of the post-implementation survey. Relational Coordination Analytics scored each question for within group and between group performance and created an overall summary RC score using established and proprietary scoring algorithms. RC scores are reported on a 5-point scale. Supplementary analyses were conducted by the study team.

The impact of iPACE on well-being was assessed using an adaptation of the Mini-Z,³⁰  a validated tool that assesses perceptions of stress/burnout, workplace function/culture, and electronic health record usage. For this project, one item was removed, “The amount of time I spend on the electronic health record (EHR) at home,” as it did not apply to all team members. No other changes were made. The tool was scored as recommended by taking a simple sum of responses with reverse coding applied as needed.

Perceived care team experience was assessed using items adapted from the literature, the ACGME IM milestone competencies and the biannual MMC survey on institutional safety culture.^22,31,32  All questions were pilot tested with non-participant team members and modified if necessary prior to administration. A 5-point Likert scale was used.

Patient experience was assessed using survey items created with input from the MMC Patient and Family Advisory Council, the MMC Patient Experience Department, and MMC Patient Education Services for construct validity and health literacy level. The intent of this tool was to collect feedback on unique elements of the iPACE model in real time, which could result in a negative patient experience, such as size of the team or length of appointments. This information could not be obtained from other patient experience surveys (eg, HCAHPS). The surveys were distributed by the unit care coordinator to a convenience sample of patients and family members and collected anonymously.

Qualitative focus groups were conducted with iPACE team members to explore perceptions of the value of the intervention. An opportunistic recruiting strategy was employed (table 1). Participation was limited by the number of individuals who had been exposed to the model. All groups were moderated by skilled qualitative researchers who were not involved in the iPACE unit. Food was served, but no other incentives were provided for participation. Moderators used scripted open-ended questions to probe key focus areas: IP care and teaming, well-being, IP education, patient-centered care, quality of care, and efficiency of care. Cognitive testing with representatives in each group was done to ensure understandability and meaningfulness of the questions. Interviews were audio recorded and transcribed by an external transcription service. Individual participants' names and personal identifiers were removed to maintain confidentiality.

Descriptive statistics were computed for all study variables. RC scores were computed by Relational Coordination Analytics, according to their proprietary algorithm; the strengths of all relationships within work groups and between work groups were assessed overall and by individual item. Pre- and post-implementation differences in outcome variables were assessed using chi-square tests or Fisher's exact tests, as appropriate, for categorical data and t tests for continuous data, including RC scores. Given observed differences in characteristics between comparison groups (provided as online supplemental material), linear regression models were used to assess outcomes controlling for respondent age and workgroup as a secondary analysis.

Focus group transcripts were manually analyzed using an inductive approach consistent with grounded theory (minimizing preconceptions, allowing themes to emerge). A coding schema was developed and applied to the transcripts based on the interview script by 2 investigators (S.H. and L.W.). The schema was further refined by consensus as additional themes were identified then reapplied to all 4 transcripts. In cases of possible discrepancy, the data were reviewed jointly, and conflicts were resolved through discussion.

A determination of non-research was obtained from the MMC Institutional Review Board for this QI project.

There were significant differences in the populations completing the pre-iPACE and iPACE implementation surveys. The post-implementation sample had a lower response rate (pre-iPACE 74% (113); iPACE 54% (32); P = .008). This group also had lower survey completion rates (pre-iPACE 59% (89); iPACE 46% (27); P = .038). The pre-iPACE respondents were older. There were no significant differences in gender or years in practice. There were significant differences in workgroup composition for the RC analysis (provided as supplemental material). Thirteen percent (15) of the pre-implementation RC workgroups were residents versus 9% (3) in the post-iPACE workgroups.

Overall RC scores were significantly higher in the iPACE group than in the pre-iPACE group both between and within workgroups (table 2). Between workgroup scores on all 7 individual items were significantly higher in the iPACE group than the pre-iPACE group. Highly significant differences in the same direction were also seen in the within workgroup questions measuring accurate communication, shared goals, and mutual respect. Adjusting for respondent age and workgroup did not change results.

The overall mean adapted Mini-Z score (table 3) did not differ between the groups (mean [SD] pre-iPACE 24.8 [4.5]; iPACE 23.6 [4.2]; P = .22).

iPACE respondents were much more likely to report that their experience had been professionally rewarding (mean [SD] iPACE 4.4 [0.6] vs pre-iPACE 3.5 [1.0], P < .0001). There was little to no difference between the 2 groups for the remainder of measures with the exception that iPACE respondents were less likely to report improved medical documentation over the past 4 weeks than pre-iPACE respondents (mean [SD] pre-iPACE 3.0 [0.9] vs iPACE 3.6 [1.0], P = .010). Regression models controlling for age and workgroup did not change results.

Patient feedback on the iPACE model was very positive (table 4). Patients felt that they knew when to expect the team and felt like active and involved members of the care team. Potentially negative aspects of the iPACE model as implemented on the pilot unit, such as team size and appointment length and frequency, were well-tolerated.

Overall, it was thought that the iPACE model was successful in teaching best practices in IP care (table 5) and allowed for more direct observation of learners. However, there were perceived disadvantages, including concerns that the model may hinder physician role development through reduced clinical autonomy and limiting access to traditional learning modalities and venues. Learning and presenting in front of the team and patients also had the potential to cause insecurity and impact perceptions of role. Success of the model may also be overly dependent on individual team members (especially the attending physician). As a result, some residents questioned the value and practicality of the iPACE rotation. The shared team progress note was also not well received due to perceptions that it did not meet all team members' documentation needs.

This project reports the outcomes associated with implementation of the iPACE pilot on an inpatient IM unit and its impact on the clinical learning climate, specifically teaming, well-being, perceived educational value, and patient experience.

The evaluation of the iPACE model was challenging in several ways. As a QI project, rapid cycle changes in model implementation were encouraged, which prevented measurement of continuous process data. The iPACE unit was also created de novo for the purposes of this project in a new space with new staff so unit-specific pre-post implementation comparisons could not be made. This required us to use a pragmatic observational approach which impacted our ability to assess the efficacy of the intervention and created significant discrepancies between the pre-implementation sample (taken from 3 units) and the post-implementation sample (limited to 1 unit). While our response rate for the post-implementation survey was consistent with busy clinicians in a naturalistic setting, the absolute number of team members available to survey caused some groups (ie, residents, medical students, and therapists) to be underrepresented. The post-implementation survey also had a long response window of varying lengths dependent on role to attempt to capture as many respondents as possible, which led to differences in exposure in the model. We attempted to minimize this effect by asking them to reflect only on the past 4 weeks. The current plan is to continue to spread the iPACE model to other medicine units and adapt it to different clinical care settings within the institution. This will allow for a more rigorous pre-/post-implementation evaluation of the iPACE model in the future.

iPACE respondents were less likely to report improvements in medical documentation over the previous 4 weeks than pre-iPACE respondents. Feedback suggests that the common team progress note did not fully meet the needs of all team members (ie, did not incorporate nursing-specific treatments/assessments and care plans requiring separate documentation, etc). This is being addressed as part of ongoing QI initiatives.

Attending physicians who participated in the early implementation of the iPACE model were selected for their interest in IP collaborative practice. While the roster was later broadened to include all IM teaching attending physicians, participant feedback suggests that the iPACE model may be dependent on the attending and observed outcomes might have been different had attendings been initially assigned at random. Many of the nurses on the unit were also recent graduates, which may have impacted their acceptance of the model and study outcomes.

Lastly, the data suggest that some trainees had difficulty recognizing the educational value of the iPACE model. While this finding is limited by the small number of residents sampled, it is an important finding given GME priorities regarding teaming and collaborative skills development and supports previous work that residents do not always see IP teamwork or rounds as educationally meaningful.^33,34  Increasing resident independence in future iterations of the model may improve acceptance and engagement in IP training.^35,36  As the iPACE model is disseminated throughout our institution, there are plans to re-evaluate the resident schedule to provide increased autonomy and accommodate participation in traditional didactics. Future projects will also explore resident perceptions of the impact of IP education and collaborative practice on education.

The iPACE pilot as implemented on an inpatient IM teaching unit may have a positive effect on the clinical learning environment as measured by teaming and professional experience. Patients also had a positive experience with the model. While residents appreciated the team collegiality and IP skill development, acceptability may be limited by perceptions of decreased autonomy and concern for impact on physician role development.