ABSTRACT

Background

Team-based primary care has the potential to improve care delivery. However, residency scheduling and precepting models make creating functional ambulatory teams challenging.

Objective

We describe the team-based care transformation at a large academic internal medicine residency practice.

Methods

On July 1, 2016, the program transitioned to a 6+2 schedule and the clinic was divided into teams. Residents were precepted by 2 team preceptors, social work and care coordination needs were met by team-specific staff, and front desk staff were trained on maintaining primary care physician (PCP) and team continuity. Weekly team meetings provided opportunities for proactive patient and panel management, and preclinic huddles incorporated staff into team functions. Pre-transformation (June 2016) and post-transformation (June 2017) surveys were distributed to residents (n = 131), faculty (n = 14), and staff (n = 65) to assess team functioning. Patient-PCP continuity was monitored on a quarterly basis.

Results

Three hundred sixty-two of 420 surveys were returned (86%). The intervention was associated with significant improvements in resident satisfaction (from 3.05 baseline to 4.07 of 5, P < .001) and perceptions of teamwork (4.14 to 4.61 of 6, P < .001), with moderate to large effect sizes. Patient-PCP continuity significantly increased (45% to > 70%). While domain-specific improvements were seen for faculty and staff, no overall changes were noted in their perceptions of teamwork or team-based care.

Conclusions

Team-based care was implemented with significant improvements in continuity and resident satisfaction and perceptions of teamwork; however, the impact on faculty and staff was limited.

What was known and gap

Team-based primary care can improve care delivery, but it is difficult to implement in residency practices.

What is new

A team-based care transformation at a large academic internal medicine residency practice.

Limitations

Single institution study limits generalizability. Many interventions occurred simultaneously; it can be hard to directly determine the effectiveness of any individual element.

Bottom line

Team-based care was implemented in a residency practice with significant improvements in continuity and resident satisfaction and perceptions of teamwork.

Introduction

There has been increasing recognition that creating functional teams in the ambulatory setting is a crucial step in delivering high-quality care.1  Numerous studies have described the implementation of team-based care models, and while findings on health care utilization have been mixed, many have been found to improve provider and patient satisfaction and enhance chronic disease management.25  But residency practices have lagged behind. While Bodenheimer et al included team-based care as 1 of the 10 foundational building blocks of high-performing primary care clinics,6  with additional recommendations for residency practices,79  very few team-based care interventions have discussed implications for residency practices.10,11 

Residency practices face multiple challenges when conducting practice transformation. In recent years, many internal medicine (IM) residency programs have moved to X+Y scheduling models (alternating inpatient rotations with dedicated ambulatory blocks), which have been shown to improve resident satisfaction and decrease inpatient-outpatient conflicts, with a mixed effect on continuity and quality of care.1219  Residents are only present in the practice for a fraction of their training and may have prolonged absences in X+Y models. As such, resident integration into teams can be challenging, although one recent study demonstrated the benefits of multidisciplinary team meetings.20  Faculty preceptors often also conduct clinical work part-time, making functional teams and management continuity difficult. Many residency practices are hospital-based and provide care to medically and psychosocially complex patients, making team-based care even more critical.2124 

In this study, we describe our experience implementing a team-based model at a large, urban primary care residency practice. We hypothesized that a transition to a 6+2 schedule with clear outpatient teaming would result in improvements in teamwork, perceptions of team potency and the quality of care provided, resident satisfaction, and patient–primary care physician (PCP) continuity.

Methods

Setting

This study describes our transformation process at the Internal Medicine Associates (IMA) clinic, an academic practice serving approximately 20 000 patients from the East Harlem neighborhood of Manhattan, a population with a high burden of medical and psychosocial comorbidities and poor health outcomes. All Mount Sinai IM residents have their continuity clinic at IMA. The practice is divided into 3 firms, and prior to transformation, 27 faculty shared precepting duties. Residents were assigned to one of the firms, where they would precept with numerous faculty. Medical assistants (MAs), registered nurses (RNs), licensed practical nurses (LPNs), care coordinators, and front desk registrars were not assigned to a particular faculty member or resident, and 3 social workers were present to address urgent psychosocial issues.

The clinic's main challenges included low resident satisfaction (from the annual program survey), a lack of longitudinal faculty oversight of residents, and poor continuity (consistently less than 50%). There was also a perception of inadequate support for patients when their PCP wasn't available. Available clinic resources and care coordination services were underutilized and were frequently noted to be difficult to access.

Program Description

Planning for the clinical transformation began in 2015, and changes were implemented on July 1, 2016. Stakeholders, including leadership from the hospital, residency program, department of medicine, nursing, and social work, met weekly to plan for the initial year of transition, and met quarterly with executive leadership.

As part of the transformation, the residency program transitioned to a 6+2 schedule. Each firm was restructured into 3 teams (9 total), and each team included 14 residents, 2 faculty preceptors, 1 social worker, and 1 care coordinator. Three to 4 residents were always on an outpatient block to cover for their team. Residents see patients only with their team, to be precepted by 1 of 2 faculty preceptors. Patient issues for residents on other rotations were covered by their team residents on outpatient blocks, and clear protocols for handoffs were established. To ensure continuity and accountability, all patients were empaneled to a resident PCP,25  and call center and front desk staff were trained to maintain PCP and team continuity.

With 9 teams, we needed 18 core preceptors. As we had 27 part-time preceptors, this required faculty to discuss their goals with divisional leadership, and either elect to be a team preceptor or refocus on other academic or clinical activities. Clear expectations were set for team preceptors, including that preceptors must (1) precept at least 2 half-days per week and have consistent availability; (2) be responsible for the quality of care for their residents' patients (including incorporation of these metrics into incentive pay); (3) provide supervision of their team's patients and follow-up; and (4) provide mentorship and feedback to team residents. Biannual meetings with each team's preceptors were held by the ambulatory associate program director and the clinic director to monitor team dynamics. RNs/LPNs/MAs and front desk staff were firm-aligned, but due to staffing constraints were not team-aligned.

Key elements of team activity included weekly team meetings for pre-visit coordination and panel management, and daily preclinic huddles. These elements promote inter-team communication, clear role accountability and understanding, and allow for residents and faculty to engage with social workers, care coordinators, and staff regularly. Each team held a meeting every Monday from 8:30–9:30 am that was attended by both preceptors, all residents on block, and the team's social worker and care coordinator. These meetings previewed the upcoming week with automated reports detailing the scheduled patients' information, including previous no-shows, emergency department and hospital utilization, enrollment in care coordination programs, and care gaps. They allowed for residents and faculty to engage social work to assess and help with patients' needs and create care plans. Residents received reports in this meeting highlighting their panel's care gaps, with those in the top and bottom 10% of performers for each individual care gap noted. These meetings also allowed residents and preceptors to review results and patient messages and discuss anything signed out from the previous residents on block. Additionally, interns met once each outpatient block in a computer lab for 60-minute group panel management sessions with the associate program director for ambulatory care and reviewed real-time panel data; they also utilized telephone calls, letters, and care coordination services to proactively identify and address patient needs. Daily preclinic huddles were led by the MA team in each firm, and a checklist was created and streamlined after multiple piloting rounds. Key components included team introductions, staffing updates, and relevant session-specific clinical process changes. A larger huddle was held immediately after the Monday morning team meeting attended by all firm staff, faculty, and residents to highlight overall clinical process changes and updates.

Feasibility

To implement these changes, social work, care coordination, and clinical staff roles were modified, requiring buy-in from hospital and social work leadership. A full-time equivalent (FTE) of 4.5 faculty preceptors was required, divided among 18 faculty. Required IT needs in the immediate pre-transformation period included the creation of quality metric and continuity reports and taking charge of the re-empanelment process. In the first year of transformation, one full-time analyst performed this work. Our IT needs have now decreased significantly as all reports have become automated. Other small expenses were incurred for quarterly practice meetings and team training exercises.

Data Collection

Surveys of staff, residents, and faculty were done immediately before and 1 year after the clinical transformation. The study population included all staff (21 social workers and care coordinators, 16 RNs/LPNs, 12 MAs, and 16 front desk staff) and 14 team preceptors (2 were excluded due to study involvement; 2 were hired post-transformation). The residency program size was 131 residents throughout the study period. All residents were surveyed in June 2016 (including 44 who would graduate and not take the posttest), and in June 2017 (including 44 interns who did not take the pretest).

The study was approved by the Institutional Review Board of the Icahn School of Medicine at Mount Sinai.

Measures

The main measures used to evaluate the success of transformation efforts were perceptions of team-based care in clinic, PCP continuity, resident satisfaction, and resident evaluations of faculty.

Perceptions of Team-Based Care

The Millward Revised Team Survey26,27  was used to assess team function, as its domains reflected our clinical team roles and its validation in an interdisciplinary cohort. The 6 survey domains are: Team Identification, Team Potency, Shared Mental Models (degree to which the team understands and uses members' skills), Meta-Cognition (awareness of others' thought processes), Communication, and Perspective and Valuing Others. We also surveyed residents' perceptions of support in clinic from different team members (IMA Resident-Specific Survey). All questions were on a 6-point Likert scale with all values labeled from 1 (strongly disagree) to 6 (strongly agree).

Continuity

We monitored continuity using the Continuity for Physician (PHY)28  formula (calculated clinic-wide as total number of appointments where resident seeing the patient was the PCP divided by all resident appointments during preceding 3-month period). Usual Provider Continuity (calculated as the proportion of visits to the patient's PCP divided by all patient visits) was assessed at the end of each academic year.

Resident Satisfaction

The residency's annual program survey was used to determine resident satisfaction with their ambulatory experience. We compared results from the academic year prior to the clinical transformation to the year following.

Resident Evaluations of Faculty

Faculty are evaluated biannually by residents on their team on 20 items on a 1 to 5 scale. We compared results from the academic year prior to the clinical transformation to 1 year after.

Statistical Methods

Average scores on the pre- and post-transformation surveys were calculated for all team roles and reported by survey domain. To compare pre- and posttest averages, t tests were used to determine statistical significance. Effect sizes were calculated using Cohen's d, with effect sizes considered as small (d = 0.2), medium (d = 0.5), or large (d ≥ 0.8).29 

Results

We received 362 surveys out of 420 distributed (overall response rate of 86%). The response rates for each individual group surveyed were 115 of 131 (88%) residents on pretest and 119 of 131 (91%) on posttest, 12 of 14 (86%) faculty preceptors on both pre- and posttests, 17 of 21 (81%) social workers and care coordinators on both pre- and posttests, 14 of 16 (88%) RNs and LPNs on both pre- and posttests, 9 of 12 (75%) MAs on both pre- and posttests, and 12 of 16 (75%) front desk staff on both pre- and posttests.

Measures

Perceptions of Team-Based Care

Results of the Millward Revised Team Survey are presented in Table 1. While domain-specific improvements were seen in all groups except front desk staff, only residents and faculty preceptors had overall improvements in their perceptions of team-based care. There were significant improvements in the preceptor and resident domains of team identification, shared mental models, communication, and perspective and valuing others from the pre- to post-transformation periods. Residents also had statistically significant improvements in team potency. Using Cohen's d, the intervention's overall effect size was moderate for residents and small for faculty, though some domains showed large effect sizes.

Table 1

Millward Revised Team Survey Results

Millward Revised Team Survey Results
Millward Revised Team Survey Results

Results of the resident-specific questions are reported in Table 2. There were significant increases in the perception of support both while in clinic and between clinic blocks. Residents also noted increased support from all other team members with moderate or large effect sizes for all findings. Pre-transformation, 37% (43 of 113) of residents believed their panel in clinic was too large to adequately manage; post-transformation this number was 7% (8 of 119; -30.7%; Cohen's d = 1.29; P < .001).

Table 2

Resident-Specific Items

Resident-Specific Items
Resident-Specific Items

Continuity

Baseline resident continuity (PHY) was 45%, improved to 70% within 1 year, and has remained > 70% since that time. Team continuity (patients seen either by their PCP or a team member) has remained consistently > 90%. Usual Provider Continuity was 48% in the 2015–2016 academic year (prior to transformation) and most recently in the 2018–2019 academic year was 79%.

Resident Satisfaction

Prior to transformation, the overall resident rating for the outpatient experience on their end-of-year survey was 3.05 out of 5. One year after transformation it was 4.07, an increase of 1.02 on a 5-point scale (Cohen's d = 1.35, P < .001), making it the highest-rated core rotation in the residency.

Resident Evaluations of Faculty

Core precepting faculty were highly rated before transformation, with an average of 4.70 out of 5.00 overall. These improved post-transformation with averages of 4.78 and 4.85 in the 2 years post-transformation (+0.15 total; Cohen's d = 0.623; P < .001).

Discussion

Team-based primary care has the potential to improve care delivery; however, implementation of team-based care in residency clinics is challenging with little in the literature about residency practice transformation. This study describes one clinic's transformation, reporting on resident, faculty, and staff perceptions of team function, resident satisfaction, and continuity measures before and after transformation. The success of the transformation required buy-in and support from a broad group of stakeholders including the residency program, clinic faculty and staff, and hospital and social work leadership. Transparency and communication were key steps to involve all stakeholders and ultimately engage our patients in team-based care. Post-transformation, we found improved resident clinical and educational satisfaction, improved resident and faculty perceptions of support and team in the clinic, and a marked increase in continuity.

This description adds to the literature in several ways. While it has been established that team-based care can improve provider satisfaction in faculty practices,25  our innovation joins several other interdisciplinary team-based interventions successfully implemented in residency clinics.10,11  Given that these interventions have been successful across diverse family medicine and internal medicine clinics, this suggests potential generalizability. While the impact of a transition to an X+Y schedule has been previously associated with mixed effects on patient-PCP continuity,1219  we demonstrated a significant increase in continuity when that scheduling transition was accompanied by a team-based care transformation. Whether team-based care can ameliorate any potentially deleterious effects of X+Y models on continuity should be further explored. Interdisciplinary team meetings were an important component of our intervention, and this is the second study showing the impact of these meetings on team functioning,20  perhaps suggesting they should be implemented and explored more widely. Finally, while team-based care has been recommended for residency practices, few transformation descriptions exist in the literature and none in a clinic of this size.

Our data show that we need to identify strategies to further improve staff perspectives on the team-based care model. Staff had no overall changes on the team-based survey, and most domain-specific improvements had small effect sizes. The only changes with large effect sizes were in the “Perspective and Valuing Others” domain. These findings indicate that some of our team functions may be too physician-centric and speak to the need to continually involve all members of the interdisciplinary team in transformation efforts. After reviewing these results, we began to incorporate more staff into existing workflow and practice committees, brought in staff into the resident's quality improvement curriculum, further engaged social work leadership around care gap reports, designated front desk “leads” for each firm to streamline communication, and introduced 360-degree evaluations of residents to involve clinic staff in trainee feedback. Three 10% FTE roles were created for faculty to serve as firm-based leadership for process improvement projects to further involve preceptors in the transformation efforts. In the future, we plan to further expand our teams to pair MA's with individual teams, as well as more regularly assess staff perceptions of team functioning. Anecdotally, these changes have resulted in a significant culture change in the clinic with improved resident self-efficacy and “ownership” of patient outcomes; the impact of team-based care on these measures should be formally evaluated in future assessments.

This study has several limitations. Because it took place at one institution, some findings may relate to the clinic's culture, staffing, or history. The Millward survey, while intuitive, has not been broadly used or used in this particular setting. Finally, since many interventions occurred simultaneously it can be hard to directly determine the effectiveness of any individual element.

Conclusions

Our experience demonstrates the feasibility of practice transformation in a large residency practice and provides practical guidance on how to structure teams and interdisciplinary workflows to create functional and successful teams.

References

References
1
Grumbach
K,
Bodenheimer
T.
Can health care teams improve primary care practice?
JAMA
.
2014
;
291
(
1
):
1246
1251
. doi:.
2
Nelson
K,
Helfrich
C,
Sun
H,
Hebert
PL,
Liu
C-F,
Dolan
E,
et al.
Implementation of the patient-centered medical home in the Veterans Health Administration: associations with patient satisfaction, quality of care, staff burnout, and hospital and emergency department use
.
JAMA Intern Med
.
2014
;
174
(
8
):
1350
1358
. doi:.
3
Brennan-Reese
B,
Brunisholz
KD,
Dredge
C,
Briot
P,
Grazier
K,
Wilcox
A,
et al.
Association of integrated team-based care with health care quality, utilization, and cost
.
JAMA
.
2016
;
316
(
8
):
826
834
. doi:.
4
Mesra-Hebert
A,
Perzynski
A,
Rothberg
M,
Fox
J,
Mercer
MB,
Liu
X,
et al.
Implementing team-based primary care models: a mixed-methods comparative case study in a large, integrated health care system
.
J Gen Intern Med
.
2017
;
32
(
3
):
269
276
. doi:.
5
Meyers
D,
Chien
A,
Nguyen
K,
Li
Z,
Singer
SJ,
Rosenthal
MB.
Association of team-based primary care with health care utilization and costs among chronically ill patients
.
JAMA Intern Med
.
2019
;
179
(
1
):
54
61
. doi:.
6
Bodenheimer
T,
Ghorob
A,
Willard-Grace
R,
Grumbach
K.
The 10 building blocks of high-performing primary care
.
Ann Fam Med
.
2014
;
12
(
2
):
166
171
. doi:.
7
Gupta
R,
Dubé
K,
Bodenheimer
T.
The road to excellence for primary care resident teaching clinics
.
Acad Med
.
2016
;
91
(
4
):
458
461
. doi:.
8
Gupta
R,
Barnes
K,
Bodenheimer
T.
Clinic first: 6 actions to transform ambulatory residency training
.
J Grad Med Educ
.
2016
;
8
(
4
):
500
503
. doi:.
9
Brown
S,
Bodenheimer
T,
Kong
M.
High-performing primary care residency clinics: a collaboration
.
Ann Fam Med
.
2019
;
17
(
5
):
470
471
. doi:.
10
Fernald
D,
Hall
T,
Montgomery
L,
Hartman
C,
Jortberg
B,
Buscaj
E,
et al.
Colorado residency PCMH project: results from a 6-year transformation effort
.
Fam Med
.
2019
;
51
(
7
):
578
586
. doi:.
11
Forman
JH,
Robinson
CH,
Krein
SL.
Striving toward team-based continuity: provision of same-day access and continuity in academic primary care clinics
.
BMC Health Serv Res
.
2019
;
19
(
1
):
145
. doi:.
12
Francis
MD,
Wieland
ML,
Drake
S,
Gwisdalla
KL,
Julian
KA,
Nabors
C,
et al.
Clinic design and continuity in internal medicine resident clinics: findings of the educational innovations project ambulatory collaborative
.
J Grad Med Educ
.
2015
;
7
(
1
):
36
41
. doi:.
13
Heist
K,
Guese
M,
Nikels
M,
Swigris
R,
Chacko
K.
Impact of 4 + 1 block scheduling on patient care continuity in resident clinic
.
J Gen Intern Med
.
2014
;
29
(
8
):
1195
1199
. doi:.
14
Wieland
ML,
Halvorsen
AJ,
Chaudhry
R,
Reed
DA,
McDonald
FS,
Thomas
KG.
An evaluation of internal medicine residency continuity clinic redesign to a 50/50 outpatient-inpatient model
.
J Gen Intern Med
.
2013
;
28
(
8
):
1014
1019
. doi:.
15
Chaudhry
SI,
Balwan
S,
Friedman
KA,
Sunday
S,
Chaudhry
B,
Dimisa
D,
et al.
Moving forward in GME reform: a 4 + 1 model of resident ambulatory training
.
J Gen Intern Med
.
2013
;
28
(
8
):
1100
1104
. doi:.
16
Francis
MD,
Julian
KA,
Wininger
DA,
Drake
S,
Bollman
K,
Nabors
C,
et al.
Continuity clinic model and diabetic outcomes in internal medicine residencies: findings of the educational innovations project ambulatory collaborative
.
J Grad Med Educ
.
2016
;
8
(
1
):
27
32
. doi:.
17
Mariotti
JL,
Shalaby
M,
Fitzgibbons
JP.
The 4:1 schedule: a novel template for internal medicine residencies
.
J Grad Med Educ
.
2010
;
2
(
4
):
541
547
. doi:.
18
Bates
CK,
Yang
J,
Huang
G,
Tess
AV,
Reynolds
E,
Vanka
A,
et al.
Separating residents' inpatient and outpatient responsibilities: improving patient safety, learning environments, and relationships with continuity patients
.
Acad Med
.
2016
;
91
(
1
):
60
64
. doi:.
19
Dearinger
AT,
Wilson
JF,
Griffith
CH,
Scutchfield
FD.
The effect of physician continuity on diabetic outcomes in a resident continuity clinic
.
J Gen Intern Med
.
2008
;
23
(
7
):
937
941
. doi:.
20
Lyons
A,
Walker
RJ,
Prigmore
HL,
Muntz
M.
Multidisciplinary team meetings: a venue to improve team engagement and quality in residency continuity clinic
.
J Gen Intern Med
.
2019
;
34
(
10
):
2001
2002
. .
21
Babbott
SF,
Beasley
BW,
Reddy
S,
Duffy
FD,
Nadkarni
M,
Holmboe
ES.
Ambulatory office organization for internal medicine resident medical education
.
Acad Med
.
2010
;
85
(
12
):
1880
1887
. doi:.
22
Zebrack
JR,
Fletcher
KE,
Beasley
BW,
Whittle
J.
Ambulatory training since duty hour regulations: a survey of program directors
.
Am J Med
.
2010
;
123
(
1
):
89
94
. doi:.
23
Willett
LL,
Estrada
CA,
Adams
M,
Arora
V,
Call
S,
Chacko
K,
et al.
Challenges with continuity clinic and core faculty accreditation requirements
.
Am J Med
.
2013
;
126
(
6
):
550
556
. doi:.
24
Nadkarni
M,
Reddy
S,
Bates
CK,
Fosburgh
B,
Babbott
S,
Holmboe
E.
Ambulatory-based education in internal medicine: current organization and implications for transformation. Results of a national survey of resident continuity clinic directors
.
J Gen Intern Med
.
2011
;
26
(
1
):
16
20
. doi:.
25
Wajnberg
A,
Fishman
M,
Hernandez
C,
Kweon
S,
Coyle
A.
Empanelment in a resident teaching practice: a cornerstone to improving resident outpatient education and patient care
.
J Grad Med Educ
.
2019
;
11
(
2
):
202
206
. doi:.
26
Millward
LJ,
Jefferies
N.
The Team Survey: a tool for health care team development
.
J Adv Nurs.
2001:35(2):276–287.
doi:.
27
Valentine
M,
Nembhard
I,
Edmondson
A.
Measuring teamwork in health care settings: a review of survey instruments
.
Harvard Business School
. ,
2020
.
28
Ejlertsson
G,
Berg
S.
Continuity-of-care measures. An analytic and empirical comparison
.
Med Care
.
1984
;
22
(
3
):
231
239
. doi:.
29
Sullivan
G.
Using effect size: or why the P value is not enough
.
J Grad Med Educ
.
2012
:
4
(
3
):
279
282
. doi:.

Author notes

Funding: The authors report no external funding source for this study.

Competing Interests

Conflict of interest: The authors declare they have no competing interests.