Background

Internal medicine programs are redesigning ambulatory training to improve the resident experience and answer the challenges of conflicting clinical responsibilities. However, little is known about the effect of clinic redesign on residents' satisfaction.

Objective

We assessed residents' satisfaction with different resident continuity clinic models in programs participating in the Educational Innovations Project Ambulatory Collaborative (EPAC).

Methods

A total of 713 internal medicine residents from 12 institutions in the EPAC participated in this cross-sectional study. Each program completed a detailed curriculum questionnaire and tracked practice metrics for participating residents. Residents completed a 3-part satisfaction survey based on the Veterans Affairs Learners' Perception Survey, with additional questions addressing residents' perceptions of the continuous healing relationship and conflicting duties across care settings.

Results

Three clinic models were identified: traditional weekly experience, combination model with weekly experience plus concentrated ambulatory rotations, and a block model with distinct inpatient and ambulatory blocks. The satisfaction survey showed block models had less conflict between inpatient and outpatient duties than traditional and combination models. Residents' perceptions of the continuous healing relationship was higher in combination models. In secondary analyses, the continuity for physician measure was correlated with residents' perceptions of the continuous healing relationship. Panel size and workload did not have an effect on residents' overall personal experience.

Conclusions

Block models successfully minimize conflict across care settings without sacrificing overall resident satisfaction or resident perception of the continuous healing relationship. However, resident perception of the continuous healing relationship was higher in combination models.

What was known

Residents' choice of a career in primary care is dependent on their experience during training, including satisfaction with the clinic learning and care environment.

What is new

A multisite study of internal medicine programs participating in the Educational Innovations Project Ambulatory Collaborative assessed resident satisfaction with 3 different resident continuity clinic models.

Limitations

Limited sample consisting of 12 residency programs; lack of randomization; satisfaction could be influenced by the factors external to the continuity model.

Bottom line

The block model successfully minimized conflict across care settings, whereas the combination model resulted in enhanced resident perception of a continuous healing relationship.

Editor's Note: The online version of this article contains tables describing resident satisfaction and key practice metrics by postgraduate year level.

Multiple stakeholders in internal medicine have called for redesign of residency training,13 particularly in education in the ambulatory setting.4 Medical educators have become increasingly concerned about the disconnect between the health care system's growing demand for well-trained outpatient practitioners and the predominantly inpatient focus of residency training.58 Training in ambulatory settings is devalued relative to inpatient rotations9 and is often dysfunctional.10,11 In response to these concerns, the Accreditation Council for Graduate Medical Education (ACGME) modified its requirements to emphasize ambulatory care education.12 

Residency programs are working to define the ideal structure for continuity clinics. Residents feel that conflicting responsibilities across care settings are a barrier to an optimal outpatient experience,1,2,4,13,14 yet residents and program directors tend to prefer models that maintain longitudinal clinics during inpatient rotations.14 Satisfaction with preceptors and continuity of care also affect resident satisfaction and valuation in clinic.9,1518 Finally, adoption of innovative models is complicated by limited availability of outcome measures and lack of multi-institutional comparisons to demonstrate effectiveness.14,19,20 

The Educational Innovations Project (EIP) was developed by the ACGME Internal Medicine Residency Review Committee to allow educational leaders to experiment with new educational approaches.19 Programs across the country, including many EIP programs, are designing and testing innovative ambulatory training models.1923 In this study, we evaluated resident satisfaction with continuity clinics across varied models from institutions participating in the EIP Ambulatory Collaborative (EPAC).

Study Population and Setting

The sample for this study consisted of 713 internal medicine residents from 12 programs (table 1) participating in the EPAC. Texas Tech University Health Sciences Center at El Paso provided project oversight and performed data collection and data analysis as an unbiased, independent entity.

Study Design

This was a multi-institutional, cross-sectional study undertaken to assess the effect of clinic structure on resident satisfaction. Secondary analyses were done to assess the effects of postgraduate year (PGY) level, continuity of care, panel size, and workload on resident satisfaction. Data were collected between September 1, 2010, and May 31, 2011. We chose this time frame because residents' patient populations are relatively stable during this part of the academic year. One institution implemented a long block ambulatory experience from the 17th to the 28th month of residency,24 and the time frame for data collection was shifted to adapt to this model.

Curriculum Description Questionnaire

Prior to the start of data collection, program leadership from each institution completed a curriculum description questionnaire developed by a panel of experts from participating institutions and described in detail the organization and structure of their continuity clinic experience. Responses were reviewed using a grounded theory approach to identify comparable structural themes (table 2). Three patterns for continuity clinics emerged and were confirmed by program leader review: (1) a traditional weekly clinic experience; (2) a combination weekly clinic experience with additional ambulatory block rotations throughout the year; and (3) a block structure with discrete inpatient and ambulatory rotations. Similarities across the combination group included the reduction of clinics during inpatient call rotations and the addition of specific ambulatory rotations with an increased emphasis on ambulatory care. Likewise, in block model programs, core similarities included the elimination of continuity clinics during inpatient call rotations and protected time for continuity clinics during the ambulatory block segments. In general, block model programs assign residents in teams to ensure that 1 resident from the team is always available in clinic. We analyzed the effect of these 3 clinic structures on resident satisfaction.

Workload and Continuity Variables

The total number of patient visits completed by each resident during the study period was divided by the number of clinics attended to estimate workload per clinic. Panel size was defined as the number of patients followed by each resident in their continuity clinic at the end of the data collection period.

Continuity of care was calculated by the continuity for physician (PHY) method.25,26 PHY is the percentage of visits in which residents see their own patients. Thus, PHY reflects continuity from the perspective of the provider and is appropriate for evaluating the educational experience of residents.25 

Resident Satisfaction Survey

At the end of the study period, residents completed a 3-part satisfaction survey. The survey was administered in May and June 2011 to most of the residents and after the ninth month to residents in the ambulatory long block format. The first 2 survey sections were based on the continuity clinic version of the Veterans Affairs Learners' Perception Survey (LPS).27 This survey addresses satisfaction with clinical preceptors, learning environment, working environment, clinical environment, and physical environment, and personal experience in the continuity clinic and has substantial evidence for validity.2831 Responses to this portion of the survey use a 5-point Likert scale from “very satisfied” to “very dissatisfied” and an option of “not applicable.”

Additionally, we included this open-ended question from the continuity clinic version of the LPS: “With what percentage of your patients do you feel you have a continuous healing relationship?”

Due to concerns about conflicts between inpatient and outpatient duties, we developed a set of questions specific to this issue, modeled after the continuity clinic version of the LPS, and added them to the survey. Residents were asked whether their present model of inpatient and ambulatory care training minimized conflict, promoted safe care, provided the best learning experience, and allowed enough time to effectively manage patients in each setting. A 4-point Likert scale was used, with responses ranging from “definitely does” to “definitely does not.”

All participating sites received approval from their local Institutional Review Board.

Statistical Analysis

Independent variables included in the analysis were clinic structure, panel size, workload, PGY level, and PHY. Three aspects of resident satisfaction were the dependent variables, including (1) results of the LPS; (2) resident perception of the continuous healing relationship; and (3) resident responses assessing inpatient and outpatient duties and settings. Mean values were calculated. The responses were analyzed as continuous data32 to capture the strength of satisfaction or dissatisfaction in the responses. We compared the 3 clinic models using analysis of covariance, with a subsequent least significant difference test for means found to be statistically significant. The results are expressed in 95% confidence intervals (CI). The same analysis was conducted in the comparison of PGY levels. Pearson correlation coefficients were calculated to evaluate potential relationships between resident satisfaction and other independent variables. We used SAS version 9.3 software for statistical analysis (SAS Institute Inc, Cary, NC).

The overall response rate for the resident satisfaction survey was 75% (537 of 713 respondents). Clinic models are described in table 2, and response rates by type of clinic model were 67% (88 of 131), 84% (209 of 250), and 72% (240 of 332) for groups 1, 2, and 3, respectively. Response rates by PGY level were 70% for PGY-1, 77% for PGY-2, and 78% for PGY-3 residents.

The results of the resident satisfaction survey by clinic model are shown in table 3. Section 1 shows mean overall scores for each of the 6 domains of the LPS, using a scale of 1 (very satisfied) to 5 (very dissatisfied). Mean overall satisfaction with faculty preceptors was lower for the traditional group than for those in both the combination and block model programs (table 3). Mean resident satisfaction with the overall physical environment of the clinic was better in traditional and block model programs than that in the combination model program. There were no differences in resident satisfaction among the 3 models in overall learning environment, working environment, clinical environment, overall personal experience, or mean score of all domains.

Residents' responses to the question, “With what percentage of your patients do you feel you have a continuous healing relationship?” are shown by clinic structure in table 3, Section 2. The mean for the combination model was statistically higher than those for both the traditional and the block groups. There was no difference in response to this question between the traditional and block groups.

Section 3 in table 3 shows results addressing conflicting duties across care settings. The individual scores for each question in addition to the overall mean score for this section were statistically better in block programs than in the combination program. Block programs also performed better than traditional programs in responses to the questions addressing minimizing conflict between inpatient and outpatient responsibilities, safe care in the inpatient wards, time to manage patients in the inpatient setting, and in the overall mean score. There were no differences in responses in this section between the traditional and combination programs.

Data for resident satisfaction with the 3 models and practice metrics by PGY level are provided as online supplemental material. Pearson correlation coefficients between resident satisfaction and practice metrics with corresponding P values are shown in table 4. Panel size and workload did not have an effect on overall personal experience, resident perception of the continuous healing relationship, or perceived conflict in duties across care settings. The PHY continuity measure was positively associated with resident perception of the continuous healing relationship.

While single institutions have described improvements after implementation of block models,23,33 we present the results of the first multi-institutional assessment of resident satisfaction with continuity clinic that compared a traditional weekly clinic with combination and block models. Separation of inpatient and outpatient clinical duties was a key feature of the block model design. Although there were theoretical concerns that block designs could undermine residents' perceptions of the continuous healing relationship, our results indicate this was not the case. Although resident satisfaction was generally comparable across groups, satisfaction with faculty preceptors, reported to be an important determinant of the value residents place on continuity clinics,9,15 was lowest in the traditional model. One explanation is that resident clinic days are typically based on call schedules in traditional models, which in turn, tend to minimize continuity with faculty preceptors.

Other practice variables can potentially affect the quality of the resident outpatient experience and thus resident satisfaction.9,15,17,18 In the literature, the association between continuity of care and resident satisfaction has been variable. Aspects of continuity and longitudinal care, such as schedules that support continuity and involvement during hospitalization and intervisit care, have been associated with improved satisfaction.9,15 Continuity of care, defined as whether the resident had previously seen the patient, was not independently associated with resident satisfaction.17 In the family medicine literature, trainees valued continuity because it improved efficiency and because residents felt rewarded when recognized by patients.18 In our study, PHY was not associated with overall resident satisfaction. However, PHY was positively associated with the residents' general perceptions of the continuous healing relationship. The importance of this is highlighted by a recent study linking resident satisfaction with continuity of relationships and the likelihood of entering general internal medicine as a career.34 Workload and panel size were not associated with resident satisfaction. Prior studies have shown that the variety and type of patients comprising the panel and the illnesses encountered have a greater effect on resident satisfaction than the absolute number of patients seen.15,17 The ambulatory workloads at the participating institutions were typical and did not cross the theoretical threshold at which differences in resident satisfaction would become apparent. Additionally, PGY level did not affect overall resident satisfaction, which is consistent with findings in the pediatric literature.15 

Limitations of our study include a small sample, consisting of 12 internal medicine residency programs, a subset of the total number of US programs, and heterogeneity among participating institutions. Response rates to the satisfaction survey varied among programs, and we were unable to assess whether nonresponders differed from responders in terms of satisfaction. The study did not use randomization, and the results could be influenced by factors external to the ambulatory training model. The value placed on continuity clinic and ambulatory training may bias resident satisfaction. Finally, factors reported as challenges by participating programs could influence resident satisfaction, such as electronic health record implementation, variation and shortages in clinic staffing, and turnover in ambulatory faculty.

Recent policy changes have increased the amount of time residents spends in continuity clinics. However, residents will choose a career in ambulatory primary care only if their experiences are satisfying and if they have positive faculty role models. Based on our findings, factors to consider when planning residents' continuity experience include the structure of the ambulatory clinic model, with each model having some advantages and posing some challenges, and each model affecting patients' as well as the residents' experience. Future research should address patient satisfaction with the care delivered in each of the 3 continuity models.

1
Meyers
FJ
,
Weinberger
SE
,
Fitzgibbons
JP
,
Glassroth
J
,
Duffy
FD
,
Clayton
CP
.
Redesigning residency training in internal medicine: the consensus report of the Alliance for Academic Internal Medicine Education Redesign Task Force
.
Acad Med
.
2007
;
82
(
12
):
1211
1219
.
2
Weinberger
SE
,
Smith
LG
,
Collier
VU
.
Redesigning training for internal medicine
.
Ann Intern Med
.
2006
;
144
(
12
):
927
932
.
3
Fitzgibbons
JP
,
Bordley
DR
,
Berkowitz
LR
,
Miller
BW
,
Henderson
MC
.
Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine
.
Ann Intern Med
.
2006
;
144
(
12
):
920
926
.
4
Nadkarni
M
,
Reddy
S
,
Bates
CK
,
Fosburgh
B
,
Babbott
S
,
Holmboe
E
.
Ambulatory-based education in internal medicine: current organization and implications for transformation. Results of a national survey of resident continuity clinic directors
.
J Gen Intern Med
.
2011
;
26
(
1
):
16
20
.
5
Bowen
JL
,
Salerno
SM
,
Chamberlain
JK
,
Eckstrom
E
,
Chen
HL
,
Brandenburg
S
.
Changing habits of practice. Transforming internal medicine residency education in ambulatory settings
.
J Gen Intern Med
.
2005
;
20
(
12
):
1181
1187
.
6
Wiest
FC
,
Ferris
TG
,
Gokhale
M
,
Campbell
EG
,
Weissman
JS
,
Blumenthal
D
.
Preparedness of internal medicine and family practice residents for treating common conditions
.
JAMA
.
2002
;
288
(
20
):
2609
2614
.
7
Arora
V
,
Guardiano
S
,
Donaldson
D
,
Storch
I
,
Hemstreet
P
.
Closing the gap between internal medicine training and practice: recommendations from recent graduates
.
Am J Med
.
2005
;
118
(
6
):
680
685
;
discussion 685–687
.
8
Green
LA
,
Fryer
GE
Jr,
Yawn
BP
,
Lanier
D
,
Dovey
SM
.
The ecology of medical care revisited
.
N Engl J Med
.
2001
;
344
(
26
):
2021
2025
.
9
Sisson
SD
,
Boonyasai
R
,
Baker-Genaw
K
,
Silverstein
J
.
Continuity clinic satisfaction and valuation in residency training
.
J Gen Intern Med
.
2007
;
22
(
12
):
1704
1710
.
10
Keirns
CC
,
Bosk
CL
.
Perspective: the unintended consequences of training residents in dysfunctional outpatient settings
.
Acad Med
.
2008
;
83
(
5
):
498
502
.
11
Babbott
SF
,
Beasley
BW
,
Reddy
S
,
Duffy
FD
,
Nadkarni
M
,
Holmboe
ES
.
Ambulatory office organization for internal medicine resident medical education
.
Acad Med
.
2010
;
85
(
12
):
1880
1887
.
12
ACGME program requirements for resident education in internal medicine
.
Effective July 1, 2009
. .
13
Salerno
SM
,
Faestel
PM
,
Mulligan
T
,
Rosenblum
MJ
.
Disruptions and satisfaction in internal medicine resident continuity clinic differ between inpatient and outpatient rotations
.
Teach Learn Med
.
2007
;
19
(
1
):
30
34
.
14
Thomas
KG
,
West
CP
,
Popkave
C
,
Bellini
LM
,
Weinberger
SE
,
Kolars
JC
,
et al.
Alternative approaches to ambulatory training: internal medicine residents' and program directors' perspectives
.
J Gen Intern Med
.
2009
;
24
(
8
):
904
910
.
15
Serwint
JR
,
Feigelman
S
,
Dumont-Driscoll
M
,
Collins
R
,
Zhan
M
,
Kittredge
D
.
Factors associated with resident satisfaction with their continuity experience
.
Ambul Pediatr
.
2004
;
4
(
1
):
4
10
.
16
Xakellis
GC
,
Gjerde
CL
.
Ambulatory medical education: teachers' activities, teaching cost, and residents' satisfaction
.
Acad Med
.
1995
;
70
(
8
):
702
707
.
17
Barnett
DR
,
Bass
PF
3rd,
Griffith
CH
III,
Caudill
TS
,
Wilson
JF
.
Determinants of resident satisfaction with patients in their continuity clinic
.
J Gen Intern Med
.
2004
;
19
(
5, pt 1
):
456
459
.
18
Delva
D
,
Kerr
J
,
Schultz
K
.
Continuity of care: differing conceptions and values
.
Can Fam Physician
.
2011
;
57
(
8
):
915
921
.
19
Mladenovic
J
,
Bush
R
,
Frohna
J
.
Internal medicine's Educational Innovations Project: improving health care and learning
.
Am J Med
.
2009
;
122
(
4
):
398
404
.
20
Warm
EJ
,
Schauer
DP
,
Diers
T
,
Mathis
BR
,
Neirouz
Y
,
Boex
JR
,
et al.
The ambulatory long-block: an Accreditation Council for Graduate Medical Education (ACGME) educational innovations project (EIP)
.
J Gen Intern Med
.
2008
;
23
(
7
):
921
926
.
21
Mariotti
JL
,
Shalaby
M
,
Fitzgibbons
JP
.
The 4∶1 schedule: a novel template for internal medicine residencies
.
J Grad Med Educ
.
2010
;
2
(
4
):
541
547
.
22
Hoskote
S
,
Mehta
B
,
Fried
ED
.
The six-plus-two ambulatory care model: a necessity in today's internal medicine residency program
.
J Med Educ Perspectives
.
2012
;
1
:
16
19
.
23
Wieland
M
,
Halvorsen
A
,
Chaudhry
R
,
Reed
D
,
McDonald
F
,
Thomas
K
.
An evaluation of internal medicine continuity clinic redesign to a 50/50 outpatient-inpatient model
.
J Gen Intern Med
.
2013
;
28
(
8
):
1014
1019
.
24
Warm
EJ
.
The ambulatory long block: a systems-based practice innovation
.
Virtual Mentor
.
2008
;
10
(
5
):
295
299
.
25
Darden
PM
,
Ector
W
,
Moran
C
,
Quattlebaum
TG
.
Comparison of continuity in a resident versus private practice
.
Pediatrics
.
2001
;
108
(
6
):
1263
1268
.
26
McBurney
PG
,
Moran
CM
,
Ector
WL
,
Quattlebaum
TG
,
Darden
PM
.
Time in continuity clinic as a predictor of continuity of care for pediatric residents
.
Pediatrics
.
2004
;
114
(
4
):
1023
1027
.
27
Keitz
SA
,
Holland
GJ
,
Melander
EH
,
Bosworth
HB
,
Pincus
SH
.
The Veterans Affairs Learners' Perceptions Survey: the foundation for educational quality improvement
.
Acad Med
.
2003
;
78
(
9
):
910
917
.
28
Laponis
R
,
O'Sullivan
PS
,
Hollander
H
,
Cornett
P
,
Julian
K
.
Educating generalists: factors of resident continuity clinic associated with perceived impact on choosing a generalist career
.
J Grad Med Educ
.
2011
;
3
(
4
):
469
474
.
29
Kaminetzky
CP
,
Keitz
SA
,
Kashner
TM
,
Aron
DC
,
Byrne
JM
,
Chang
BK
,
et al.
Training satisfaction for subspecialty fellows in internal medicine: findings from the Veterans Affairs (VA) Learners' Perceptions Survey
.
BMC Med Educ
.
2011
;
11
:
21
.
30
Cannon
GW
,
Keitz
SA
,
Holland
GJ
,
Chang
BK
,
Byrne
JM
,
Tomolo
A
,
et al.
Factors determining medical students' and residents' satisfaction during VA-based training: findings from the VA Learners' Perceptions Survey
.
Acad Med
.
2008
;
83
(
6
):
611
620
.
31
Kashner
TM
,
Henley
SS
,
Golden
RM
,
Byrne
JM
,
Keitz
SA
,
Cannon
GW
,
et al.
Studying the effects of ACGME duty hours limits on resident satisfaction: results from VA learners' perceptions survey
.
Acad Med
.
2010
;
85
(
7
):
1130
1139
.
32
Streiner
DL
,
Norman
GR
.
Health Measurement Scales: A Practical Guide to Their Development and Use. 3rd ed
.
Oxford University Press
:
Oxford
;
2003
.
33
Warm
EJ
.
Interval examination: the ambulatory long block
.
J Gen Intern Med
.
2010
;
25
(
7
):
750
752
.
34
Peccoralo
LA
,
Tackett
S
,
Ward
L
,
Federman
A
,
Helenius
I
,
Christmas
C
,
et al.
Resident satisfaction with continuity clinic and career choice in general internal medicine
.
J Gen Intern Med
.
2013
;
28
(
8
):
1020
1027
.

Author notes

Maureen D. Francis, MD, FACP, is College Master and Associate Professor, Medical Education, Texas Tech University Health Sciences Center at El Paso; Kris Thomas, MD, is Associate Professor of Medicine, Consultant in the Division of Primary Care Internal Medicine, and Associate Program Director, Internal Medicine Residency, Mayo Clinic; Michael Langan, MD, FACP, is Assistant Professor and Associate Program Director, Internal Medicine Residency, The Ohio State University Wexner Medical Center; Amy Smith, MS, is an Instructor, School of Medicine and Public Health, University of Wisconsin-Madison; Sean Drake, MD, FACP, is Program Director, Internal Medicine Residency, Henry Ford Hospital, and Clinical Assistant Professor, Wayne State University; Keri Lyn Gwisdalla, MD, is Associate Program Director, Internal Medicine Residency, Banner Good Samaritan Medical Center/Phoenix VAHCS, and Assistant Professor of Clinical Medicine, University of Arizona College of Medicine-Phoenix; Ronald R. Jones, MD, FACP, is Associate Director, Internal Medicine Center, Summa Health System, and Associate Professor, Internal Medicine, Northeast Ohio Medical University; Katherine A. Julian, MD, is Professor of Clinical Medicine and TrackDirector, Primary Care General Internal Medicine Residency Program, University of California, San Francisco; Christopher Nabors, MD, is Assistant Professor and Associate Program Director, Internal Medicine Residency, New York Medical College at Westchester Medical Center; Anne Pereira, MD, MPH, FACP, is Program Director, Internal Medicine Residency, and Assistant Chief of Medicine for Faculty Development, Hennepin County Medical Center, and Associate Professor ofMedicine, University of Minnesota; Michael Rosenblum, MD, FACP, is Program Director and Director, Primary Care Track, Internal Medicine Residency, Baystate Medical Center, and Assistant Clinical Professor, Tufts University School of Medicine; Andrew Varney, MD, is Professor of Clinical Medicine and Program Director, Internal Medicine Residency, Southern Illinois University School of Medicine; Eric Warm, MD, FACP, is Professor of Medicine and Program Director, Internal Medicine Residency, University of Cincinnati Academic Health Center; and Melchor Ortiz, PhD, is Professor, Biostatistics and Epidemiology, Texas Tech University Health Sciences Center at El Paso.

Funding: The authors report no external funding source for this study.

Conflict of interest: The authors declare they have no competing interests.

The authors would like to thank the following for their contributions to the design of the study and data management at the participating institutions: Jayne Peterson, MD, Banner Good Samaritan Medical Center; Reva Kleppel, MSW, MPH, Baystate Medical Center; Mark Wieland, MD, and Andrew Halvorsen, MS, Mayo Clinic–Rochester; David Wininger, MD, The Ohio State University Wexner Medical Center; Lynn Clough, PhD, and David Sweet, MD, Summa Health System/NEOMED; Rebecca Shunk, MD, Maya Dulay, MD, and Pat O'Sullivan, PhD, University of California, San Francisco; and Bennett Vogelman, MD, and Robert Holland, MD, University of Wisconsin. The authors would also like to thank Mark Francis, MD, MS, Arthritis and Osteoporosis Associates of New Mexico, for editing the manuscript for publication, and the Alliance for Academic Internal Medicine for providing administrative support for the project and meeting space for the EIP Ambulatory Collaborative.

Supplementary data