Clinic Design, Key Practice Metrics, and Resident Satisfaction in Internal Medicine Continuity Clinics: Findings of the Educational Innovations Project Ambulatory Collaborative

Multiple stakeholders in internal medicine have called for redesign of residency training,1–3 particularly in education in the ambulatory setting.4 Medical educators have become increasingly concerned about the disconnect between the health care system's growing demand for well-trained outpatient practitioners and the predominantly inpatient focus of residency training.5–8 Training in ambulatory settings is devalued relative to inpatient rotations9 and is often dysfunctional.10,11 In response to these concerns, the Accreditation Council for Graduate Medical Education (ACGME) modified its requirements to emphasize ambulatory care education.12 

Residency programs are working to define the ideal structure for continuity clinics. Residents feel that conflicting responsibilities across care settings are a barrier to an optimal outpatient experience,1,2,4,13,14 yet residents and program directors tend to prefer models that maintain longitudinal clinics during inpatient rotations.14 Satisfaction with preceptors and continuity of care also affect resident satisfaction and valuation in clinic.9,15–18 Finally, adoption of innovative models is complicated by limited availability of outcome measures and lack of multi-institutional comparisons to demonstrate effectiveness.14,19,20 

The Educational Innovations Project (EIP) was developed by the ACGME Internal Medicine Residency Review Committee to allow educational leaders to experiment with new educational approaches.19 Programs across the country, including many EIP programs, are designing and testing innovative ambulatory training models.19–23 In this study, we evaluated resident satisfaction with continuity clinics across varied models from institutions participating in the EIP Ambulatory Collaborative (EPAC).

The sample for this study consisted of 713 internal medicine residents from 12 programs (table 1) participating in the EPAC. Texas Tech University Health Sciences Center at El Paso provided project oversight and performed data collection and data analysis as an unbiased, independent entity.

This was a multi-institutional, cross-sectional study undertaken to assess the effect of clinic structure on resident satisfaction. Secondary analyses were done to assess the effects of postgraduate year (PGY) level, continuity of care, panel size, and workload on resident satisfaction. Data were collected between September 1, 2010, and May 31, 2011. We chose this time frame because residents' patient populations are relatively stable during this part of the academic year. One institution implemented a long block ambulatory experience from the 17th to the 28th month of residency,24 and the time frame for data collection was shifted to adapt to this model.

Prior to the start of data collection, program leadership from each institution completed a curriculum description questionnaire developed by a panel of experts from participating institutions and described in detail the organization and structure of their continuity clinic experience. Responses were reviewed using a grounded theory approach to identify comparable structural themes (table 2). Three patterns for continuity clinics emerged and were confirmed by program leader review: (1) a traditional weekly clinic experience; (2) a combination weekly clinic experience with additional ambulatory block rotations throughout the year; and (3) a block structure with discrete inpatient and ambulatory rotations. Similarities across the combination group included the reduction of clinics during inpatient call rotations and the addition of specific ambulatory rotations with an increased emphasis on ambulatory care. Likewise, in block model programs, core similarities included the elimination of continuity clinics during inpatient call rotations and protected time for continuity clinics during the ambulatory block segments. In general, block model programs assign residents in teams to ensure that 1 resident from the team is always available in clinic. We analyzed the effect of these 3 clinic structures on resident satisfaction.

The total number of patient visits completed by each resident during the study period was divided by the number of clinics attended to estimate workload per clinic. Panel size was defined as the number of patients followed by each resident in their continuity clinic at the end of the data collection period.

Continuity of care was calculated by the continuity for physician (PHY) method.25,26 PHY is the percentage of visits in which residents see their own patients. Thus, PHY reflects continuity from the perspective of the provider and is appropriate for evaluating the educational experience of residents.25 

At the end of the study period, residents completed a 3-part satisfaction survey. The survey was administered in May and June 2011 to most of the residents and after the ninth month to residents in the ambulatory long block format. The first 2 survey sections were based on the continuity clinic version of the Veterans Affairs Learners' Perception Survey (LPS).27 This survey addresses satisfaction with clinical preceptors, learning environment, working environment, clinical environment, and physical environment, and personal experience in the continuity clinic and has substantial evidence for validity.28–31 Responses to this portion of the survey use a 5-point Likert scale from “very satisfied” to “very dissatisfied” and an option of “not applicable.”

Additionally, we included this open-ended question from the continuity clinic version of the LPS: “With what percentage of your patients do you feel you have a continuous healing relationship?”

Due to concerns about conflicts between inpatient and outpatient duties, we developed a set of questions specific to this issue, modeled after the continuity clinic version of the LPS, and added them to the survey. Residents were asked whether their present model of inpatient and ambulatory care training minimized conflict, promoted safe care, provided the best learning experience, and allowed enough time to effectively manage patients in each setting. A 4-point Likert scale was used, with responses ranging from “definitely does” to “definitely does not.”

All participating sites received approval from their local Institutional Review Board.

Independent variables included in the analysis were clinic structure, panel size, workload, PGY level, and PHY. Three aspects of resident satisfaction were the dependent variables, including (1) results of the LPS; (2) resident perception of the continuous healing relationship; and (3) resident responses assessing inpatient and outpatient duties and settings. Mean values were calculated. The responses were analyzed as continuous data32 to capture the strength of satisfaction or dissatisfaction in the responses. We compared the 3 clinic models using analysis of covariance, with a subsequent least significant difference test for means found to be statistically significant. The results are expressed in 95% confidence intervals (CI). The same analysis was conducted in the comparison of PGY levels. Pearson correlation coefficients were calculated to evaluate potential relationships between resident satisfaction and other independent variables. We used SAS version 9.3 software for statistical analysis (SAS Institute Inc, Cary, NC).

The overall response rate for the resident satisfaction survey was 75% (537 of 713 respondents). Clinic models are described in table 2, and response rates by type of clinic model were 67% (88 of 131), 84% (209 of 250), and 72% (240 of 332) for groups 1, 2, and 3, respectively. Response rates by PGY level were 70% for PGY-1, 77% for PGY-2, and 78% for PGY-3 residents.

The results of the resident satisfaction survey by clinic model are shown in table 3. Section 1 shows mean overall scores for each of the 6 domains of the LPS, using a scale of 1 (very satisfied) to 5 (very dissatisfied). Mean overall satisfaction with faculty preceptors was lower for the traditional group than for those in both the combination and block model programs (table 3). Mean resident satisfaction with the overall physical environment of the clinic was better in traditional and block model programs than that in the combination model program. There were no differences in resident satisfaction among the 3 models in overall learning environment, working environment, clinical environment, overall personal experience, or mean score of all domains.

Residents' responses to the question, “With what percentage of your patients do you feel you have a continuous healing relationship?” are shown by clinic structure in table 3, Section 2. The mean for the combination model was statistically higher than those for both the traditional and the block groups. There was no difference in response to this question between the traditional and block groups.

Section 3 in table 3 shows results addressing conflicting duties across care settings. The individual scores for each question in addition to the overall mean score for this section were statistically better in block programs than in the combination program. Block programs also performed better than traditional programs in responses to the questions addressing minimizing conflict between inpatient and outpatient responsibilities, safe care in the inpatient wards, time to manage patients in the inpatient setting, and in the overall mean score. There were no differences in responses in this section between the traditional and combination programs.

Data for resident satisfaction with the 3 models and practice metrics by PGY level are provided as online supplemental material. Pearson correlation coefficients between resident satisfaction and practice metrics with corresponding P values are shown in table 4. Panel size and workload did not have an effect on overall personal experience, resident perception of the continuous healing relationship, or perceived conflict in duties across care settings. The PHY continuity measure was positively associated with resident perception of the continuous healing relationship.

While single institutions have described improvements after implementation of block models,23,33 we present the results of the first multi-institutional assessment of resident satisfaction with continuity clinic that compared a traditional weekly clinic with combination and block models. Separation of inpatient and outpatient clinical duties was a key feature of the block model design. Although there were theoretical concerns that block designs could undermine residents' perceptions of the continuous healing relationship, our results indicate this was not the case. Although resident satisfaction was generally comparable across groups, satisfaction with faculty preceptors, reported to be an important determinant of the value residents place on continuity clinics,9,15 was lowest in the traditional model. One explanation is that resident clinic days are typically based on call schedules in traditional models, which in turn, tend to minimize continuity with faculty preceptors.

Other practice variables can potentially affect the quality of the resident outpatient experience and thus resident satisfaction.9,15,17,18 In the literature, the association between continuity of care and resident satisfaction has been variable. Aspects of continuity and longitudinal care, such as schedules that support continuity and involvement during hospitalization and intervisit care, have been associated with improved satisfaction.9,15 Continuity of care, defined as whether the resident had previously seen the patient, was not independently associated with resident satisfaction.17 In the family medicine literature, trainees valued continuity because it improved efficiency and because residents felt rewarded when recognized by patients.18 In our study, PHY was not associated with overall resident satisfaction. However, PHY was positively associated with the residents' general perceptions of the continuous healing relationship. The importance of this is highlighted by a recent study linking resident satisfaction with continuity of relationships and the likelihood of entering general internal medicine as a career.34 Workload and panel size were not associated with resident satisfaction. Prior studies have shown that the variety and type of patients comprising the panel and the illnesses encountered have a greater effect on resident satisfaction than the absolute number of patients seen.15,17 The ambulatory workloads at the participating institutions were typical and did not cross the theoretical threshold at which differences in resident satisfaction would become apparent. Additionally, PGY level did not affect overall resident satisfaction, which is consistent with findings in the pediatric literature.15 

Limitations of our study include a small sample, consisting of 12 internal medicine residency programs, a subset of the total number of US programs, and heterogeneity among participating institutions. Response rates to the satisfaction survey varied among programs, and we were unable to assess whether nonresponders differed from responders in terms of satisfaction. The study did not use randomization, and the results could be influenced by factors external to the ambulatory training model. The value placed on continuity clinic and ambulatory training may bias resident satisfaction. Finally, factors reported as challenges by participating programs could influence resident satisfaction, such as electronic health record implementation, variation and shortages in clinic staffing, and turnover in ambulatory faculty.

Recent policy changes have increased the amount of time residents spends in continuity clinics. However, residents will choose a career in ambulatory primary care only if their experiences are satisfying and if they have positive faculty role models. Based on our findings, factors to consider when planning residents' continuity experience include the structure of the ambulatory clinic model, with each model having some advantages and posing some challenges, and each model affecting patients' as well as the residents' experience. Future research should address patient satisfaction with the care delivered in each of the 3 continuity models.