A Review of the Medical Education Literature for Graduate Medical Education Teachers

The rapidly evolving medical education literature has the potential to impact the practice of clinical teachers in graduate medical education (GME). New developments in resident assessment, changing clinical learning environments, worthy innovations in teaching, and a greater understanding of residents' experiences are all examples of active recent scholarship, which should be brought to the attention of busy clinical teachers to help them update and improve their practice in GME. Our summary of the literature provides general internist teachers with an update of the medical education literature published in 2011 that is relevant to GME.

The authors, who are general internist educators, conducted a table of contents search of 15 principal journals, and a PubMed search of other clinical journals for education articles published in 2011. The principal journals were major medical education journals and premier clinical journals read by general internists. The journals and the PubMed search results, obtained by using search terms related to medical education, are listed in the References section of this paper. After identifying original research in GME, we reviewed the abstracts for rigorous methods and clearly reported educational outcomes and interest to general internal medicine (IM) educators. Working in pairs to maximize the probability of capturing articles of high relevance for clinical teachers, we identified candidate articles for discussion by the entire group. We determined the final list of articles, intended for a 90-minute oral presentation, through consensus of our expert group, with a preference for studies that brought new insights into practical educational issues for the general IM audience. We used a similar consensus process to group the articles into themes. Themes emerged from consensus of how sample articles illustrated broader, relevant topics in the GME literature.

Our search of the tables of contents of medical education journals (table 1) yielded 33 candidate articles. From this group, we selected a final list of 12 articles.1–12 A search of the National Library of Medicine's PubMed database yielded 97 articles, none of which were selected for this review from our perceptions of their relevance and quality after reading the title and abstract. The 12 chosen articles were grouped into 4 themes, shown in table 2, and are presented below.

The authors assessed the reliability of group versus individual faculty assessments of resident competency in a continuity clinic setting.1 The background to the study was the knowledge that resident assessments often are limited by a “halo effect” (in which positive ratings in one domain are applied or generalized to other domains) and by inconsistency across faculty raters. Clinical assignments and resident assessments at this study institution were organized into 6 firms with 24 residents and 7 to 11 preceptors per firm. Faculty had fixed weekly precepting assignments, while residents had rotating clinic days. The intervention was a quarterly 90-minute group evaluation session led by each firm chief; this was held after faculty had completed individual evaluations. Six to 8 residents, grouped by year of training, were discussed during each session. The firm chief completed the evaluations after the group discussion.

Average assessment scores for each resident were statistically higher for group than individual scores (3.92 ± 0.51 versus 3.83 ± 0.38, P  =  .0001) in areas of illness management and provision of follow-up and in residents' commitment to education, which was the sole measure evaluated for the domain of professionalism. Overall interrater reliability was higher for combined group and individual assessments compared to individual assessments alone (intraclass correlation coefficient, 95% confidence interval [CI]  =  0.828, 0.785–0.866 versus 0.749, 0.686–0.804). Interitem correlation was lower for group assessments (0.49) than individual assessments (0.68), signifying a reduction in the halo effect. Faculty reported that group assessment improved their understanding both of resident strengths and weaknesses and of the overall resident assessment process.

The study has several limitations, including uncertainty whether higher scores for residents in evaluations done by the group reflect these residents' actual level of competency. In addition, evaluations in this system focused largely on the Accreditation Council for Graduate Medical Education competency of patient care, and the applicability of this approach for evaluating other competencies is not known.

This study is relevant to educators because the group evaluation system described is generalizable to other residency programs and appears to improve interrater reliability and reduce the halo effect. It also appears to enhance evaluators' understanding of individual resident performance.

The researchers sought to determine whether scores on objective structured clinical examinations (OSCEs) were affected by prior relationships between faculty evaluators and trainees.2 Faculty evaluated 177 residents on knowledge and physical examination skills in an annual formative OSCE containing 13 clinical scenarios and 20 physical examination stations. Following the OSCE, faculty completed a brief questionnaire noting prior supervisory contact and their previous impression of trainees' performance on a 5-point rating scale. A rating ≤3 (15%) was considered to be negative familiarity, and a rating of 4 or 5 was considered positive familiarity. To correct for examiner stringency, each examiner's mean score was subtracted from the ratings, generating a discrepancy score.

Positive familiarity was associated with a statistically significantly higher rating (0.37 points higher; SE  =  0.11, P  =  .0017) and negative familiarity was associated with a nonsignificantly lower rating (0.20 points lower; SE  =  0.15, P  =  .1981). The same association was seen both for subjectively rated OSCE stations and for physical examination stations rated by checklist.

Limitations of this study include the observation that familiarity assessments, obtained retrospectively, may have been biased by OSCE performance, and results may differ for higher stakes examinations. The important takeaway for educators from this study is that familiarity with trainees may introduce bias in examiner ratings, which does not improve when scoring uses a checklist. OSCE evaluations by faculty unfamiliar with trainees may improve score reliability.

This study from New Zealand describes the impact of a new grade of “conditional pass” to replace “borderline” in the evaluation structure for medical students.3 Students receiving conditional pass were given information about specific steps for converting this standing to “pass after conditions met.” Course directors met regularly with the dean and associate dean for student affairs and fed forward information about student progress. The study compared conditional pass rates with the prior borderline status and used a high-stakes final (year 5) examination as a further comparator between the groups.

There were 3531 evaluations on 701 students in 3 years. Conditional passes increased from 1.4% in year 1 to 1.7% in year 2 and 4.1% in year 3. A total of 91 students (20% of 454 students in the cohort) had conditional pass in 2 years as compared to 4 (1.1% of the 379 students in the cohort) who were judged to be borderline in the preceding 2 years (P < .001). Of the students with a conditional pass, 20 students (4.4% of the 454 students in the cohort) ultimately failed owing to not making the improvements to convert their conditional pass, compared to 4 students (1% of 400 students in the cohort) who failed under the former system (P < .01). There was no difference in performance on the high-stakes final examination when comparing the old and new systems. For the 30 students with more than 1 conditional pass, the odds ratio of failure was 95% CI (7.7–46.1). The conditional pass system particularly tended to identify issues with professionalism and problems in interacting with patients.

A limitation of this study is that the historical cohorts may not have been comparable. Educational implications of this study include the transferability of the conditional pass to residency education settings and the utility of this approach in identifying concerns about professionalism and allowing them to be addressed.

This single institution study used Lean Six Sigma methodology to eliminate waste and variation in resident rounding and improve the efficiency of the rounding process on the pediatric hospitalist service.4 Six trained observers watched pediatrics and family medicine resident rounding behavior on a team, and other volunteers documented time spent in 15 daily activities. A value stream map of the rounding process was developed, defining “value” from the perspective of the patient, and a waste analysis was used to generate potential solutions.

At baseline, formal educational activities occupied the most time, and only 17% of resident time was spent in direct patient care. Most waste related to “inventory” (eg, formal lectures during rounds, waiting until rounding to discuss patients) and overprocessing (multiple notes written for same patient). Following implementation of new lean approaches that included discussion of patients at the bedside, elimination of prerounds, development of a single collaborative note, and completion of all orders before moving to the next patient, time spent on activities that did not directly contribute to patient care decreased 64% (18 minutes versus 6.5 minutes, P  =  .005) and mean time per patient decreased 50% (28 minutes versus 14 minutes, P  =  .014). From the patients' perspective, resident time spent in activities directly affecting their care increased to 55%. In the postintervention surveys, residents' opinions regarding family-centered care, quality of care, and education and efficiency improved, although parents' opinions did not change.

Limitations of this study include its single-site nature and small number of observations, limiting generalizability, as well as the lack of objective measures of educational outcomes. The relevance of this study for GME educators is that this application of the lean approach is easily reproducible and may be used by other institutions to enhance educational efficiency in the inpatient setting.

This study assessed the effects of a restructured teaching service on resident, student, and patient outcomes. The researchers converted 1-resident/1-intern teams to 1-resident/2-intern teams and reduced intern patient caps from 11 to 7.5 Medical students were assigned to each team the same way before and after the resident changes. In addition, day float and transition teams were eliminated and resident teams were linked exclusively with an attending physician, rather than with 2 or 3. Assessments included resident and student rotation evaluations, patient logs, and hospital-collected patient outcome data, comparing data before versus after changes.

After the intervention interns covered fewer patients per day (6.3 compared to 9.9); however, the total number of patients covered by residents increased (2501 to 2916). Residents and interns reported greater enjoyment of the rotation and more time in direct patient care. Time with medical students, however, decreased. Students evaluated a greater percentage of patients on their teams (33% to 46%, P < .001). After adjusting for multiple covariates, the median length of stay of patients decreased from 5 to 4 days (P < .001) and fewer patients required intensive care unit (ICU) care (11.2% to 7.9%, P < .001).

Limitations of this study included unidentified confounding differences between cohorts that may have contributed to observed differences. In addition, the study did not assess resident learning. This study is relevant to GME educators as an example of how a teaching service can be redesigned and studied to demonstrate improvements that are beneficial to residents, students, and patients.

Research on the potential unintended effects of the duty hour limits used data from a voice dictation system to examine residents' discharge summary dictation to determine whether it was a source of unrecognized work hour violations.6 The dictation system was queried by using unique resident identifiers, and data were matched with resident call schedules and reported days off to identify definite or potential duty hour violations. This objective measurement was compared to resident self-report of duty hour violations.

Thirty-nine of 42 residents (93%) had complete schedule, survey, and transcription data available and were included. From objective dictation timing data, 32 of 39 residents (82%) incurred definite work hour violations and 36 of 39 (92%) had either potential or definite work hour violations. Only 2 of 39 (5%) had self-reported work hour violations using the institution's web-based system. On average, residents completed 30 dictations in 1 month (range, 16–45 dictations), spending a total of 6.5 hours (range, 2.5–13 hours). Fourteen percent of total dictating time (54 of 389 minutes) was in definite violation of duty hours, and an additional 15% (60 of 389 minutes) was defined as potential violation.

Limitations of this study include its small sample and having been done at a single site, which reduced generalizability. Its relevance to educators is that it examines the effect of indirect patient care activities on duty hour compliance, raising the question of whether there may be important educational or patient safety outcomes to consider in factoring these activities into work hour calculations. While this question cannot be answered by this study, educators should be prompted to examine how indirect patient care is accounted for in duty hours at their institutions.

A cross-sectional study of medical students, IM residents, and faculty at 3 medical schools sought to determine how physical examination findings influence probability assessment and diagnostic decision making.7 Participants were given history and preexamination probabilities for 4 conditions (ascites, congestive heart failure, streptococcal pharyngitis, and acute anterior cruciate ligament tear) and 2 separate examination scenarios (one with positive and the other with negative findings). They were then asked to estimate postexamination probabilities and make a diagnostic choice (report condition present, order more tests, or report condition absent).

Six hundred eighty-four of 906 invited individuals (75%) participated, including similar proportions of students, residents, and faculty. For positive findings in 2 of 4 of the scenarios, participants gave postexamination probabilities that were significantly less (P < .001) than those derived from the literature (based on published likelihood ratio data for corresponding physical examination findings). For negative findings in 3 of 4 scenarios, participants gave postexamination probabilities that were significantly greater (P < .001) than those derived from literature. In the 4 cases with positive findings, 17% to 38% of participants ordered more diagnostic tests when the condition was at greater than 85% probability. Results demonstrated that students, trainees, and faculty consistently underestimate the impact of examination findings on probability of a condition, especially negative findings.

Limitations of the study include the fact that just 4 conditions were assessed and that the use of conditional probability may be artificial, in that most physicians who practice do not explicitly calculate probabilities. Although the authors gave explicit instructions that testing was to be ordered solely to confirm or refute the presence of the condition and not to determine etiology, treatment, or next step, these distinctions may have been difficult for participants to make in practice. The relevance of this study for GME educators is that incorporating physical examination findings with specific test characteristics into the teaching of physical examination skills may improve learners' efficiency in ordering diagnostic studies.

This study examined the impact of a clinical librarian integrated into a general medicine inpatient team on residents' and medical students' awareness, confidence levels, and self-reported use of evidence for patient care decisions.8 The librarian spent an average of 10 to 12 hours per week with the team, accompanying them on rounds and morning report, teaching short sessions, performing searches, and volunteering information to residents and students about issues encountered during rounds and conferences. The control team was similar to the intervention group but did not have a librarian.

A total of 34 learners on the intervention team and 16 learners on the control team completed an anonymous pre-post survey (overall response rate 50 of 84, 59%). For the intervention team, 79% of respondents asked for search strategy advice, 68% asked the librarian to complete a search, 74% learned from the librarian as part of a group discussion, 88% reported that literature they obtained on their own using skills learned from the librarian contributed to change in a treatment plan, and 79% reported that literature obtained by the librarian contributed to change in a treatment plan. The addition of the clinical librarian to the care team had positive effects on self-reported learner attitudes, information retrieval strategies, and clinical decision making.

Limitations of this study are its single-site nature and lack of reporting the costs of the intervention. The implication of the study is that clinician educators at institutions with librarians who are accessible may consider including them on the clinical team.

The authors surveyed interns at 5 institutions at the beginning and end of internship year to explore the association of burnout with resident characteristics and aspects of the learning environment.9 Burnout was assessed by using the Maslach Burnout Inventory, sleep deprivation was estimated by using the Epworth Sleepiness Scale, and a 10-item personality inventory was included to correlate personality type with risk of burnout.

One hundred eighty-five of 263 eligible residents (70%) completed both surveys; categorical residents were more likely to do so. A total of 86 residents reported developing burnout for an incidence of 75% (liberal definition of burnout) or of 50% (strict definition). Those developing burnout were more likely to report a disorganized personality style (P  =  .019). There was no significant association between the incidence of burnout and duty hours, current rotation, demographics, support network, loan debt, monthly feedback, career plan, or psychiatric history.

Limitations of this study include the use of self-reported information for items, such as personality type, or history of depression/anxiety. Participating programs also were highly competitive academic programs, and the prevalence of burnout reported may not be representative of IM residents in general. The educational implication of this study is that the incidence of new burnout at 50% to 75% suggests that burnout may be the norm during the first year of training. The association of disorganized personality style and incident cases of burnout may allow program directors to identify residents at risk and intervene with early support, if personality style can be measured early in internship.

A group of researchers used data from surveys conducted in 2007 and 2009 across 7 US medical schools to validate a 7-item screening instrument, the Medical Student Well-Being Index (MSWBI), to identify students with severe distress.10 Response rates were 52% (2248 of 4287 students) in 2007 and 61% (2682 of 4400 students) in 2009. Overall, 50% of students had burnout, and 47% screened positive for depression. Forty-one percent had low mental quality of life, 11% had suicidal ideation in the previous 12 months, and 11% had serious thoughts of dropping out of medical school. At MSWBI scores ≥3 for men or ≥4 for women, quality-of-life scores met the diagnostic threshold for “low mental quality of life,” which was one of the study's specified outcomes.

A study limitation is that the choice of 3 specific outcomes from their survey to indicate medical student distress may not identify all distressed students. The practical relevance of this study for educators is that a 7-item screening instrument can be used to identify learners at risk of severe distress. Although developed for medical students, this instrument is applicable to the GME setting, whether by programs or by residents themselves in an effort to identify residents at risk.

A cross-sectional survey of fourth-year medical students in 11 US schools assessed factors that influenced students initially interested in IM to choose other specialties.11 Students were described as “choosers” if they preferred IM, “switchers” if they had once preferred IM but changed their views, and “never considered” if they had never considered IM. Of the 1177 of 1439 students responding (82%), 23% were choosers, 34% were switchers, and 43% were “never considered.” A higher percentage of choosers came from private schools (P < .005). Approximately equal proportions of switchers and “never considered” went into another primary care specialty (family medicine, obstetrics-gynecology, pediatrics, and psychiatry). Choosers (88%) and switchers (93%) made most of their decisions in or after their IM core clerkship. Choosers were more satisfied with IM core clerkships (P < .001) and subinternships (P < .0001) and more likely to have honors grades (P < .001), but less likely to consider grades as an important factor in the decision (P < .001). Types of patients internists see were an important influence for all groups.

Limitations of this study include its cross-sectional methodology, which cannot assess cause and effect relationships, and in this case may have deterred switchers from choosing IM. The educational relevance of this study is that students' perceptions of IM patients are powerful factors that may incent students toward or away from IM.

This study examined the relationship between reduced quality of life and increased burnout and changes in medical knowledge as measured on the Internal Medicine In-Training Examination (IM–ITE).12 Survey questions assessing burnout, quality of life, and satisfaction with work-life balance were given to 21 208 residents undergoing the IM–ITE. Quality of life was measured by using a single item with 5 levels, ranging from “as bad as it can be” to “as good as it can be.” Burnout was assessed by using 2 items from the Maslach Burnout Inventory.

A total of 16 394 of 21 208 residents (77%) responded to some or all variables on the 2008 survey, with 8396 residents also responding to the 2009 survey, allowing for inclusion in a longitudinal analysis related to learning. There were no demographic differences in responders versus nonresponders. Fifteen percent of respondents reported reduced quality of life, and 33% reported being dissatisfied with their work-life balance. Postgraduate year (PGY)–1 and PGY-2 residents, women, primary care residents, and US medical graduates were more likely dissatisfied with work-life balance. At least 1 symptom of burnout was present in 52% of respondents. Women and residents in primary care programs reported more frequent burnout symptoms, which increased with increased educational debt. Reduced quality of life, lower satisfaction with work-life balance, and burnout were all associated with lower IM–ITE scores in each year, and residents with these risk factors did not increase their IM–ITE scores as much as their colleagues year after year.

Limitations of the study include the small differences in IM–ITE scores and their questionable significance and the use of limited measures for resident well-being. Implications of this study for educators include the association of primary care residencies with burnout, reduced quality of life and dissatisfaction with work-life balance, and the apparent association between resident distress and lower knowledge scores, which may indicate that addressing knowledge gaps is an important component of a comprehensive assistance plan for distressed residents.

Our review of the 2011 medical education literature relevant to GME in IM found 4 major themes, which have potential immediate application to educational practice in GME. Resident assessment, adapting to the changing clinical learning environment, innovations in teaching, and resident distress are all clearly emergent topics of relevance to GME teachers. Our selection process, while highly judgment driven, identified clear messages that can be put to use in enhancing educational practice in GME. While we sought to identify the studies with the highest quality and greatest relevance to educators, limitations of the studies selected include their single-site and small sample nature and the frequent lack of objective measures of outcomes. These limitations are shared with the larger body of medical education literature.

The articles discussed in this update have the potential to inform the medical education practice of general internist educators and may also address the needs of GME teachers in other specialties. Readers are encouraged to consider how the evidence presented here may influence their work at their own institutions.