Background

The transition from internship to residency training may be a stressful time for interns, particularly if it involves a change among programs or institutions after completing a preliminary year.

Objective

We explored whether an e-learning curriculum would increase interns' preparedness for the transition to the first year of clinical anesthesiology training and reduce stress by improving confidence and perceived competence in performing professional responsibilities.

Intervention

We tested a 10-month e-learning program, Successful Transition to Anesthesia Residency Training (START), as a longitudinal intervention to increase interns' self-perceived preparedness to begin anesthesiology residency training in a prospective, observational study and assessed acceptance and sustainability. After a needs assessment, we administered the START modules to 22 interns, once a month, using an integrated learning management and lecture-capture system. We surveyed interns' self-assessed preparedness to begin anesthesiology residency before and after completing the START modules. Interns from the prior year's class, who did not participate in the online curriculum, served as controls.

Results

After participation in the START intervention, self-assessed preparedness to begin residency improved by 72% (P  =  .02). Interns also felt more connected to, and had improved positive feelings toward, their new residency program and institution.

Conclusion

Participation in our novel 10-month e-learning curriculum and virtual mentorship program improved interns' impression of their residency program and significantly increased interns' subjective assessment of their preparedness to begin anesthesiology residency. This e-learning concept could be more broadly applied and useful to other residency programs.

Editor's Note: The online version of this article contains the Preparedness Survey used in this study, as well as a table describing the START curriculum. In addition, there are 5 video files: video 1, Demonstration of START Moodle LMS [Learning Management System] Website; video 2, Example of Learning Trigger; video 3, Example of Debriefing; video 4, Example of Welcome Bucket and Building Relationships; and video 5: Demonstration of START Lecture Capture System.

The transition from internship to residency training may be a particularly stressful time for trainees.1 Interns in noncategorical residency programs, in which training starts with an internship year that is often completed at a different program, may find the transition particularly challenging. This separation between internship and residency occurs in a variety of specialty programs, including anesthesiology, dermatology, neurological surgery, orthopedic surgery, otolaryngology, radiology, and urology.

Residents describe the stress of this transition stemming from the assumption of professional responsibility beyond their competence or experience.2 In addition, resident confidence levels in performing a range of clinical and practical activities may be inversely proportional to their psychologic distress levels.3 A program designed to increase interns' preparedness to start residency training may improve confidence and perceived competence in performing professional responsibilities.

To assist interns in transitioning to the first year of clinical anesthesiology training, we developed the Successful Transition to Anesthesia Residency Training (START) program. We hypothesized that participation in this 10-month, educational, e-learning, and virtual mentorship program would increase interns' self-assessed preparedness to begin anesthesiology residency. This e-learning concept could be broadly applied to other aforementioned specialties.

Intervention

Needs Assessments

Twenty-two incoming Stanford anesthesiology residents participated in START during their internship year, with the 24 incoming residents from the prior year serving as the historic controls. We developed a general needs assessment survey in collaboration with the residency program director (A.M.) and fielded it to current postgraduate year (PGY)-2, PGY-3, and PGY-4 residents. Challenges and concerns of beginning residency were ascertained, including clinical issues and life outside the hospital. A more targeted needs assessment was then conducted on the control cohort, from which we developed the goals and objectives for the 10 START modules.

START Curriculum and Implementation

The START curriculum encompasses 10 learning modules, delivered monthly, and blends e-learning with virtual mentorship, combining various modes of delivery (text, video, group interaction).4 The START curriculum is provided as online supplemental material. The curriculum addresses the Accreditation Council for Graduate Medical Education's 6 competencies. Each monthly module contains 5 components, including a feedback component for learners to evaluate the module, used for continuous improvement of the program.

The course website was constructed using an online Learning Management System (LMS) (http://moodle.org; provided as online supplemental video 1), which links to all resources, quizzes, and assignments. With personalized accounts to this virtual classroom, interns were able to access all course materials, post messages in forums, and view exam grades at their convenience. The START curriculum used time-shifted and place-shifted learning, allowing learners to participate anytime and anywhere.5,6 

The curriculum includes the following components:

  • Video podcasts that typically last 25 minutes, featuring a learning trigger (a dramatic, clinical vignette, provided as online supplemental video 2) followed by a focused debriefing (provided as online supplemental video 3). The remainder of the podcasts entails a virtual mentoring session designed to facilitate the development of recognition and familiarity between interns and residency department members (faculty members personally welcome students, then offer advice about beginning residency; provided as online supplemental video 4).

  • Course-cast video lectures are more-detailed, formal didactic sessions that were recorded via Lecture Capture technology (Panopto, Inc, Pittsburgh, PA; http://www.panopto.com; provided as online supplemental video 5).

  • Interactive/collaborative activities accompanied most modules to foster relationships between interns and their future department.

  • Premodule and Postmodule Quizzes were available for each module; the interns completed a premodule quiz assessing baseline knowledge before participating in the lectures and interactive components of the program. The postmodule quiz was taken immediately after completing a module, to eliminate confounders, such as learning that may have occurred in interns' other didactic and clinical activities. Grades and answers were shown instantaneously. Full explanations were revealed at the beginning of the next module to enhance learning across modules.

Preparedness Survey

We developed a survey assessing participants' preparedness to perform 14 basic anesthesia skills; this used a 5-point Likert scale (from “not at all prepared” to “extremely prepared”; provided as online supplemental material). We administered the survey at the beginning of interns' exposure to the START curriculum and after they completed the program, before beginning clinical anesthesiology training.

Secondary outcomes surveyed throughout the program included changes in self-assessed confidence and concern about beginning residency; enjoyment of START, perception of its usefulness, and qualitative assessments of the monthly podcasts, didactic lectures and dissemination; and participants' feelings about interactions with classmates, mentors, and faculty.

Needs Assessments

Most of the cohort of residents responding to the general and targeted needs assessment reported feeling inadequately prepared to complete basic anesthesia tasks during their first few weeks of residency. The control group of the prior year's incoming PGY-2 residents agreed that a program to facilitate the transition from internship to residency would be useful. The control group also reported that it would have been useful to receive short, practical lectures on basic principles of anesthesia through an online, e-learning website or to have a current resident actively offer advice and answer questions before they began residency.

We evaluated the internal consistency of the survey instrument by computing Cronbach α coefficient of reliability. The value of 0.926 indicated a high degree of internal consistency. Test-retest reliability of the preparedness survey instrument was conducted on a cohort of 10 PGY-2 residents, completing the identical instrument 1 week apart. All questions had a Pearson R coefficient of correlation greater than 0.7, indicating highly correlated test-retest reliability.

The study was granted an exempt status by the Stanford Institutional Review Board.

Statistical Analysis

To compare changes within a group, a repeated-measures analysis of variance (ANOVA) was computed, with time as the repeated measure, followed by the Student-Newman-Keuls test for post hoc comparisons.7 Statistical analyses was performed using SAS for Windows v 9.1.3 (SAS Institute, Cary, NC). Continuous, normally distributed variables were compared using paired t test and the Student t test of 2 independent means for matched and independent sample comparisons. Discrete outcome measures were analyzed using χ2 or Fisher exact tests.

Survey test-retest reliability and principal component analysis was conducted using SPSS 20.0.0 (IBM Corporation, Armonk, NY). Factor analysis was conducted using the varimax model for component variable rotation. Statistical significance was considered P < .05.

Preparedness to Begin Residency

After participating in START, each intern's self-assessed preparedness score improved from baseline by an average (SD) of 72% (114%). Raw composite scores of the entire cohort increased from 16.1 (9.8) to 21.0 (7.3) (P  =  .02; figure 1). Completion of the internship year alone did not contribute significantly to this improvement. Preparedness scores of START students before beginning internship were not significantly different from preparedness scores of the control group after completing internship (29% [18%] and 24% [8%]; P  =  .23). Five participants reported a slight reduction in preparedness to begin residency.

FIGURE 1

Self-Assessed Preparedness Scores for Each Intern Based on Results of Preparedness Survey Taken Before and After Participation in the Successful Transition to Anesthesia Residency Training (START) Programa

a Inset: The self-assessed, preparedness scores of each START intern were assessed at t  =  −10 months (before beginning START) and t  =  0 months (after START but before residency); composite scores were out of a maximum 56 points possible; the whiskers indicate the smaller of 1.5× the interquartile distance or the distance to the highest or lowest point; and the dots indicate the outliers. Asterisks indicate significance where P < .05.

FIGURE 1

Self-Assessed Preparedness Scores for Each Intern Based on Results of Preparedness Survey Taken Before and After Participation in the Successful Transition to Anesthesia Residency Training (START) Programa

a Inset: The self-assessed, preparedness scores of each START intern were assessed at t  =  −10 months (before beginning START) and t  =  0 months (after START but before residency); composite scores were out of a maximum 56 points possible; the whiskers indicate the smaller of 1.5× the interquartile distance or the distance to the highest or lowest point; and the dots indicate the outliers. Asterisks indicate significance where P < .05.

Close modal

Curriculum Assessment and Participation

All interns participated in all educational activities (N  =  22). In response to monthly assessments, most reported that the amount of material and the time they dedicated to START was “just right.” Participants provided an average of 1.6 spontaneous, positive comments per month, mostly about the podcasts and the program as a whole. Each month, quiz scores improved by an average of 23%, from 7.9 to 9.7 out of 10.0 (P < .001). Quiz answer choices were randomized, and the correct answer was not revealed in the pretest (only in the posttest).

Program Assessment Feedback

The program fostered a sense of connectedness between the interns and the residency program. Despite long duty hours and workload challenges in different internships, schedules, and time zones, 86% of participants (19 of 22) agreed that START made them feel more connected to the residency program (figure 2). In the final program assessment, 100% of participants (22 of 22) agreed that START made them feel that the residency program leadership cared about their education.

FIGURE 2

After Taking the START Program, Interns Were Asked How Well They Agreed With the Statements Labeled on the Y-axis About the Stanford Anesthesiology Residency Program

FIGURE 2

After Taking the START Program, Interns Were Asked How Well They Agreed With the Statements Labeled on the Y-axis About the Stanford Anesthesiology Residency Program

Close modal

Participants provided spontaneous feedback throughout the program. When addressing which aspects of START helped them most to feel more prepared to begin residency, the top reasons were attributed to content of faculty podcasts and lectures as well as the interaction and virtual face time with department faculty (table).

table

Sampling of Spontaneous Feedback From Students Supporting Successful Transition to Anesthesia Residency Training (START) as an Effective E-Learning Intervention

Sampling of Spontaneous Feedback From Students Supporting Successful Transition to Anesthesia Residency Training (START) as an Effective E-Learning Intervention
Sampling of Spontaneous Feedback From Students Supporting Successful Transition to Anesthesia Residency Training (START) as an Effective E-Learning Intervention

The transition from internship to residency requires rapid acquisition of new knowledge, patient care skills, and new responsibilities. “No prior experience” is one of the top reported reasons for residents' distress during the beginning of residency.1 In the absence of any previously reported comparable educational program to our knowledge, START was designed to mitigate this perceived lack of preparation to begin residency by engaging interns in a novel e-learning opportunity. Not only did participation in START increase interns' self-assessed preparedness to begin residency, but their perception of their residency program improved as well.

Increase in self-assessed preparedness can be attributed to many factors, from exposure to curriculum content to faculty involvement and the close attention they received, cultivating students' self-motivation to improve. The 5 participants (23%) reporting a slight reduction in preparedness may have been overwhelmed by the immediacy of beginning residency. They could have initially overestimated their preparedness before gaining an increased awareness of the field of anesthesiology through START, or their response may reflect a greater time-lapse from their fourth-year anesthesiology rotations in medical school.

Participants noted that START motivated them to learn more about anesthesiology during their internship year. This shows that the e-learning curriculum may have the potential to increase interns' knowledge base, along with their enthusiasm for the program and improved quiz scores. The START program may reduce the slope of the learning curve associated with the first months of residency, providing students with additional time and energy to study and further broaden their educational base.

The constant interaction and increased satisfaction with the department creates an invaluable bond between the intern and the training institution. Students may feel less anxious about transitioning from internship to residency. The American Society of Anesthesiology recently established a wellness initiative addressing issues of occupational health and safety associated with the clinical practice of anesthesiology. Any program that can limit stress or anxiety associated with resident life should be of interest to residency program directors.8 

An e-learning intervention like START might benefit any specialty residency program seeking to accomplish the goals of increased preparedness for residency. The feasibility of implementing such a program should not necessarily deter residency program directors. Various LMSs are free and open source, allowing for customization. Faculty involvement may take no more time than another mode of teaching would take yearly, such as time spent to prepare an in-person lecture series or hands-on simulation sessions.

Our study has several limitations. Because the START curriculum was limited to a cohort at a single institution, results may not be generalizable to other programs. Use of historic controls also limits our ability to attribute improvements in residents' self-perceived preparedness to the curriculum.

Although our data suggest that participation in START is well received and produces measurable benefits in residency preparedness, START is not intended to replace an integrated internship or to teach interns practical skills for the operating room. Improvement in clinical proficiency and objective assessment of preparedness to begin residency by faculty were not assessed.

To our knowledge, START represents the first e-learning and virtual mentorship program in any field of medicine designed to prepare interns for residency training. Such an educational e-learning intervention can improve interns' perception of their training institution and subjective preparedness to begin residency. The START curriculum is currently being implemented at several other institutions.

1
Schwartz
AJ
,
Black
ER
,
Goldstein
MG
,
Jozefowicz
RF
,
Emmings
FG
.
Levels and causes of stress among residents
.
J Med Educ
.
1987
;
62
(
9
):
744
753
.
2
Paice
E
,
Rutter
H
,
Wetherell
M
,
Winder
B
,
McManus
IC
.
Stressful incidents, stress and coping strategies in the pre-registration house officer year
.
Med Educ
.
2002
;
36
(
1
):
56
65
.
3
Williams
S
,
Dale
J
,
Glucksman
E
,
Wellesley
A
.
Senior house officers' work related stressors, psychological distress, and confidence in performing clinical tasks in accident and emergency: a questionnaire study
.
BMJ
.
1997
;
314
(
7082
):
713
718
.
4
Graham
CR
.
Blended learning systems: definition, current trends, and future directions
.
In:
Bonk
,
CJ
,
Graham
CR
,
Cross
J
,
eds
.
The Handbook of Blended Learning: Global Perspectives, Local Designs
.
San Francisco, CA
:
Pfeiffer Publishing
;
2006
:
3
21
.
5
Chu
LF
,
Young
CA
,
Ngai
LK
,
Cun
T
,
Pearl
RG
,
Macario
A
.
Learning management systems and lecture capture in the medical academic environment
.
Int Anesthesiol Clin
.
2010
;
48
(
3
):
27
51
doi:10.1097/AIA.0b013e3181e5c1d5
.
6
Chu
LF
,
Chan
BK
.
Evolution of web site design: implications for medical education on the internet
.
Comput Biol Med
.
1998
;
28
(
5
):
459
72
.
7
Zar
JH
.
Biostatistical Analysis. 4th ed
.
Upper Saddle River, NJ
:
Prentice Hall
;
1999
.
8
American Society of Anesthesiologists
.
Wellness
.
ASA Newsl
.
2009
;
73
(
8
):
1
64
.

Author notes

All authors are in the Anesthesia Informatics and Media (AIM) Lab at Stanford University School of Medicine. Larry F. Chu, MD, MS (BCHM), MS (Epidemiology) is Associate Professor; Lynn K. Ngai, BS, and Chelsea A. Young, BS, are Research Assistants; Ronald G. Pearl, MD, PhD, is Professor; Alex Macario, MD, MBA, is Professor; and T. Kyle Harrison, MD, is Clinical Assistant Professor, Anesthesia (Affiliated), and Staff Physician, Palo Alto Veterans Affairs Health Care System.

Funding: This study was supported in part by the National Institute of General Medical Sciences of the National Institutes of Health (Bethesda, MD). Dr Chu's work was supported by a career development award from the National Institute of General Medical Sciences of the National Institutes of Health, 5K23GM071400-05.

Plenary lecture presented at Society for Education in Anesthesia Spring Meeting, Pittsburg, PA, June 5, 2010; poster presented at American Society of Anesthesiologist Annual Meeting, San Diego, CA, October 18, 2010; poster presented at the Third World Congress of Social Media and Web 2.0 in Health, Medicine, and Biomedical Sciences, Maastricht, The Netherlands, November 29, 2010.

We would like to acknowledge Erin Reiland, BA, and Janine Roberts, BS, for their administrative support with this project.

Supplementary data