Objective: This study examined changes in anxiety associated with different modes of student orientation to a manikin-based simulation lab. It was purposed that the addition of an online orientation prior to the actual lab would save time for more learning content during the session.
Methods: Anxiety scores were gathered from groups of interns, using a visual analog scale. Some students experienced a 30-minute in-person orientation while others completed an online module. One-way analysis of variance and the Kruskal-Wallis test were used for analysis.
Results: Mean anxiety scores were not statistically different (χ2 = 2.51, p = .29) between the group that received a 30-minute in-person orientation and the online group. At the end of the entire introductory phase, there was a significant difference between year cohorts (F = 9.61, p < .001), indicating overall higher anxiety for one of the years receiving in-person orientation. However, when looking at the remaining in-person orientation year vs the online module year, there was no significant difference seen (p = .56).
Conclusions: Successful transition, resulting in substantial gain to learning time, was observed by changing an in-person orientation to an online format. Anxiety levels were noted to fluctuate significantly from year to year regardless of orientation method.
Simulation provides superior learning outcomes relative to more typical teaching formats in a health-professions–based curriculum. For example, Singer et al.1 recently demonstrated that simulator-trained first-year residents showed higher clinical competence in critical care compared to even third-year residents who were traditionally trained. Giuliano and McGregor have published results showing learning was retained after 6 months from a single simulation exercise.2,3
In addition, Cohen et al noted a 7:1 rate of return on investment in reducing catheter-related bloodstream infection due to simulation training.4 Despite this value, the perceived cost of manikin-based simulation remains extremely high. Dollar ranges are between $30 and $200,000 for the manikin alone, dependent on features and manufacturer, and most educational facilities will require more than 1. Further, economic consideration must also include indirect costs, facilities, and the personnel needed to run and teach simulation labs. Finally, technology continuously changes, and simulation costs associated with maintenance and upgrades must be considered. Lapkin and Levett-Jones noted that the average cost per student could be as high as almost $300 for a high-fidelity simulation intervention.5
With such high costs, it is imperative that educators are both efficient and effective in the use of their time in these environments. Rhodes and Curran6 identified 3 components to a simulation: the presimulation orientation, simulation, and postsimulation debriefing. The postsimulation debriefing component included 4 evaluation procedures and took a total of 50 minutes to complete. In our setting,2 we identified 5 phases to the lab experience: orientation, enactment, debriefing, repeat enactment, and final debriefing. Typically, the orientation was completed through a face-to-face briefing prior to the enactment phase and had been estimated to take up to 1 half hour of a 2-hour simulation experience. The intention of this student orientation was 2-fold: first, to ensure that all learners understood and respected the rules of the lab and the appropriate use of this expensive technology, and second, to manage student anxiety in this technology-rich environment.7
Managing student anxiety is understood to have tremendous implications for clinical learning.8,9 While heightened emotion can be extremely beneficial to learning in health care,10 excess anxiety is associated with feelings of diminished competence, which may be detrimental.7,11 Engels et al have identified 2 types of anxiety: 1 based in apprehension and the other based in arousal.12 The brain responds to each type differently, and when excessive, the result is either excessive worry (apprehension type) or panic attacks (arousal type).
This study examined changes in anxiety associated with different modes of student orientation to a manikin-based simulation lab. Student anxiety levels are typically high when entering a simulation environment, and lab time is relatively short and expensive. Thus it is critical that students do not enter the environment excessively anxious and that all time spent with the simulation experience be available for learning. It was proposed that the addition of an online orientation prior to the actual lab would save time for more learning content during the session. Concerns were raised, however, that student anxiety could increase with prior knowledge of the technology. The focus of this study was to compare anxiety scores prior to entering the lab for students who had the original in-person orientation verses those who had the onlineversion.
This study was approved by the research ethics board of Canadian Memorial Chiropractic College (approval number 1402X03). Fourth-year students entering their final year of study were grouped into 23 different patient management teams (PMTs), which consist of 5–9 students and 1 licensed clinician. Within the first 3 months of this final year, each PMT was provided with a high-fidelity manikin-based simulation experience during the PMT's normally scheduled 2-hour administrative time. For the purpose of this study, students from the graduating classes of 2010, 2011, and 2012 were included.
The manikin-based simulation experience used for all 3 years was a myocardial infarction that progressed to arrest, occurring during a patient examination for typical back pain prior to any treatment. A life-size manikin (Hal S3000 or Susie S2000; Gaumard, Miami, FL) was used as the patient. The simulation lab coordinator provided the voice to the manikin and operated the computerized physiology to progress the patient to arrest.
Prior to creating the online orientation video, students from the 2010 and 2011 year went through an in-person orientation that took 30–35 minutes to complete. Students from these 2 graduating classes arrived and met outside of the lab on their scheduled day. In recognition of the role that anxiety may play in learning, students were asked to indicate their perceived level of anxiety on a 100-mm visual analog scale (VAS) prior to entering the simulation environment. Once inside the lab, the coordinator took the students through an in-person verbal orientation using a slide presentation to guide the process. The in-person orientation introduced the lab, reviewed rules, and then proceeded to a full demonstration. The demonstration introduced the hardware in the lab and the availability of equipment to convert the environment into a health care office setting. This demonstration also allowed for a complete explanation and demonstration of the manikin functionality as well as an opportunity for the students to handle the manikins and ask questions to clarify any concerns. The orientation ended with a review of the surveys and forms used for data-gathering purposes as well as to highlight the overall purpose of the exercise. A second VAS was taken at that time, and a third VAS was acquired from the student upon role selection.
In 2012 an online orientation video was created to replace the in-person lab orientation. The process to develop this video began with a meeting that included the simulation coordinator, the director of curriculum and faculty development, the coordinator of technology enabled learning, the director of integrative learning, and media services representatives. The team developed a storyboard highlighting the creation of 6 sub-videos and 5 voice-overs that would replace the in-person process completely. The 6 sub-video topics were (1) introduction to lab and team; (2) components of the simulation lab (force-sensing table and manikins); (3) manikin simulation component; (4) physiology of the manikins; (5) rules of the lab; and (6) simulation process.
The online orientation video ended with the students being required to take a quiz upon completion. The module was designed so that students were not allowed to complete the quiz unless they actually played each of the online orientation components completely. If they did not meet this requirement, they were prompted by the system to complete what was missed prior to proceeding to the quiz.
The online orientation module was included as a course requirement and placed on the online learning management system for student access. Students were then sent an e-mail informing them of the video and the mandatory nature of its completion prior to attending their scheduled manikin lab. Students completed the online orientation individually on their own computers. On the day of the students' scheduled labs, the lab coordinator tracked their completed quiz participation for the online orientation video. Students were not allowed to participate in any component of the lab unless this orientation had been completed. There were no noncompliers in the year this online orientation was introduced. Equal knowledge transfer from the in-person orientation to the online orientation was assured via the learning outcomes associated with this quiz.
Before entering the lab, students were required to rate their level of anxiety on a VAS from 0 (no anxiety) to 100 (maximum anxiety). Once this was done, students entered the lab, and the introduction/orientation consisted only of the coordinator asking if the students had any questions about the online video before role selection. Time points for the VAS from students in the online orientation group were prior to entering the lab, upon completion of the introduction, and upon role selection. From this point forward, the lab structure remained the same as for students who did not have the online orientation.
Anxiety scores for the 3 years (2 using in-person orientation and the last using the online orientation) were entered into a spreadsheet and organized by role (doctor, receptionist, etc). Data were entered using a double data entry method to avoid error.13 Outcomes were summarized using descriptive statistics. Data were tested for normality. One-way analysis of variance was applied to determine if students' overall anxiety levels differed by year. Where the normality assumption was not met, the Kruskal-Wallis test was applied.
Secondary analysis was conducted on performance measures using a 1-tailed t test upon determination of a significant difference between groups related to anxiety. A 1-tailed test was chosen since the direction of interest was toward higher anxiety resulting from the online orientation. Performance was measured on those students assuming the role of doctor in each group by assessing videotapes and using a previously documented assessment method.3 The effect size was calculated thereafter in order to inform future investigations.
There were a total of 471 students who participated in simulation experiences during their internship year over the time of this data collection. Baseline anxiety scores (those scores taken prior to entering the simulation lab) were not completed for the first 21 interns who entered the lab in 2010 and were therefore considered missing. Anxiety scores were taken after the in-person orientation for both 2010 and 2011; however, for 2 interns in 2011, the anxiety VAS was missed and therefore these are also taken as missing.
In-person orientations were completed once inside the simulation lab for the 2010 and 2011 years and outside the lab for the 2012 academic year. Therefore, this time point is consistent across all years; however, the 2012 cohort had more knowledge of the simulation process. The descriptive data related to this is provided in Table 1.
The normality assumption for the data in Table 1 was not met; therefore, the Kruskal-Wallis test was used. Kruskal-Wallis indicated no statistically significant difference between years (χ2 = 2.51, p = .29).
All students had been provided all introductory materials prior to role selection in all years. Therefore, data analysis was completed again, considering anxiety scores taken just prior to that role selection phase. Table 2 provides the descriptive data related to anxiety scores taken at this time.
The normality assumption was again not met. The Kruskal-Wallis test for equality of years indicated a statistically significant difference (χ2 = 16.18, p = .00). Since there was no ability with the Kruskal-Wallis test to determine which groups were different from each other, the data were analyzed again using a 1-way analysis of variance, which also showed an overall difference between groups (F = 9.61, p = .00). Post-hoc Scheffe tests looking for differences between groups indicated significant differences between 2010 and 2011 (p = .02) and between 2012 and 2011 (p = .00). There was no significant difference between 2010 and 2012 (p = .56).
Table 3 provides the performance scores for those students who played the role of doctor in each of the 2010 and 2011 academic years, where the orientation method was consistent but the anxiety levels were different.
One tailed t test of the results in Table 3 indicated t = 1.22, p = .11. The effect size for this trend was d = .38.
This study sought to gain tangible benefits through the time-saving alternative of moving an in-person orientation phase in the simulation lab environment to an online venue. The purpose of the student orientation to the simulation environment was to promote students' academic success in managing the clinical scenario they were about to encounter. One half hour of in-person orientation time that had taken away from the 2-hour simulation experience was gained through the use of online technology. Given a potential average cost per student as high as almost $3005 per high-fidelity simulation intervention, this study was successful in achieving its goal to retrieve valuable learning time.
Concern had been raised that by introducing an online orientation video exercise prior to arrival at the simulation event, students would have the time and opportunity to increase their anxiety levels and therefore potentially diminish their performance.7,11 Feedback from students in the in-person group years suggested that anxiety generally extended from 3 main concerns: a lack of familiarity with the lab setting and the functioning of the manikins and their level of fidelity; a lack of information (routinely not provided in our setting) as to which scenarios the students could encounter; and initial apprehension of being allocated the role of the doctor. This study was particularly related to the first concern. Results from this investigation indicated that there was no difference in anxiety levels prior to entering the simulation lab regardless of orientation venue (χ2 = 2.51, p = .29). However, there was a statistically significant difference in anxiety scores between years prior to role allocation (χ2 = 16.18, p = .00). This difference was associated only with the second year that the in-person orientation was provided and was not associated with the new online venue.
This raised the interesting question of whether, given the same orientation venue, a difference in student anxiety would result in a change in performance. Secondary analysis indicated a potential trend (p = .11) of lower performance scores for the student group with higher anxiety prior to role allocation. Given the potential for this trend, the effect size was calculated as d = .38. Therefore, for 80% power using a 2-tailed t test with a p value of .05, a study to accurately assess this trend would require approximately 115 subjects per anxiety group (lower anxiety verses higher anxiety).
Despite the need for further investigation, the difference in anxiety and trend toward a difference in performance in groups unrelated to change of orientation venue suggest that students may fluctuate between stretch and panic14,15 regardless of the simulation framework. As per Gordon et al,10 the emotions related to a clinical encounter may be critical in anchoring situational learning. Every clinician can easily remember his or her most challenging cases. However, managing anxiety is equally critical to clinical performance,8,9 and if left unchecked, anxiety can lead to depression and burn out. Consideration must therefore continue to be given to the emotional safety of students in the simulation lab environment.10
Limitations to this study include the generalization of anxiety into a single dimension using a VAS when it is understood to be a multidimensional attribute.12,15–17 Further investigation considering the various facets of anxiety could be useful in guiding curriculum development through clinical simulation methods. Also, 21 students in the original in-person orientation group did not complete the anxiety VAS prior to entering the lab. In addition, each simulation environment is a unique experience partially governed by the characteristics of the attending faculty. In this investigation, the clinical faculty remained constant. Future investigation modifying faculty characteristics may enrich the data further.
Successful transition was observed from an in-person orientation to an online orientation in a simulation lab. Success in this context was based on the notion that increased anxiety with the potential to hamper performance was not observed, and the online quiz assured that the learning objectives at orientation had been reached. Orientation transition resulted in a substantial gain in learning time. Anxiety levels were, however, noted to fluctuate significantly from year to year even when the orientation method was held constant. Further study is needed to understand these fluctuations and the potential impact this might have on learning.
The authors thank Anthony Michael Giuliano for his assistance with data management.
FUNDING AND CONFLICTS OF INTEREST
This work was funded internally. The authors have no conflicts of interest to declare relevant to this work.
Dominic Giuliano is the director of education, year 3 of undergraduate education at Canadian Memorial Chiropractic College (6100 Leslie Street, Toronto, Ontario, M2H 3J1, Canada; firstname.lastname@example.org). Marion McGregor is acting dean for graduate education and research programs at Canadian Memorial Chiropractic College (6100 Leslie Street, Toronto, Ontario, M2H 3J1, Canada; email@example.com). Loretta Howard is a sessional lecturer at the Ontario Institute for Studies in Education, University of Toronto (252 Bloor Street West, Toronto, Ontario M5S 1V6, Canada; firstname.lastname@example.org). Rebecca Taylor is the coordinator of technology enabled learning in the department of undergraduate education at Canadian Memorial Chiropractic College (6100 Leslie Street, Toronto, Ontario, M2H 3J1, Canada; email@example.com). Rachel Statz is a research assistant at Canadian Memorial Chiropractic College (6100 Leslie Street, Toronto, Ontario, M2H 3J1, Canada; firstname.lastname@example.org). Madolyn Linka is a research assistant at Canadian Memorial Chiropractic College (6100 Leslie Street, Toronto, Ontario, M2H 3J1, Canada; email@example.com). Christina Bagnell is a research assistant at Canadian Memorial Chiropractic College (6100 Leslie Street, Toronto, Ontario, M2H 3J1, Canada; firstname.lastname@example.org). Address correspondence to Dominic Giuliano, 6100 Leslie Street, Toronto, Ontario, M2H 3J1, Canada; email@example.com. This article was received November 12, 2015, revised February 1, 2016, and accepted April 22, 2016.
Concept development: DG, MM, LH, RT. Design: DG, MM, LH, RT. Supervision: DG, MM, RS, CB, ML, RT, CB. Data collection/processing: DG, LH, RS, CB, ML. Analysis/interpretation: MM, LH. Literature search: DG, MM, LH, RS, ML, CB. Writing: DG, MM, LH, RT, RS, CB, ML. Critical review: DG, MM, LH, RT, RS, CB, ML.