Simulation is commonly incorporated into medical and health programs as a method of skill practice and evaluation and can be effective at improving athletic training student learning outcomes when purposefully designed.
The purpose of this study was to determine what level of impact participation in supervised practice after debriefing within a simulation-based cardiovascular emergency scenario using the Laerdal SimMan in a university simulation center in the United States had on athletic training students' clinical performance.
Quantitative quasi-experimental cohort design with repeated measures study.
Convenience sample of undergraduate athletic training students (n = 46) enrolled in a professional program at a university in the Midwest.
Participation in supervised practice of cardiopulmonary resuscitation skills after debriefing in a simulation.
Clinical competency with associated cardiopulmonary resuscitation skills using the Laerdal Learning Application software program that interfaces with the simulation hardware.
There was a statistically significant interaction between groups (F1,10 = 18.70, P < .05, η = 652) indicating participants in the supervised practice after debriefing group were significantly higher (mean = 0.72, SD = 0.05) than those that did not have supervised practice after the debriefing (mean = 0.17, SD = 0.05).
The design and development of a simulation experience is optimized when there is deliberate consideration of what components and exposure to these learning components will lead to certain outcomes. Even though supervised practice after debriefing has been identified as optional for skill-based simulations, the current study demonstrates that the supervised practice of clinical skills component is vital within emergency cardiovascular simulation encounters for participants to increase clinical competency.
Without purposeful simulation design, simulation experiences may be suboptimal or ineffective for those participating.
Though standards of simulation provide specific guidelines to simulation design, facilitation, and assessment, suggestions have been made to include supervised practice in skill-based activities, but this is not listed as a requirement for successful experiences to improve clinical competence.
Supervised practice of clinical skills after debriefing within emergency cardiovascular simulation encounters is vital for participants to increase clinical competency.
Simulation-based training provides an ideal adjunct to clinical experiences within athletic training programs because it provides students with a safe and realistic learning environment for the practice of low incident encounters such as cardiovascular emergencies.1–3 Lasater2 and Wayne et al4 state simulation participants can improve clinical performance through repeated practice of an appropriate response to a specific clinical situation. Therefore, simulation is commonly incorporated into medical and health education programs to supplement clinical teaching and improve clinical learning in low incident events.1–5 Though simulation-based training has previously demonstrated positive outcomes, it is important to consider that the design of the experience must be purposeful and consistent with best practices to maximize expected outcomes.6,7 To date, research on athletic training student evaluation during simulation training has focused primarily on students' perceptions of their own performance including a rating of their self-efficacy or a knowledge assessment after a simulated cardiopulmonary resuscitation (CPR) experience.1,8
In a recent report of the standards of best practice in simulation design, the International Nursing Association for Clinical Simulation and Learning (INACSL) developed a list of 11 criteria that should be considered to facilitate the effectiveness of simulation-based experiences.9 The argument is made that, without purposeful simulation design, simulation experiences may be suboptimal or ineffective for those participating.3,7,9 Therefore, as facilitators and faculty, we cannot assume that, just because we are providing learners with a simulation-based experience, it will be effective unless we consider the standards of practice.
One criterion outlined by INACSL is that simulation-based experiences should be followed by a debriefing or supervised practice or feedback session or both.9–12 A guided debriefing session immediately after the simulated scenario allows an opportunity for reflection.1,11 Debriefing allows students to identify actions they took and any actions they would change when presented with a similar scenario. Additionally, students are able to discuss feelings and reactions to the scenario in the debriefing.12 Typically, this faculty-facilitated discussion consisted of what was done well, identification of changes that could have been done differently, and the thought processes behind the decisions that were made.7,10 The students share their reactions and the feelings they had before the simulation, during the scenario, and after the completion of their simulation experience.10,12 Gordon et al13 suggest debriefing sessions after simulation allows for the learner to reflect on the case, identify strengths and weaknesses, and gain feedback necessary for skill improvement and are considered one of the most important components of the simulation experience.
In the case of a skills-based or testing simulation activity, the INACSL debriefing standard suggests that the formalized debriefing could either be replaced or supplemented with supervised practice or feedback or both so the simulation participants will be guided to further improve clinical competence.9 Though it can be characterized by varying levels of instructor interaction and independent practice, for the purpose of this study, supervised or guided practice is defined as a period of time in which the simulation facilitator interacts with the learner in real time while he or she is practicing clinical or skill-based activities. The facilitator may use output from simulation along with his or her observations as the learner is practicing the skill-based activity and then will give guidance and prompts for the leaner to correct his or her techniques so that he or she can improve his or her techniques.10,11 It is important to note within this identified standard of simulation that, while debriefing is a required element, the INACSL uses intentionally vague guidelines and simply suggests incorporating supervised practice in skill-based activities.
The current study aims to evaluate if supervised practice should be a required standard component after debriefing in CPR simulations for athletic training students. Given that CPR is a skill that can be practiced with simulation, the purpose of this quantitative quasi-experimental study was to determine what level of impact participation in a simulation-based cardiovascular emergency scenario using INACSL best practices with simulation design with the Laerdal SimMan (Laerdal Medical, Stravanger, Norway) in a university simulation center in the United States had on undergraduate athletic training students' clinical performance. Specifically related to the central research question in this cohort design with repeated measures study, I aimed to challenge the optional skill-based practice suggestion within the debriefing standard of practice and asked: “Is supervised practice of CPR skills postdebriefing essential for improvements in clinical competency?”
After review and signing the informed consent form, a convenience sample of 46 undergraduate athletic training students enrolled in a professional program completed the study. Participants in the study had completed between 3 semesters (n = 22, 47.8%) and 5 semesters (n = 24, 52.2%) in the professional athletic training program which was 8 semesters in total. The participants who had completed 5 semesters within the professional athletic training program (n = 24) had previously participated in a high-fidelity simulation experience 1 year prior as a requirement of a clinical practicum course, while those that had completed 3 semesters (n = 22) had no experience with simulation. Age of the participants ranged from 19 to 31 years with most participants 20 years of age (34.8%, n = 16) or 21 years of age (41.3%, n = 19). A total of 71.8% (n = 33) of the participants held ongoing basic life support (BLS) certifications renewed annually as part of the professional athletic training program requirement with 21.7% (n = 10) participants having had BLS certification before entering the athletic training program for lifeguard, emergency medical technician, or babysitting roles. Of all the participants, the majority (95.7%, n = 44) had never observed or participated in care during a real-life cardiovascular emergency experience, and 2 participants reported observing a cardiovascular emergency but not participating actively in care for the victim. While all 46 study participants completed the simulation in its entirety, 24 of study participants were randomly selected to be active participants within the study (the others were selected to be observers), and it was only those pairs of participants that were tracked and recorded in the data. For transparency in study design, all 46 study participants were reported here.
After institutional review board approval at the institution where data were collected, participants self-selected into groups of 4 based on available simulation times posted and reported as a group to the simulation center on the day of the study. Groups of 4 were further divided into pairs when they reported to the simulation center by randomly drawing a color that represented a group: participation or observation. Based on their group assignment, they either participated in or observed a high-fidelity emergency cardiovascular simulation using the Laerdal SimMan. Each group began the simulation-based experience with a prebriefing which is an INACSL standard to orient participant(s) to the space and the activity expectations. The 2 participants in the group were instructed to respond to the scenario as if they would a real-life clinical encounter, whereas the observers in the group were instructed not to intervene in the scenario in any way and strictly were just to watch.
Two cardiovascular emergency scenarios were used in this study, both which required the participants to identify a cardiovascular emergency and perform subsequent treatment to include chest compressions, ventilations using a bag-valve mask, and the use of an automated external defibrillator. The scenarios were developed using the INACSL standard framework for scenario or case design and were reviewed by the simulation lab coordinator. The participants began outside of the room that contained the high-fidelity mannequin (victim) and were read an opening sentence describing the scenario background. Within the scenarios, the victim had some level of consciousness to start and then at some point would decline. The researcher was positioned in the computer control room, which was out of sight of the participants. However, the computer control room allowed the researcher to see and hear the participants throughout the scenario while also controlling the physiological responses of the victim. The scenarios took from 10 to 15 minutes to complete. Immediately after the scenario, the group of 4 (participants and observers) reported to the debriefing room with the researcher and participated in a researcher-led debriefing session which lasted between 15 to 20 minutes. The debriefing session included a discussion about what was done well, what went poorly, and what the participants may change in future clinical encounters to improve outcomes. The researcher used the simulation output from the computerized system to guide the feedback to the participants during debriefing. This live-tracking computer software measures performance indicators during the simulated scenario and provides scores and objective output of the participants' actions. For example, the ventilation score is calculated based on the rate, volume, and force of the ventilations delivered to the mannequin. In addition to this feedback during the debriefing, the observers also discussed what actions they saw from the participants during the simulation.
Immediately after debriefing, half of all participants (selected by alternating every other group postdebrief) participated in a supervised practice of clinical skills for 10–15 minutes. During this supervised practice, the researcher sat in the control room and monitored the students' actions on the computer software. Using the overhead speaker, the researcher instructed each participant to perform compressions and ventilations and, while he or she was doing this, would give them feedback such as: “Push harder. Your compressions are too shallow,” “Your hand placement is too far to the right side,” or, “Tilt the head back more. The breaths are not going in.” The researcher would give corrections until the participant was able to perform the skills at a satisfactory level as confirmed by the simulation software. Each participant spent between 5 and 10 minutes working individually with the researcher during these sessions, and then the partners would switch, allowing the other to work individually. The other half of the participants were dismissed for the day immediately after completing the debriefing, as is standard in simulation best practices.
Two weeks later, all participants (those that completed the supervised practice and those that were dismissed immediately after debriefing) returned to the simulation center and completed a second emergency cardiovascular simulation using the Laerdal SimMan. Performance measures were tracked again using the computer software. Only the pair that actively participated in Simulation 1 and Simulation 2 were tracked and recorded using the software. Those in Simulation 1 that had the role of observation were not included in the data, nor were they measured for their CPR clinical competency performance. After the scenario and guided debriefing, all participants were thanked for their time and dismissed from the simulation center.
Clinical competency with associated CPR skills was measured using the Laerdal Learning Application (LLEAP) software program (Laerdal Medical) that interfaces with the simulation hardware. The LLEAP allows the simulation instructor to manage all Laerdal personal computer–operated simulators from 1 platform. Options within the application to use preprogramed scenarios or to run a simulation on manual mode allow for total control of all parameters. The emergency cardiovascular care scenarios developed for this study were run in LLEAP manual mode.
The LLEAP gathers output in real time as individuals are interacting with the high-fidelity mannequin hardware. Among many things the LLEAP system tracks are the event log and the BLS results. The event log describes in real time when treatments are performed on the mannequin, such as when ventilation is given and how long it lasts. The BLS results provide measurement scores of the simulation participant and compare those scores to required performance levels for effectiveness. A sample BLS result page and key to how to read the data is presented in Figure 1. It is important to note the overall performance level is a calculation that factors in all of the individual skills, such as hand placement, compression depth, as well as time. For example, if a participant were to place his or her hands correctly and also perform the compressions at an acceptable depth, but there was a delay in when he or she initiated the care or if there were a time period during the treatment in which he or she did not keep up with the care, the victim would have less chance at a positive outcome because the response time and sequence was poor. Therefore, his or her overall performance level would be lower, even though the skills were done correctly.
Data gathered from the BLS results from the simulation scenarios were recorded and analyzed in the IBM SPSS Statistical Package for Windows (version 24; IBM Corp, Armonk, NY). Data were screened for accuracy, missing data, univariate outliers, and normality. No violations to the assumptions about the data were found.
A 2 × 2 repeated measures analysis of variance (measure: first simulation, second simulation × group: supervised practice, no supervised practice) was used to test levels of clinical competency with emergency cardiovascular care skills with level of significance set to 0.05.
After the first simulation, mean BLS scores including performance level, compression score, ventilation score, compressions with correct hand placement, compressions fully released, and deep enough compressions were calculated and are presented in Table 1. The average performance level on the first simulation was 19% with target performance levels of 75%–100% for advanced performers and 50%–74% for intermediate performers.
There was a statistically significant interaction based on overall performance between groups (F1,10 = 18.70, P < .05, η = 652), indicating participants in the supervised practice after debriefing group were significantly higher (mean = 0.72, SD = 0.05) than those that did not have supervised practice after the debriefing (mean = 0.17, SD = 0.05). This may be viewed in Figure 2. The average performance level on the second simulation was 17% for those that did not have supervised practice and was 72% for those that did have supervised practice after the debriefing on the first simulation.
There was no significant main effect for any of the clinical competency scores from the first simulation to the second simulation for the group that had no supervised practice, indicating no effect on clinical competency scores with BLS skills after participation in a high-fidelity simulation scenario including debriefing. A full report of the second simulation outcomes may be viewed in Table 2.
Approximately 50 psychomotor skills are involved in CPR that must be properly performed to maximize a victim's chance at survival.14,15 One example of a skill component in CPR is performing chest compressions. Many studies14–16 measure CPR skill competency along a continuum from 0% to 100% for each of the core CPR skill components. Greig et al16 measured BLS skill performance over a 3-year time period with undergraduate nursing students. The authors concluded CPR skill competency among components varied greatly over time, but even within a 1-year time period after BLS recertification, CPR skill retention was poor.16 There is great concern with the results of this and similar studies15,17,18 that there is a risk that health care providers who complete BLS certifications may still lack the skill competency to be successful in the event of a cardiac emergency, or skill competency may decline over time. The results of the current study demonstrate that this group of athletic training students who had undergone the BLS certification requirement for the athletic training program and recertified with this requirement annually all lacked the clinical competency to successfully perform CPR and are performing well below even the intermediate provider level.
Factors such as clinical incidence, self-efficacy, motivation, and teaching practices have all been identified as influencing competence in CPR-related skills.19 Teaching practices include both the type of instruction and what fidelity of mannequins and other materials are used for training. The majority of BLS courses use low-fidelity part-task trainers such as the plastic head or foam body Resusci-Anne device.20,21 These part-task trainers are the lowest cost option and provide opportunities to train significantly more individuals in 1 class session than alternative practice options, such as using a full-body high-fidelity integrated simulator.
However, a disadvantage in using part-task trainers in CPR training is the learner will not have the same opportunities to gain feedback on skill performance as he or she might with a higher-fidelity option.20 For example, 1 skill competency component of CPR includes providing correct ventilation volume. A learner may deliver a ventilation to a low-fidelity device such as the Resusci-Anne but will not be able to accurately measure the amount of air that successfully entered the lungs. Computer software applications such as the LLEAP software within high-fidelity equipment allow the instructor to measure and record the specific amount of volume that enters the mannequin's lungs when a breath is delivered. Then the instructor can give this feedback to the student learner, which allows him or her to make adjustments if necessary as he or she is gaining practice and developing competency. However, it cannot be assumed that, just because a participant experienced failure to deliver a ventilation and was given objective feedback during debriefing, he or she will be able to correct this skill and become competency in subsequent cardiac emergency encounters. The current study demonstrates that, without an opportunity for supervised practice after the simulation and debriefing, participants still lack skill competency even though, after the simulation, their self-efficacy at performing these skills is high.
Ventilation volume consistently is reported as the poorest performed skill component of CPR.16,22,23 Madden23 measured competency with this CPR component immediately after nursing students' training with low-fidelity part-task trainers and found that no students passed the criteria for skill competency, and they were unable to deliver the proper amount of air during ventilations. Devlin suggests that the type of instruction and practice that the participant experienced likely had a significant effect in his or her lack of ability to perform this skill, claiming he or she was “insufficiently trained and poorly practiced.”22(p203) Within the current study, the average ventilation score after the first simulation was 31% and was 24% for the groups that did not have supervised practice after debriefing. Even though the best practices standards of effective simulation were followed with this group, those groups that did not have supervised practice did not increase their level of clinical competency with this skill. However, when the supervised practice was added which allowed the participants a chance for real-time feedback and hands-on practice of performing the skill, participants increased to 72%. This finding provides evidence that a supplement of a supervised practice component to the best practice standards of simulation is required for CPR outcome objectives.
The second lowest ranked CPR skill component is adequate depth of chest compression.16,22,23 Again, the depth of chest compression is not measurable with a low-fidelity part-task trainer, so a lack of feedback on improper technique is likely contributing to poor performance with this skill component. In addition to this factor, Devlin22 also identifies critical thinking and self-efficacy as factors to influence a participant's ability to correctly perform chest compressions. The concern is that there may be a disconnect in a participant's belief that he or she can perform the skills (self-efficacy) and how effective he or she actually is at performing that skill.22 Literature does support a relationship between self-efficacy and competency; however, the current study demonstrates that, if the student is not given an opportunity for supervised practice after debriefing, then regardless of his or her level of self-efficacy, his or her clinical competency with this skill will be low.
Professional competency is embedded into all medical and health care educational programs and subsequent professional practice.19,24 Studies widely report that clinical competence deteriorates rapidly over periods of time in which the skill or knowledge is not being used.25,26 This is one factor that prompts the integration of clinical experiences into health care educational programs. When a student is provided with deliberate practice opportunities in a clinical environment, clinical competence and clinical performance can be developed and maintained.19
The most common way that athletic training students gain performance competence is by actually participating in real-world encounters in the clinical setting. As part of all athletic training education programs, athletic training students are assigned to clinical sites where they are under the supervision of a preceptor who is usually a certified athletic trainer.27 Therefore, both athletic trainers and athletic training students tend to share real-world encounters in the clinical setting. Some clinical encounters such as cardiac emergencies are high risk, meaning they involve the death or permanent disability of the victim but have a low incidence in most athletic training students; therefore, it is unlikely that the athletic training student will gain performance competence by actually participating in these encounters.1,8
Simulation bridges this gap and provides learners with environments of varying reality in which they can transition classroom knowledge into clinical performance.28 Hands-on application of skills encourages learners to acquire competency of these skills through experience.13 Several studies2,5,13 have demonstrated that a learner's performance in a realistic simulated environment can accurately predict actual clinical performance. In fact, simulation is among 1 of several methods that are prevalent in measuring athletic training student performance of clinical proficiencies.23 Lasater2 reported that, when used as an adjunct to clinical practice, simulation supported the development of confidence, competency, and appropriate clinical judgment for nursing students in high-stress hospital codes. To measure these reported simulation outcomes on clinical performance, Abrahamson et al29 designed a study in which 6 analyses of clinical performance were measured after anesthesia residents completed simulation training followed by actual clinical encounters of those skills. These authors found that those residents who successfully performed endotracheal intubation training in simulation were significantly more proficient in performing this procedure on actual patients.29 Therefore, transitioning student learners from novices to experts through repeated practice opportunities in simulation translated to improved clinical performance abilities and pose significantly less threat to patient safety.18,21,30 The positive effects of simulation within athletic training education have long been established, as several studies show positive effects of CPR simulation on athletic training student knowledge, confidence, and self-efficacy.1,8 However, it is important to consider simulation criteria necessary to achieve the outcomes and objectives. The current study demonstrates that, within a simulation-based cardiac emergency experience, participants require the scenario, a debriefing session, as well as an opportunity for supervised practice of CPR skills.
LIMITATIONS AND SUGGESTIONS FOR FURTHER RESEARCH
The current study occurred over a relatively short time period (2 weeks). In a study on nursing students' acquisition and retention of CPR knowledge and skills, Madden23 found there was a significant deterioration in CPR cognitive knowledge and skill competency in 10 weeks after BLS low-fidelity CPR training. Other studies20,30 echoed these findings, suggesting that, in as little as 2 weeks after traditional CPR training, there is deterioration in both cognitive knowledge and skill ability. It is very important to note that, while there was a significant deterioration in CPR performance, there was not a total loss of skills learned in training.20,30 In fact, CPR performance remained significantly improved for a time period of 1 year from pretraining to posttraining, even after the deterioration of skills in the weeks after training. Future studies should investigate the long-term effect of CPR skill retention after a high-fidelity simulation with supervised practice.
The current study aims to evaluate if supervised practice should be a required standard component after debriefing in CPR simulations for athletic training students. Given that CPR is a skill that can be practiced with simulation, the purpose of this quantitative quasi-experimental study was to determine what level of impact participation in a simulation-based cardiovascular emergency scenario using INACSL best practices with simulation design with the Laerdal SimMan in a university simulation center in the United States had on professional athletic training students' clinical performance. The aim of the current study was to evaluate if supervised practice should be a required standard component after debriefing in CPR simulations for athletic training students. All simulation-based experiences require purposeful planning to achieve desired outcomes, and it is important to consider best practices such as the INACSL Standards of Best Practice in Simulation. The design and development of a simulation experience should consider what exposure to these learning components will lead to certain outcomes. Even though a multiphase structure of simulation design, facilitation, and assessment are outlined in detail, supervised practice of clinical skills after debriefing is identified as an optional component in skill-based types of simulation encounters. The current study demonstrates that the supervised practice of clinical skills component is vital within emergency cardiovascular simulation encounters for participants to increase clinical competency.