Improvement of Immediate Performance in Neonatal Resuscitation Through Rapid Cycle Deliberate Practice Training Open Access

To graduate from pediatrics residency, the Accreditation Council for Graduate Medical Education requires residents demonstrate procedural competency in bag-mask ventilation and umbilical venous catheter placement in neonates.¹  Despite these requirements, many pediatrics residents do not feel comfortable with neonatal resuscitation (NR).^2–5  This may be due to residents receiving less delivery room exposure because of work hour restrictions, a curriculum shift with a greater focus on outpatient-based experiences, increased supervision, and the incorporation of midlevel providers into the health care team.^5,6 

Many training programs have developed neonatal simulation curriculums in an effort to address these limitations. Studies have shown that simulation-based training in neonatal resuscitation improves observed abilities and confidence levels in residents.^7–9  However, retention of NR skills quickly declines. Significant decreases in megacode assessment scores have been described as early as 1 month, with further decrease within 3 to 4 months.¹⁰  Also, learner confidence in NR skills has not always shown to correlate with observed abilities.⁴ 

Most traditional simulation methods implement a simulation debriefing (SD) technique. A case scenario is presented, the health care team carries out the entire simulation, and a debriefing session follows. Hunt et al¹¹  described the rapid cycle deliberate practice (RCDP) method, which allows the simulation facilitator to provide debriefing throughout the simulated case. The simulated case is presented, and the team begins resuscitation efforts until a teaching moment presents itself. The facilitator provides teaching points, and then the team restarts the resuscitation exercise. Each time the health care team advances further through the scenario. RCDP is based on 3 main principles: (1) maximizing the time the learner spends “doing it right” and further developing muscle memory; (2) facilitators provide proven solutions or strategies for overcoming common obstacles; and (3) participants learn to embrace feedback. Since the learner has multiple chances to go through the scenario, the feedback is looked at less like a skills test and more like “coaching.”^11–13 

It is important to determine which method of simulation training will promote the highest performance. This question is applicable to many different types of resuscitation training, not just NR. The objective of our study was to compare pediatrics residents' observed abilities, confidence level, and recall in NR after receiving RCDP as opposed to SD.

A prospective, randomized control study compared 2 educational techniques at a large pediatrics academic training program. The study occurred over a 1.5-year period, with 3 to 4 interns enrolled each month.

Thirty-eight pediatrics interns were randomized to an NR teaching simulation session using either RCDP or SD. All participants were Neonatal Resuscitation Program (NRP) certified at the start of their internship and on a neonatology or newborn nursery rotation when enrolled. Randomization was performed in blocks of 4 interns to account for variations in abilities during the first year of academic training. Each intern completed a presurvey, which delineated his or her previous experience and confidence level, rated on a 5-point Likert scale (1, strongly agree; 3, neutral; 5, strongly disagree), with NR.

Each 45-minute NR teaching session was facilitated by the same neonatology fellow trained in NRP instruction and RCDP methods (figure). All scenarios were performed on the Laerdal SimNewB high-fidelity simulator (Laerdal Medical, Wappingers Falls, NY). The case scenario was a term infant born with a heart rate of 50 beats per minute and no respiratory effort. Interns were taught all aspects of NR in accordance with the 2015 NRP guidelines (except intubation due to time constraints and the guidelines placing emphasis on effective positive pressure ventilation [PPV] by bag-mask ventilation).¹⁴  Teaching points and overall learning experience were kept consistent between the RCDP and SD groups with prewritten scripts, setup checklists, and teaching point checklists. To monitor for inconsistencies, teaching was observed by a senior neonatologist NRP instructor trained in simulation.

Immediately following the simulation session the interns underwent a 15-minute videotaped simulation test with a similar scenario. The neonatal resuscitation team included the intern and 2 research assistants who were provided instruction on their roles. The pediatrics intern led the resuscitation without any guidance from the facilitator or team. Following completion of the simulation, the participants filled out a postsurvey addressing confidence. The simulation scenario was recorded on the Laerdal Debrief Viewer software (Laerdal Medical) and a second video camera. For recall, the interns underwent a second videotaped simulation 4 months from their initial session.

The primary outcome of the study was performance on the initial videotaped simulation test scored on the Megacode Assessment Form (MCAF) by 2 blinded neonatologists (provided as online supplemental material). This checklist was expanded from the integrated skills station performance checklist published in the NRP instructor manual by adding single task skills for assessment.¹⁴  The MCAF was tested on 3 interns, not included in the final study results, and then edited by the research team to improve interrater reliability and specific emphasis on each task. Interrater reliability was 88% to 90%. Secondary outcomes included confidence level in NR and the amount of time elapsed to perform critical actions. The interns' performance during the recall session was evaluated in the same manner. Actions that were deemed as harmful, such as not flushing the umbilical venous catheter prior to placement and forgetting cord tie placement, were recorded.

Written informed consent was obtained from all participants, and participation was voluntary. The study was approved by the Columbia University Institutional Review Board.

Sample size calculations were based on a similar institutional study that utilized an NRP refresher training as the intervention to improve NR in pediatrics interns.¹⁵  Thirty-four participants were necessary to find a difference in MCAF scores of 11% between study groups with a standard deviation of 10, alpha of 0.05, and power of 80%. We increased our sample size to 38 to account for dropout, technical issues, and test cases.

Using R statistical software (The R Foundation for Statistical Computing, Vienna, Austria), independent sample t tests were used to compare the interns' cumulative scores on the initial MCAF, recall MCAF, and the time to preform critical interventions. The number of interns who performed critical actions within appropriate cutoff times, had increased confidence level, and performed harmful actions was compared using Fisher's exact tests. A linear regression analysis was used to adjust the association between intervention and outcomes for NR exposure in the neonatal intensive care unit (NICU) or nursery during the time interval between initial and recall sessions.

Of the 38 interns enrolled, the first 3 were not included in the analysis due to changes in developing the scripts, checklists, and the MCAF. An additional intern was excluded because of technical issues, leaving a total of 34 interns in the analysis. Previous exposure to NR did not differ between the RCDP and SD groups (table 1).

MCAF scores immediately following the simulation were higher in the RCDP group than the SD group (table 2). When assessing the MCAF by corresponding chapter in the NRP 2015 textbook, chapter 3 on PPV had the greatest difference in score (89% [6.6] versus 81% [8.5]; P < .005; 95% confidence interval [CI] −12.3 to −1.7).¹⁴  The most common error that occurred during the simulation tests was not increasing the oxygen (35% versus 88%, P < .005).

More interns in the RCDP group initiated PPV within 1 minute, ventilated the patient for at least 25 seconds prior to starting chest compressions, and administered epinephrine earlier. No differences were seen in the number of harmful actions performed between the 2 groups (table 2). Learners self-reported increased confidence in neonatal resuscitation regardless of the teaching method.

The mean time elapsed between the first session and the recall session was 132 (± 13) days for both groups. The recall MCAF scores were not different between the 2 groups (table 3). However, the decrease in subject score from the first session to the recall session was greater in the RCDP group than the SD group. In the recall study the average time and number of interns who performed PPV, chest compressions, and administered epinephrine were the same between the 2 groups. Recent clinical neonatal resuscitation exposure improved scores regardless of the initial simulation teaching method (78% [6.8] versus 68% [10]; P < .003; 95% CI −0.15 to −0.03).

Immediately following RCDP interns had improved observed abilities and decreased time to perform critical interventions in NR simulation as compared to those trained with the SD method. These differences were not sustained at the 4-month recall session, and learner confidence was not affected by type of teaching.

The immediate improvement in observed abilities following RCDP teaching is consistent with the current literature. Lemke et al¹³  showed that team performance was improved in simulated pediatric advanced life support taught with RCDP compared to traditional SD. Hunt et al¹¹  showed decreased time to compressions and defibrillation from the onset of pulseless ventricular tachycardia in a hospital pediatric simulated arrest after RCDP teaching.

While RCDP methodology has great appeal, since deliberate practice has proved an effective teaching strategy and the rapid cycle notion creates feasibility, the literature on recall following RCDP teaching is limited. The length of the recall period could affect the results. Recall MCAF scores were higher for interns who were in the NICU or nursery between the initial teaching and recall simulation test. This illustrates the importance of “just-in-time training” and refresher courses. Using the RCDP training method for just-in-time training and refresher training before residents begin their newborn nursery or their NICU rotation could improve learner skills. A curriculum with frequent NRP-based RCDP simulations also could improve learner recall from an initial NRP course.

The RCDP method was just as feasible to use as the SD method. Time was limited to 45 minutes for both teaching sessions, and the facilitator was able to quickly reset the equipment for each RCDP cycle. RCDP methodology is best used for simulation that follows an algorithm, which requires the intern to perform precise procedures. Algorithms typically provide the best way to respond so the facilitator has clear points to stop the team and provide direct feedback. It is unlikely that RCDP would work well when there are many different correct approaches to solving a problem. RCDP has been studied in both neonatal and pediatric resuscitation training and would likely be useful in other fields of medicine.

Our study has limitations, including the limitations of the MCAF tool for measuring NR skills. There is not a set passing score on the MCAF because it is based on the NRP program's integrated skills station performance checklist, which does not have a set passing score for the skills portion of the NRP provider course.¹⁴  Another limitation is that some interns who scored high on the MCAF also performed harmful actions that could lead to a negative outcome in a real-life scenario. The generalizability of the study is limited considering the small number of participants at a single site.

Future studies should be performed to determine if frequent brief RCDP NR simulations improve overall retention. Varying amount of times between training sessions and considering a set passing score to determine mastery would be interesting aspects to consider in future studies.

Pediatrics interns had improved observed abilities and decreased time to perform critical interventions in NR simulation immediately following RCDP as compared to those trained with the SD method. Neither approach was superior in improving confidence level and recall 4 months later.