Abstract
Pediatric lumbar puncture (LP) is a common invasive procedure performed by physicians in training. The Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education recognize simulation as a tool for deliberate practice and standardized assessment of procedural performance.
We sought to perform a detailed review of simulated LP performance to elucidate reasons for pediatric residents' reported 26% failure rate.
Participants were enrolled in a single 30-minute session between July 2008 and January 2009. Data collected included former experience and training via questionnaire and video review of intern performance of a simulated LP on an infant model. Intern performance was assessed against a list of 10 procedural elements. Acquisition of cerebrospinal fluid (CSF), the number of elements performed on the first 2 attempts, and specific types of training/experience were analyzed for associations.
All 32 enrolled interns endorsed receiving some previous LP training. Training on a model was infrequent (38%). Interns reported performing a median of 2 LPs prior to enrollment (interquartile range, 2–4). Seven of 31 interns (22%) had yet to perform a live LP. Eleven of 32 interns (34%; 95% confidence interval [CI], 18%–51%) acquired CSF during the first 2 simulated attempts. No specific type of prior training or experience was statistically associated with either the number of procedural elements or successful CSF acquisition (all P > .05). Interns performed a median of 7 of 10 procedural elements (interquartile range, 5.5–8). Early stylet removal was never performed. Complete removal of the stylet with all CSF checks was significantly associated with CSF acquisition (odds ratio, 9; 95% CI 0.98, 84.2). Avoidance of a spinous process upon skin entry was associated with a trend toward increased CSF acquisition (odds ratio, 3.5; 95% CI 0.76, 16.1).
Despite performing many common procedural elements, pediatric interns generally lack the ability to successfully acquire CSF during a simulated infant LP. Expert performance of an infant LP likely requires complete stylet removal with each check for CSF and early spinous process avoidance. A simulated infant LP allowed assessment of intern procedural performance as well as description of elements critical to successful CSF acquisition.
Editor's Note: The online version of the article contains an Appendix, the Experience and Training Questionnaire used in this study.
Background
Lumbar puncture (LP) is one of the most common invasive procedures performed on children by physicians in training.1 Both the Association of American Medical Colleges (AAMC) and the Accreditation Council for Graduate Medical Education (ACGME) consider this an essential skill for pediatric residents to learn.2,3
Despite its critical nature, standard descriptions of LP procedural steps are generally not evidence based and may lack important aspects associated with success. Physicians in training are reported to fail at pediatric LP up to 26% of the time.4 In addition, 2 reports suggest that increasing LP experience during residency has a limited association with improvement in performance.4,5 A detailed review of LP performance might allow the identification of novel and potentially critical elements of the procedure, as well as to help design training methods.6–11 The AAMC and the ACGME recognize simulation as a tool for deliberate practice and standardized assessment of procedural education, as well as for eliminating immediate harm to patients.7,8
We performed this study in order to: (1) describe pediatric interns' prior training and experience with LP, (2) analyze for associations between prior training and experience and the successful acquisition of cerebrospinal fluid (CSF) from an infant model, (3) describe pediatric interns' ability to perform elements of LP via simulation, and (4) analyze for associations between performance of specific procedural elements and successful acquisition of CSF from an infant model during a simulated LP.
Methods
This is a cross-sectional, descriptive study. We enrolled a convenience sample of pediatric interns at Cincinnati Children's Hospital Medical Center from July 2008 through January 2009. In a single, 30-minute session, interns completed a questionnaire on LP training and experience and then performed a simulated LP on an infant model. Interns were blinded to the specific study aims. The training questionnaire divided prior LP training and experience into general versus pediatric (online appendix).
Each simulation session was performed in a patient room in the Emergency Department at Cincinnati Children's Hospital Medical Center. A study investigator confirmed proper model functioning by performing a successful LP on the model (Baby STAP; Laerdal Company, Stavanger, Norway) prior to intern presence. Flesh-colored tape was subsequently applied to conceal needle entry sites. The infant model was then placed on a bed along with a commercial LP tray, gloves (sterile and nonsterile), and 3 standard sizes of LP needle (22-gauge/1.5-inch, 22-gauge/2.5-inch, and 20-gauge/3.5-inch). Two study investigators were present for each simulation: one acting as holder and the other videorecording the intern performance. The “holder” positioned the model based on intern preference.
Each simulation began with the second study investigator (videographer) reading a standard introduction to the intern, which included instructions to direct all communication to the holder, to vocalize each procedural step, to perform the procedure as on a live patient, and that no procedural guidance would be provided.
Following the introduction, each intern was asked to describe proper preparation for a pediatric LP. Although expected to perform all aspects of the procedure, each intern received credit for vocalizing or initiating the following steps: the donning of sterile gloves, the placement of sterile drapes, and the use of local anesthetic. Interns were allowed up to 10 minutes or 5 attempts to obtain CSF from the infant model.
A preliminary list of procedural elements was created a priori from the literature, investigator experience, expert opinion, and standard reference texts.12,13 The primary criterion for inclusion in the list was the potential to impact successful acquisition of CSF, defined as a steady flow of CSF from the LP needle. This list was subsequently modified, in an iterative fashion, as 3 circumstances arose: (1) it was subsequently determined, during observation of intern performance, that an initially included procedural element was unlikely to affect CSF acquisition (eg, elements related to antisepsis), (2) the assessment of an element was limited by the model (eg, flexion of model's lumbar spine did not increase interspinous distance), or (3) the investigators observed a potentially undescribed aspect of the procedure. To assess validity, 7 pediatric emergency medicine attendings and fellows performed an LP on the infant. Five of 7 (71%) obtained CSF on the first or second attempt. All performed 10 of 10 procedural elements. The final list of 10 procedural elements is given in the box, with element definitions listed in the annotation.
Two study investigators (B.T.K. and C.M.P.) used the final list to independently assess each intern's performance. Videotaping and video review were separated by several months. Each reviewer was blind to the intern's experience/training responses at both points in time. A third investigator (M.M.) was included to evaluate any areas of disagreement, and reviewer consensus was ultimately reached for all videos recorded. The study was declared exempt by the Cincinnati Children's Hospital Medical Center Institutional Review Board.
Study outcomes included the number of fundamental elements performed and the proportion of interns successfully obtaining CSF in the first 2 attempts. An attempt was defined as any penetration of the model skin with the spinal needle, regardless of needle repositioning or depth of advancement. Procedural elements (number performed) were analyzed for associations with specific types of training and experience. Predictor variables analyzed for associations with the proportion of interns acquiring CSF on the first 2 attempts included each procedural element, the total number of elements performed, and specific types of training and experience.
Training/experience and the performance of each procedural element are presented as proportions. The number of interns successfully obtaining CSF is presented with a 95% confidence interval (CI). The total number of fundamental elements performed is presented as a median and interquartile range. Wilcoxon rank sum, Spearman correlation coefficient, and Fischer exact test were used, as appropriate, to determine the statistical significance of differences between groups. A 2-sided P value < .05 was considered statistically significant.
Results
Thirty-two of 40 eligible interns were enrolled and videotaped performing a simulated LP. One intern failed to return an experience/training questionnaire. Interns represented 23 medical schools. Nineteen interns (62%) reported no prior training on a model, and 7 interns reported never having performed an LP on a live patient. The median number of live LPs previously performed was 2.
There was no statistically significant association between any type of training/experience and the total number of procedural elements performed (all P > .05; table 1). There were no specific types of LP training that were significantly associated with the ability to obtain CSF.
Eleven of 32 interns obtained CSF from the model on the first or second attempt (34%; 95% CI, 18%–51%), and 3 interns obtained CSF on a third or later attempt, totaling 14 of 32 interns with successful CSF acquisition (44%; 95% CI, 27%–61%). The median number of procedural elements performed was 7 of 10 (interquartile range, 5.5–8). Most interns acknowledged the need for local anesthesia (66%), found and palpated appropriate landmarks before (100%) and after (88%) cleaning, chose the correctly sized spinal needle (59%), chose appropriate vertical (94%) and horizontal (100%) entry points on the model, and had a grossly appropriate angle of entry (88%). None employed early stylet removal.
Individual element performance is presented in table 2. The odds of acquiring CSF on the first 2 attempts were 9 times greater in those interns who removed the stylet with all CSF checks on all attempts (95% CI for odds ratio, 0.98, 84.2). No other individual procedural element had a statistically significant association with CSF acquisition. The odds of acquiring CSF on the first 2 attempts were 3.5 times greater in those interns who avoided a spinous process (95% CI for odds ratio 0.76, 16.1). Interns who obtained CSF performed a median of 8 (interquartile range, 7–9) procedural elements compared with a median of 6 (interquartile range, 5–8) for those who were unsuccessful (P = .037).
Discussion
In this single-institution sample of pediatric interns performing a simulated infant LP, there was no specific type of training/experience statistically associated with the performance of more procedural elements or the acquisition of CSF. Despite ubiquitous exposure to training/experience with LP and majority performance of standard procedural elements, nearly 70% of interns failed to obtain CSF on the first or second attempt. The CSF acquisition statistically occurred more frequently when the intern completely removed the stylet on all attempts. There was a trend toward increased acquisition when interns avoided a spinous process immediately upon skin entry.
Few studies have described the association of prior LP training/experience of physician trainees and LP procedural performance. Lammers et al9 found that for 42 emergency medicine residents performing an LP on an adult model, the aggregate procedural element performance rate was positively associated with the following: prior LP performance on a live patient, supervision of prior performance, and experience in the last month of medical school. Considering that greater than 80% of interns performed 8 of 10 of our procedural elements, we had a limited ability to detect specific associations. However, without a reported analysis for an association between the performance of individual elements and either previous training/experience or CSF acquisition, Lammers et al likely included many elements unassociated with successful acquisition, and there is no way to determine which elements were associated with specific types of training and experience.
Studies of trainee performance of LPs on models and live patients have reported variable success in acquiring CSF. Lammers et al9 reported that 13 of 42 emergency medicine interns (31%) obtained CSF from an adult model. In a single-institution study of 38 pediatric interns, Gaies et al8 reported that approximately 64% (estimated from reported data) obtained CSF on an infant model, and on live patients in the clinical setting, residents at all levels of training obtained CSF 74% of the time.4 Published studies either reported no per-attempt data or used self-reports. With similar success rates, our results and those of Lammers et al are likely more accurate estimates of intern performance per LP attempt, given the direct-observation nature of data collection.
Although standard references for LP procedural elements generally represent expert opinion, no prior study has used simulation to investigate for an association between the performance of specific procedural elements and the acquisition of CSF. Both Baxter et al4 and Nigrovic et al5 found that local anesthetic application and early stylet removal on a live infant LP were associated with CSF acquisition. Despite general endorsement of the need for local anesthesia, technical limitations of the model precluded analysis of its relationship to individual intern performance.
We iteratively assessed intern performance of a simulated infant LP, evaluating for potentially novel procedural aspects that contribute to successful CSF acquisition, and postulated that standard descriptions of the procedure were not comprehensive. No referenced study included the 2 elements we found to be associated with successful acquisition of CSF—complete removal of the stylet on all attempts and avoidance of the spinous process upon skin entry. Complete stylet removal was the only analyzed procedural element with a statistically significant association with successful acquisition of CSF. Omitting this element could ultimately prevent recognition of CSF acquisition.
Incomplete stylet removal was statistically associated with acquisition of CSF. Because the needle is never entirely unobstructed by the presence of the stylet, the “CSF” flow is not as readily noticeable, if at all, when in the proper anatomical space for collecting fluid from the model. Thus, by removing the stylet only partially (failing to clear the hub), a resident might mistakenly conclude that the LP needle tip is not in the subarachnoid space, when completely removing the stylet would have demonstrated fluid. Anecdotally, investigator evaluation of needle position at the conclusion of multiple “unsuccessful” attempts incidentally revealed the presence of CSF when the stylet was completely removed. None of the interns in our sample performed early stylet removal. Because it was possible to perform early stylet removal and incomplete stylet removal on separate attempts by the same intern, both elements were included in the final list.
Avoiding the spinous process immediately upon skin entry was associated with CSF acquisition on the first 2 attempts, although the difference between groups was not significant at the 5% level. Although it is biologically plausible that repetitive local trauma to a spinous process could inhibit successful CSF acquisition by obscuring landmarks, the recognition of this association has not previously been well described. The CI for the odds ratio suggests that with a larger sample size, this element would have a statistically significant association with CSF acquisition on the model.
There are several implications of our study findings. To ensure the effectiveness of education and to promote patient safety, simulation could be used as assessment of a resident's baseline and subsequent performance of LP, and possibly other critical procedures. As a concept, procedural simulation, as opposed to practice on a model, could become a standard for resident education. Procedural simulation could also be used to further assess the elements we found associated with LP success as well as other novel aspects of the performance of LP and other critical procedures. Validated procedural elements would ideally be incorporated into standard curricula.
Our study has several limitations, with the primary concern being the validity of the infant model. Informal feedback from participating interns and faculty revealed unique model characteristics: increased skin stiffness, a fused vertebral column limiting adjustment of interspinous distances, a lack of tissue edema and bleeding, visible but nonpalpable iliac crests, and an inability to move or respond to the procedure. The procedural success rates of pediatric emergency medicine attendings and fellows were comparable with previously published results on live patients, however, suggesting that these differences were unlikely to have falsely affected intern performance of either measured outcome (total elements and CSF acquisition). The AAMC and the ACGME have considered the risk-benefit profile of education on models versus live patients, and they continue to support model use for instruction and assessment.14,15
By including relatively brief insertions of the needle into the model skin as “attempts,” and by qualifying more than 2 attempts as unsuccessful performance, our estimate of intern performance may be low. These limits were chosen in order to balance immediate situational learning (falsely elevating baseline performance) and inadvertent model skin puncture (falsely lowering baseline performance). Although it is possible that an intern would be more likely to acquire CSF with additional simulated or live attempts, this is not substantiated by our data, because only 3 additional interns ultimately obtained CSF.
The performance of interns from 1 year at a single institution may limit generalizability. Our diverse sample of 32 pediatric interns, however, represented more than 20 medical schools, and the sample size is comparable to similar studies. Our preliminary video review and the iterative nature of this study limited our ability to perform a formal assessment of interrater reliability for procedural elements. However, revision of the analyzed elements, both individually and in total, resulted in explicit consensus definitions.
Conclusions
Pediatric interns may know and perform most aspects of LP procedure while lacking the ability to perform the procedural elements critical to successful acquisition of CSF. We have defined 2 previously undescribed elements of the LP procedure—striking a spinous process upon skin penetration and failing to completely withdraw the stylet during CSF checks—which were associated with failure to obtain CSF during a simulated LP on an infant model. Although our findings are exploratory, these elements are likely aspects of expert and successful performance of LP on live patients, and should be incorporated into the education and assessment of LP procedural competence for physicians in training.
References
Author notes
All authors are at Cincinnati Children's Hospital Medical Center. Carla M. Pruden, MD, MPH, is Pediatric Emergency Fellow, Division of Emergency Medicine; Benjamin T. Kerrey, MD, is Assistant Professor of Clinical Pediatrics, Division of Emergency Medicine; Matthew Mittiga, MD, is Assistant Professor of Clinical Pediatrics, Division of Emergency Medicine; and Javier Gonzalez del Rey, MD, MEd, is Professor of Clinical Pediatrics, Director, Pediatric Residency Program, and Associate Director, Division of Emergency Medicine.
The authors would like to thank Mona Ho and Terri Byczkowski for their help with the statistical analysis, and Linda Goldenhar and the University of Cincinnati School of Medicine for providing a seed grant in support of this study.