Objective Structured Assessment of Technical Skills to Teach and Study Retention of Fourth-Degree Laceration Repair Skills

Fourth-degree lacerations are infrequent but serious complications that involve the tearing of the vaginal epithelium, tissues of the perineal body, anal sphincter complex, and the rectal mucosa at the time of vaginal delivery.1 Prior studies2 have shown a high incidence (up to 60%) of perineal pain, anal incontinence, and dyspareunia after severe perineal injury during childbirth. Perineal lacerations after childbirth are classified as first-, second-, third-, and fourth-degree on the basis of the tissues involved; third- and fourth-degree are considered serious because the external anal sphincter is involved in the laceration. Unfortunately, residents are universally undertrained in the repair of fourth-degree lacerations because of their infrequency.3 However, it is vital that graduating residents be able to repair a fourth-degree laceration even if they did not encounter one during their training. Prior research has demonstrated that the combination of the Objective Structured Assessment of Technical Skills (OSATS) and a written examination before and after an educational workshop is a valid and reliable way of assessing residents' improvement in surgical skills and knowledge in the repair of fourth-degree lacerations.4,5 

We sought to expand this work by including a 6-month follow-up OSATS and written examination to evaluate the retention of information learned during the workshop. We hypothesized that residents' technical skills and knowledge to repair fourth-degree perineal lacerations will reflect significant improvement 6 months after the intervention in comparison with the pretest prior to the workshop.

Obstetrics and gynecology residents from The George Washington University (postgraduate year [PGY]–2 to PGY-4) who attended a workshop to learn fourth-degree laceration repair as part of their regularly scheduled didactic session were included in the study. We chose not to include PGY-1 residents in the study because they were less likely to be comfortable with repair of more basic first- and second-degree obstetric lacerations, and we did not expect them to be familiar with third- and fourth-degree repair yet. The PGY-2 to PGY-4 residents first took a pretest (composed of a multiple-choice written test and a hands-on test) to assess their baseline knowledge and skills. The written test had a maximum score of 10 points. The hands-on portion of the evaluation involved performing a fourth-degree laceration repair on an inanimate model; residents were observed and graded by using a standardized OSATS checklist with a maximum score of 18 points. We then performed the educational intervention. This included a lecture, a demonstrational video, and an opportunity to practice on the same inanimate model. The residents then completed a posttest on the same day as the educational workshop, which involved the same written and hands-on components as the pretest. The same posttest, both written and hands-on components, was administered 6 months after the educational intervention to determine if the residents retained the knowledge and skills they were originally taught. The written examination and the OSATS were graded by someone other than the primary investigator of the study. Copies of the written test and the OSATS checklist are attainable from the authors.

All available residents were expected to attend the educational session as part of their regularly scheduled weekly didactic session. Residents who were on a night float rotation, on vacation, or on an off-service rotation were not able to attend. Although all available residents were expected to attend the educational session, it is possible that some chose not to attend. The 6-month posttest was administered as part of the residents' end-of-year examination (which includes other written questions and hands-on scenarios). Since all PGY-2 to PGY-4 residents took the end-of-year examination, we had an unplanned control group who took the 6-month posttest but had not participated in the educational intervention. At the 6-month written posttest, informed consent was obtained from all PGY-2 to PGY-4 residents to participate in the study.

This study was determined to be exempt by The George Washington University and Hospital Institutional Review Board.

Scores of the intervention group were compared to their previous scores by using a paired t test. Scores of the intervention group were compared to the control group by using an unpaired t test.

Of the 28 PGY-2 to PGY-4 residents, 17 (61%) participated in the educational intervention, and took the pretest, immediate posttest, and 6-month posttest. For the control group, 11 additional residents took the 6-month posttest but had not completed the educational intervention.

Results of the written and OSATS tests before the educational intervention, immediately after the intervention, and 6 months later are summarized in table 1. Improvements compared to the pretest scores were statistically significant both immediately after, and after 6 months, for both the written and OSATS components. Scores for both aspects of the test did decrease by a statistically significant amount at 6 months, compared to immediate posttest scores, but the range of difference in score was small. The confidence intervals listed are the range of the score improvement (delta), not for the score itself.

table 2 compares the test scores of residents who attended the educational workshop 6 months prior (intervention group) to the residents who had not attended the workshop (control group). Both written and OSATS test scores of the group that attended the workshop were significantly better than those of the control group.

This study found that residents improved on the written examination and OSATS concerning the management of fourth-degree perineal lacerations after completing the educational workshop, and maintained most of this improvement 6 months after the intervention. In addition, residents who participated in the educational intervention performed better on both evaluations than residents who did not participate.

This is 1 of few published studies that explore whether the improvement after an educational intervention or simulation is retained after a significant time interval following the intervention. Regarding repair of third- and fourth-degree obstetric lacerations, some studies6,7 have reevaluated subjects 4 or 5 weeks after the intervention. Studies in other fields have also only reevaluated subjects 4 to 6 weeks after the intervention. Of note, 2 studies in anesthesiology8,9 stood out because they evaluated the retention of skills to perform an emergency cricothyroidotomy up to 1 year after a simulation was conducted. Another study10 evaluated the loss of skills monthly after simulation; deterioration of skills started 4 months later but less practice was required to regain skills than originally required.

There are several limitations to this study. The principal limitation is the small sample size of residents. In the future, we may be able to study residents from multiple programs to obtain a larger sample size. The same tests (written and OSATS) were administered at each interval, so some of the improvement may have been from the residents learning the test rather than learning the material. Finally, fourth-degree laceration repair on an inanimate model varies greatly from the real clinical scenario in many ways: the inanimate model is immobile and positioning is optimal, the model has clean lines of laceration and clear tissue planes, and there are no obscuring body fluids. However, the experience of performing the repair even on a low-fidelity model is most likely better training than no exposure at all.

Notably, the control group's OSATS scores were higher for the pretest than those of the intervention group. Several factors may have contributed to this difference in baseline scores. The material covered in each didactic session was made available to all residents after the session, so the control group had access to specific information about laceration repair before their first attempt at the tests. Second, since it was part of the end-of-year examination, residents were given a list of the topics covered at the resident didactic sessions and may have studied the topic of obstetric laceration repair before the examination. Third, residents may have had additional clinical experience in the 6 months before the test was administered. Lastly, there could have been a selection bias in that a resident who felt comfortable with fourth-degree laceration repair may have chosen not to attend the educational session.

Residents retained much of the surgical skills and knowledge they learned at an educational workshop for fourth-degree laceration repair 6 months after the workshop. In addition, those residents performed better than residents who had not attended the workshop.