Abstract
Evidence suggests that simulation leads to improved operative skill, shorter operating room time, and better patient outcomes. Currently, no standardized laparoscopy curriculum exists for gynecology residents.
To design a structured laparoscopy curriculum for gynecology residents using Delphi consensus methodology.
This study began with Delphi methodology to determine expert consensus on the components of a gynecology laparoscopic skills curriculum. We generated a list of cognitive content, technical skills, and nontechnical skills for training in laparoscopic surgery, and asked 39 experts in gynecologic education to rate the items on a Likert scale (1–5) for inclusion in the curriculum. Consensus was predefined as Cronbach α of ≥ 0.80. We then conducted another Delphi survey with 9 experienced users of laparoscopic virtual reality simulators to delineate relevant curricular tasks. Finally, a cross-sectional design defined benchmark scores for all identified tasks, with 10 experienced gynecologic surgeons performing the identified tasks at basic, intermediate, and advanced levels.
Consensus (Cronbach α = 0.85) was achieved in the first round of the curriculum Delphi, and after 2 rounds (Cronbach α = 0.80) in the virtual reality curriculum Delphi. Consensus was reached for cognitive, technical, and nontechnical skills as well as for 6 virtual reality tasks. Median time and economy of movement scores defined benchmarks for all tasks.
This study used Delphi consensus to develop a comprehensive curriculum for teaching gynecologic laparoscopy. The curriculum conforms to current educational standards of proficiency-based training, and is suggested as a standard in residency programs.
Simulation improves operative skill, but no standardized laparoscopy curriculum exists for gynecology residents.
A structured laparoscopy curriculum was designed using experts and the Delphi consensus methodology.
Potential for selection bias; lack of data to validate the curriculum.
The resulting curriculum conforms to current educational standards for proficiency-based training.
Editor's Note: The online version of this article contains a geographical representation of expert responses, a flow diagram detailing expert consensus, and a table of expert levels of proficiency for curricular tasks.
Introduction
Laparoscopic surgery has become the standard of care for many gynecologic operations. Numerous obstacles exist for learning the principles and technical skills necessary for minimally invasive operations. This challenges the traditional preceptorship model of surgical education.1–6
Simulation has been shown to be the most significant contributor to laparoscopic training of residents.7 Traditional box trainers and virtual reality (VR) simulators allow novice trainees to progress along the steep part of the learning curve before entering the operating room.1,4,8,9 Surgical bodies emphasize that training in laparoscopic surgery should take place in a simulated environment before the learner seeks experience in the operating room.1 The value of simulators may be enhanced through achievable proficiency benchmarks.3,4,10–12
Comprehensive curriculum development is lacking despite the established value of simulators in surgical education.2,3,7,9,13,14 Ideally, curriculum development should reflect practice patterns at diverse institutions,3 and encompass cognitive knowledge, technical skills, and nontechnical skills.3,13–15 A North American surgical skills curriculum covering these areas was recently developed for surgical residents.16
Our primary objective was to use Delphi methodology to support the development of a laparoscopy curriculum for gynecology residents. A secondary objective was to define proficiency benchmarks for the VR component of the curriculum.
Methods
Study Design
We used a modified Delphi method to achieve consensus on the components of a basic laparoscopic surgery curriculum for gynecology residents. The Delphi method—an iterative process in which responses are collated, analyzed, and presented anonymously back to participants—provides a means for obtaining opinions of experts in a systematic manner.15,17 The approach has been used to develop surgical curricula.3,18,19 An initial curriculum Delphi survey defined the curricular components (cognitive, technical, and nontechnical). This was followed by a VR Delphi survey that defined the technical VR component of the curriculum. Lastly, a cross-sectional design determined expert levels of proficiency for the defined VR curriculum.
Participant Selection for the Delphi Consensus
Sixty experts identified in a previous study20 volunteered to participate in the curriculum Delphi. Experts were either program directors or delegates from North American obstetrics and gynecology residency programs (54 US and 6 Canadian). Program directors were chosen because of their leadership role in academic obstetrics and gynecology, their institutional influence on surgical education, and their authority on departmental finances. The VR Delphi involved 10 experienced gynecologist users (9 US and 1 Canadian) of a laparoscopic VR trainer (LapSim, Surgical Science Inc). Membership in both expert panels was anonymous and concealed from participants.
Derivation of Survey Items
A traditional Delphi technique requires the participants to suggest survey items. Participants for the curriculum Delphi were supplied with a list of survey items based on literature review and local expert opinion.19,21–24 This list was compiled within the context of a general obstetrics and gynecology residency program and included 12 questions (78 subheadings) pertaining to cognitive content, 10 questions (33 subheadings) pertaining to technical content, and 2 questions (13 subheadings) pertaining to nontechnical skills.
The LapSim 2013 edition was selected for the VR Delphi. LapSim has previously been evaluated for construct validity, learning curves, and skills transfer.4,8,10,25–27 The role of the VR Delphi panel was to determine which of 13 basic and 4 gynecologic procedural tasks were relevant for junior gynecology residents.
Survey Administration
Online versions of the surveys were created using the web-based software SurveyMonkey. Prior to administration, 2 faculty members, 3 fellows, and 2 residents pretested the surveys. Individuals completed the survey and provided feedback to the authors to ensure question clarity. Participants were asked to rate the importance of including each item in the final curriculum using a Likert scale from 1 (strongly disagree) to 5 (strongly agree).
An e-mail was sent to volunteers who had agreed to participate in the curriculum Delphi.20 Five reminders were sent out at 1-week intervals. Data collection stopped 42 days after the original invitation.
The VR Delphi was distributed via e-mail. There were 2 iterations, with 2 reminders each. Data collection stopped after 35 days for the first iteration, and after 51 days for the second iteration.
Final Structure of the Curriculum
After consensus was achieved, an outline of the final curriculum was created that included items that more than 80% of experts ranked as a 4 (agree) or 5 (strongly agree; curriculum Delphi and VR Delphi). Items were divided into cognitive component, technical skills component (box trainer and VR), and nontechnical skills component.
Final Structure of the VR Curriculum
Generation of Proficiency Levels for VR Curriculum
Ten experienced gynecologic laparoscopists were identified (5 Canadian and 5 Danish) in the authors' institutions to establish the benchmark levels. An experienced surgeon was defined as an individual who had completed more than 100 advanced minimally invasive procedures.3,4 Experts selected possessed advanced laparoscopic skills, were involved in laparoscopy training of residents, and had an interest in laparoscopy education.
A member of the study team (H.H. or F.B.) familiarized each surgeon with the simulator using instructional videos. Warm-up time on the simulator was not permitted. Each surgeon performed each task at each level in a predetermined sequence without assistance from members of the study team.
Benchmark scores were based on the automatic assessment parameters generated by the simulator. The parameters of interest were time and economy of motion (angular path length and path length), which are consistently measured on all LapSim tasks and demonstrate construct validity.3,8,10,24,26 Benchmark scores for time, path length, and angular path length were calculated for each curricular component at each level.
The hospital's Research Ethics Board approved the final version of the questionnaire and the dissemination plan.
Data Analysis
SPSS version 20 (IBM Corp) was used for data analysis. For the Delphi surveys, Cronbach α, medians, and interquartile ranges were calculated and reported back to the expert panelists. Median expert scores determined proficiency levels for the identified VR curricular tasks.3
Results
Comprehensive Curriculum Delphi
Of the 60 experts contacted,20 39 (65%) participated in the curriculum Delphi (provided as online supplemental material). The first iteration data were analyzed in 3 ways to assess consensus (provided as online supplemental material). Regardless of the method of analysis, consensus was reached after the first round, and further rounds were not required. Items that were included in the final curriculum (table 1) were rated as 4 or 5 by at least 80% of experts from a cognitive (74 of 78), technical (16 of 33), and nontechnical (13 of 13) perspective. See the figure for the proposed curriculum.
Flow Diagram Depicting the Proposed Curriculum
The proposed curriculum was developed by incorporating the elements from table 1 that met criteria for inclusion based on the Delphi consensus.
Flow Diagram Depicting the Proposed Curriculum
The proposed curriculum was developed by incorporating the elements from table 1 that met criteria for inclusion based on the Delphi consensus.
VR Delphi
Few VR tasks met the criteria for inclusion in the curriculum Delphi. This obviated the requirement for a VR Delphi. Nine of 10 experts (8 US, 1 Canadian) participated. After the first iteration, Cronbach α was 0.52, indicating a lack of consensus. The 9 experts achieved consensus with the second round of the survey (Cronbach α = 0.80). LapSim tasks that at least 80% of experts rated as 4 or 5 were included in the final curriculum (table 2).
Expert Proficiency Benchmarks
Ten experienced surgeons completed the VR curriculum (6 tasks at 3 levels of difficulty), 5 from each participating institution. Half of the surgeons were men; all surgeons were right-handed, with the exception of 1 who was ambidextrous. Median scores were set as benchmarks for each task (provided as online supplemental material) because the data did not follow a normal distribution.
Discussion
This study used modified Delphi consensus methodology to establish the content of a laparoscopy curriculum for gynecology residents, conforming to current standards of proficiency-based training.11 The curriculum includes cognitive, technical, and nontechnical skills elements, aligning with Zevin's principles of comprehensive curricula design.3,6,7,13–15 North American obstetrics and gynecology programs lack a standardized laparoscopy curriculum for residents, despite multiple publications describing its necessity.7,9,19,21,22 The curricula in most programs may lack aspects of comprehensive curriculum design because they were developed at a local level.3,13,15 Comprehensive national curricula designed for junior gynecology residents were described in Denmark and the United Kingdom,9,18 yet neither included a nontechnical skills component. Previous work has demonstrated that basic task training using VR simulators translates into improved performance in the operating room.3,29
In addition, the use of simulators for task training is more effective if the proficiency benchmarks are based on expert-derived performance goals for the chosen tasks on the available simulator.3,4,7,8 10,11,30 In the present study, benchmarks were formulated based on the simulator-generated scores using time and economy of motion, which are previously validated parameters.3,4,8,25
Our efforts resulted in an international consensus statement on the elements of a structured laparoscopy curriculum for gynecology residents. Thirty-nine program directors or delegates (representing 15% of North American obstetrics and gynecology programs) participated, which enhanced the generalizability of the findings. The Delphi methodology offers the opportunity to recruit experts from a wide geographical distribution without assembling them in a central location,31 thereby decreasing costs and increasing feasibility.18,31 The anonymous nature of the Delphi methodology results in all respondents' opinions being weighted equally, thus preventing 1 person's opinions from swaying the group.31 The ability to achieve a Cronbach α > 0.80 in the first round reflects the inclusion of items that are well accepted by laparoscopic gynecologic educators.
The findings of this study and the elements selected for inclusion align with a recent UK study18 that determined a basic laparoscopic gynecology curriculum. Our contribution is unique in that it identifies a nontechnical skills component in addition to technical skills and cognitive components, similar to the national skills curriculum for surgical residents.
Our study has some limitations. A Delphi consensus relies on the inclusion of expert volunteers with knowledge and interest in a particular area,17 which may introduce selection bias. In addition, not all volunteers completed the curriculum Delphi; this may have been partly due to the long list of potential items. Regional differences in access to simulation technologies (ie, VR simulators) may have influenced the acceptance or rejection of an item. Finally, our proficiency scores may have been influenced by prior differences in participants' exposure to the VR simulator, resulting in higher scores that may be difficult for a novice laparoscopist to achieve.
We plan to use the information from this study to determine if a structured laparoscopy curriculum that includes the cognitive, technical skills, and nontechnical skills selected will translate to improved operating room performance by residents.
Conclusion
This study utilized Delphi methodology to reach consensus on the elements (cognitive, technical, and nontechnical skills) to include in a comprehensive laparoscopy curriculum for junior gynecology residents. Experienced gynecologic surgeons established benchmarks for the identified curricular VR tasks at basic, intermediate, and advanced levels. In addition, standardized curricula for selection, in-training, and certification of surgeons are lacking, and the proficiency benchmarks we identified may contribute to these applications. The curriculum conforms to current educational standards of proficiency-based training and can be offered as a standard in residency programs.
