Computer-Enhanced Visual Learning Method: A Paradigm to Teach and Document Surgical Skills

Changes in the environment for surgical residents' training are prompting residency programs to rethink and restructure how to teach surgical skills to residents.1 Among many changes causing concern, the reduction in duty hours is the one most frequently cited. Limits on residents' hours are a logical predicament because of the assumption that they result in reduced operative volume and fewer opportunities to teach residents in the operating room, with the potential to contribute to decreased surgical competency in residency graduates. We describe Computer-Enhanced Visual Learning (CEVL) as a pragmatic approach to improve residents' learning and participation in operative practice. We developed CEVL as a tool to enable residents to obtain feedback on their surgical performance in an era of significant changes in medical education and practice.2 CEVL is an alternative to the traditional process of learning procedural skills simply by performing more cases, frequently without a defined strategy to improve performance. We believe that teaching surgery using the CEVL method uses time in the operating room more efficiently, allowing residents to finish their rotations with better skills and more confidence than before. The objective of CEVL is to provide a method for learning surgery inside and outside of the operating room.

CEVL is an Internet-based program that teaches pediatric urologic surgical procedures in 3 steps. First, residents use Internet access to study the procedure. Procedures are presented as components that incorporate intraoperative video and images, explanatory text, and computer animations. Second, residents perform the procedure with supervision. Third, residents study remediation strategies using online computerized learning resources. The residents build their skill sets using the deliberate practice- and case-focused feedback that CEVL provides. We believe that the method is widely applicable to teach residents to perform routine health care procedures.

Since 2003, the Division of Urology at Children's Memorial Hospital has been exploring methods to advance the teaching of routine surgical procedures to residents. By serendipity, the current changed environment and the availability of technology created an opportune environment to explore alternative techniques to teach surgery and concurrently evaluate residents' performance and progress.

Our research began with many accumulated hours of informal weekly conversations with residents. During this time, we explored various aspects of the residents' immersion in the operating room, including how residents think and learn to perform surgery; what residents want, respect, and emulate; and why some methods of teaching surgery are more effective than others. This needs assessment identified the following 2 major desires of residents in the operative arena: (1) to learn how to perform surgery, presented as broad components and specific steps, and (2) to receive immediate focused feedback on performance and remediation of suboptimal aspects. These and other characteristics of pediatric urology residents as a group are listed in the table, along with the CEVL method responses.

The method by Ericsson3 of using deliberate practice to learn psychomotor skills has been applied to a range of activities from playing the piano to ice skating. The mantra of the method is a cycle that begins with study of the basic skill (eg, how to play a scale on the piano). Next, the learner performs the skill with teacher supervision, and the learner then receives specific feedback on how to improve performance (eg, how to adapt the finger positions to sound the fourth note in the scale). This routine facilitates improved student performance as elements that are suboptimal are identified and instructions for remediation are structured.

The CEVL method adapts the method by Ericsson3 and modifies it to present surgical skills learning as a repeated cycle of the following 3 steps: (1) mentally internalize the procedure, (2) perform the procedure, and (3) make improvements based on feedback. CEVL modification promotes the concept of “deliberate performance”–based deliberate practice. The learner internalizes the remediation strategies to execute during the next performance or the next case. This allows performance ratings to spiral upward. The CEVL logo graphically depicts this notion of internalizing a task to facilitate performance (figure).4 

Anecdotal observations recognized that, while teaching occurs in the operating room, residents derive additional benefit from resources outside of the operating room. These can enhance their operative experience and help them solidify their skills. Desired processes for learning include the review of operating room teaching dialogue via Internet access and the evaluation of data. To this end, a secure website page (http://www.cevlforhealthcare.org) was created to provide Internet access to the curriculum content. This provides residents access to study the surgical procedure and to view their performances, feedback, and remediation strategies. The elements of the CEVL program are described herein.

We aimed to formalize residents' education by presenting the information in a stylized manner. A dashboard of curricular content was built by combining the insights of several of us in biomedical visualization (E.M., R.S., and C.Y.) with surgical perspectives (M.M.). The dashboard presents the performance of surgery as a series of major procedural segments called components. Each component in turn is subsegmented into steps. The information is presented as text and visuals (figure).

The CEVL script for how to perform the surgery is publicly displayed in the operating room. An important observation on residents' behavior is that residents frequently are not yet comfortable enough to think logically and clearly in the operating room. While they may solidly understand the basics of a procedure when studying at home, insecurity may overwhelm this understanding once they enter the operating room to an extent that the resident cannot lead the operation because he/she no longer is confident of the steps involved. We found that such experiences may be minimized by developing an agreed-on CEVL script to perform the procedure, which is posted plainly for all operating room staff to view.4 In this manner, the instruments and protocols to be used are known to all operating room staff, and the script can be referred to as needed during the case. This readily available knowledge of the procedure bolsters residents' confidence (table).

Rating the performance of surgery is a checklist function to objectively assess a resident's performance of component and integrative skills. After each procedure, the supervising surgeon gives the resident qualitative and quantitative feedback on his/her performance using an online feedback and rating form. The resident and attending can access this form for review once completed, and the attending reviews the performance with the resident. The form also allows the resident to note the steps he/she would like to improve at the next case. Strategies to remediate skills are formulated collectively by the resident and the attending during the postprocedure discussion. The resident studies these steps for improvement and prepares to execute them during the next case.

CEVL provides reports of residents' performance ratings as the clinical rotation progresses. Residents and attendings may review these ratings to confirm that there is progress toward improved performance and/or to identify problem areas that need special attention. Initial results show that 96% of residents improve their CEVL scores.2 The average results improved by at least 50% from entry to best rating (P < .0001). Further research is needed to validate the CEVL scoring system and to compare it with global assessment ratings. The checklist nature of the ratings complies with Accreditation Council for Graduate Medical Education standards.

The residency PD reviews and evaluates CEVL content to determine if the template-driven performance of surgery showcased on CEVL matches the needs of the given program, and the PD is able to modify the surgical content to meet local needs. For example, if the program chooses to use a sterile button to help anchor the testis at orchiopexy, this step is added for this program's content, but the content of other programs in the CEVL system is not altered.

The PD transmits e-mail addresses of the residents who will be using the CEVL program. During registration at http://www.cevlforhealthcare.org, only authorized residents may log in to CEVL by using a secure username and password. Residents and the PD may view residents' performance in real time. At present, no financial charges are associated with the use of CEVL.

Residents and attending faculty can access CEVL on-site or from home. When logging in, they are greeted with a “Tip of the Day,” a bit of surgical wisdom to supplement the larger curriculum (eg, “Prior to starting the surgery, review the child's pertinent x-ray images with the attending to confirm the side of the operation, as well as the pathology being addressed”). The resident may choose a procedure to study, review his/her evaluation reports, or check CEVL scores for a procedure. Residents are expected to study and practice the steps depicted in the CEVL procedure module. This develops mental imagery of the planned surgery that residents rely on during subsequent performance of the surgery. Available information also includes attending physicians' rating of residents' performance, detailed feedback on cases, and structured remediation strategies for each component.

Attendings may access the online skills assessment of residents with whom they have worked to identify strengths and areas for improvement. This permits the attending to focus his/her next encounter with the resident on working to improve problem areas (H. J. Silberstein, MD. communicated, March 2009).

Each component in a resident's performance of a procedure is rated using a Likert-type scale, addressing specific elements of a procedure that are necessary for a resident's improvement and allowing attendings to structure feedback and remediation. The resident can then focus improvement on specified component skills in preparation for the next procedure (D. R. Gosset, communication, February 2009).

The CEVL curriculum is developed collaboratively between attendings and residents of the residency program, along with technical support from the CEVL staff. Residents pick a procedure for CEVL development and identify elements of a procedure that they find problematic or confusing (eg, how to find the hernia during an orchiopexy). Attending surgeons develop a standardized script (components and steps) for CEVL training on that procedure and gather images and video for curriculum content. The script and content are melded into a cohesive comprehensive module. The content of the module is presented via the CEVL dashboard. Residents and attendings then collaborate on final edits and prepare the lesson for alpha testing by the next rotation. This ensures that CEVL is continually revised and adapted to meet residents' needs. The module is tested and retested until satisfaction is attained by the PD. The module is offered to a limited number of other participating PDs for beta testing.

Initial results show that residents improved their CEVL scores. Seven attending surgeons rated 24 residents on 166 pediatric orchiopexies performed in 2006–2007. CEVL scores improved in 96% of residents over an average of 7 cases.2 Our institutional experience shows that residents and attending physicians have found the CEVL method to be a practical way to teach and learn surgery. Residents are especially pleased with the reduced uncertainty about how to perform procedures and express increased confidence during surgery. They believe that their performance is improved by methodically studying specific components of a procedure, as well as by practicing and rehearsing the specific steps involved. Archived feedback and remediation provide a clear means to improve performance at subsequent operative opportunities. Access to the surgical preferences of individual attendings before surgery familiarizes residents with attending physicians' idiosyncrasies, which further lessens apprehension in the operating room. Perhaps most important, residents appreciate that the CEVL method floats the focus of the residents' OR experience to emphasize their training.

CEVL also provides a method for assessment of a resident's surgical skills. Surgical programs are increasingly seeking means for evaluating a resident's surgical ability. This checklist method of skills evaluation allows surgical training programs to assess and monitor improvement of technical skills. Attending physicians report that the surgical curriculum platform of CEVL enhances interaction because of clearer communication with residents and reduces uncertainty by preparing residents for procedures. We observed that residents and attendings report increased satisfaction with their overall interaction and communication using the CEVL method.

Although the CEVL method of evaluation has not yet been validated as an assessment tool, recent ongoing multi-institutional research demonstrates that 3 institutions using CEVL as a teaching tool for pediatric orchiopexy showed comparable improvement in residents' skills. From this experience, we conclude that the use of CEVL is generalizable across institutions.5 Further studies to evaluate CEVL are in process. They include comparison with standard global assessments of training that are commonly used in surgical residents' evaluations. In addition, the CEVL assessment tool is being examined for reliability and validity. Initial reliability investigations show that there were no substantial differences (by 1-way ANOVA) in rating skills performance using the CEVL method among 7 raters.2 

The CEVL method is a platform to teach psychomotor skills that began with pediatric urologic procedures. We are working with other specialties within our pediatric hospital to teach basic and advanced specialty skills such as gastrostomy, eye muscle surgery, and casting and fracture repair. CEVL is also being developed to teach pediatric residents basic office examination skills such as how to examine a boy for an undescended testis or how to reduce a phimotic foreskin. Pediatric radiology training modules will demonstrate reduction of an intussusception using fluoroscopy. With Chicago's Prentice Women's Hospital, we are developing CEVL modules to assess residents' comfort with safely performing newborn circumcision (D. R. Gossett, communication, February 2009). At a national level the CEVL method is being presented as an educative innovation6 and is being used to further training in endoscopic management of posterior urethral valve.7 More recently, the use of the CEVL method has been explored for teaching Chiari 1 decompression (H.J. Silberstein, MD, communication, 3/2009) and laparoscopic nephrectomy (R. G. Moore, MD, communication, March 2009).

Our institutional experience shows that CEVL has been an effective method to teach and learn surgery by enabling residents to better engage in practicing surgical procedures in the current landscape of reduced duty hours. This has been done using Internet-based content on how to perform surgical procedures and the study of remediation strategies. CEVL is being developed for additional pediatric and adult training programs. The flexibility inherent in the CEVL program permits other institutions to use this method for teaching and assessment in their surgical programs.