The Challenge
The safe performance of procedures is among the most critical aspects confronting surgical resident education. Increasing public concerns about physician performance and patient safety have augmented the need to demonstrate competence in surgical skills.1 Assessment of residents' operative skills should be both formative (feedback to the resident outlining areas for performance improvement) and summative (a statement of the resident's competence and skill level). With progressive experience and competence, advanced responsibility can be assigned to residents in complex operations.
Intraoperative assessment of residents can be particularly challenging. Surgical skills are attained through practice and include a combination of motor skills, judgment, and medical knowledge. Furthermore, most surgical specialties do not have well-defined criteria for surgical competence; the bar for a minimal performance standard has not yet been set. A major goal of the Accreditation Council for Graduate Medical Education (ACGME) milestones is to define which skills should be attained with each year in training. Although operative skills are not explicitly included as 1 of the 6 general competencies, operative assessment will include aspects of all 6 competencies.
What Is Known
Operative assessment can be formative to provide feedback for making the procedure a better educational experience or summative to assess a resident's competence for a particular procedure or skill.2–,6 Written assessment of operative skills has been studied in a variety of surgical fields, including general medicine,7 orthopedic practice,4,6 otolaryngology,8 neurological surgery, and obstetrics and gynecology.9 More effective assessment derives from multiple assessments over time by multiple faculty members. Assessment tools can be general or procedure specific. More consistent feedback will be provided with a structured instrument developed for a specific procedure, such as carpal tunnel release.6 The development of valid and reliable assessment tools is an area of active study by various surgical specialties.1,5,8,10
Feedback (oral and written) about procedural performance is best provided to the resident as soon as possible after the procedure is done. The evaluating faculty member and resident will have a better recall of intraoperative events the same day than they will a week later.10 Use of a written assessment tool encourages the faculty and the resident to focus on areas of demonstrated competence and those areas needing improvement.
A written intraoperative assessment provides a structured format for feedback to the resident. Even with imperfect assessment tools, feedback can be effective. A commonly used technique is to use a preoperative plan, written by the resident before the operative procedure, as a feedback template. Elements of a preoperative plan include the sequential steps of the procedure, a drawing of the anatomy involved with the procedure, and a list of specific equipment needed for the procedure.4,10
The preoperative plan can be used as a structured document to review the planned operative procedure with the resident beforehand. This discussion will ensure that the resident understands the procedure and has the opportunity to ask any last-minute questions. In addition, the preoperative plan can be discussed with nursing staff and be used as a checklist for making sure equipment is available and the operating room is ready to care for the patient.4,10
Intraoperative assessment is performed with an instrument, such as the sample Objective Assessment of Technical Skills tool shown in the 2a1box. It may be used alone with a designated procedure or to complement the preoperative plan. Intraoperative assessments usually assess a resident's technical skills for a designated procedure.
Control of Hemorrhage
Applies pressure to stop bleeding first
Asks assistant to suction field
Inspects injury by carefully releasing the inferior vena cava
Ensures all equipment needed for repair is at hand before starting
Controls bleeding points (uses deBakey forceps, Satinsky clamps, or proximal/distal pressure)
Repair
Select appropriate suture (4.0, 5.0, 6.0 polypropylene)
Select appropriate, needed driver (vascular)
Select appropriate forceps (deBakey)
Needle loaded one-half to two-thirds from tip 90% of time
How Can You Start Today
Review the ACGME requirements for your specialty. Some surgical specialties designate a certain number and specific types of operative cases.
Review the assessment tools already used by your program, highlight common elements, and determine whether the tool is procedure specific.
Review the most common procedures performed by your surgical specialty. The best formative and summative assessments occur with commonly seen procedures, assessed multiple times by multiple assessors. Having sufficient volume is an important aspect for progressive improvement and milestone achievement.
Using information in steps 1 to 3, discuss with faculty which procedures faculty should be required (or a reasonable cross section) to complete an intraoperative assessment of resident performance.
What You Can Do Long Term
Review the literature for assessment tools with some validity evidence for operation assessment. Check with your national specialty organizations for useful assessment tools that may have evidence of validity. It is sometimes best to network with those in your specialty to obtain assessment instruments for the more common procedures.
Adapt or develop an instrument for the common procedures. The best instruments are those that consistently measure the performance objectives, even with differing assessors.
Plan faculty development sessions to review the goals, process, safety, and tools associated with intraoperative assessment of residents.
Develop a remediation plan for those residents determined to need improvement through your assessments. This can include a “surgical mentor,” simulation center training, or remedial skills training.
Plan to assess the reliability and validity of your assessment tool and results. The effort required will vary with the stakes of the particular assessment (instruments for summative evaluation require more evidence of validity than do instruments used for formative feedback).
Review milestones for your specialty or develop milestones that will be evaluated by your assessment instruments.
Resources
Author notes
All authors are at University of Michigan Health System. Paul Dougherty, MD, is Associate Professor and Residency Program Director of Orthopaedic Surgery; Steven J. Kasten, MD, MHPE, FACS, is Associate Professor of Surgery, Program Director of the Integrated Plastic Surgery Residency Program, and Associate Director of the Craniofacial Anomalies Program; R. Kevin Reynolds, MD, is George W. Morley Collegiate Professor of Obstetrics and Gynecology and Professor of Obstetrics and Gynecology; Mark E. P. Prince, MD, is Associate Professor of Otorhinolaryngology; and Monica L. Lypson, MD, is Assistant Dean for Graduate Medical Education and Associate Professor of Internal Medicine and Medical Education at University of Michigan Medical School.