Real-Time Dual Video Conferencing of Simulated Tracheostoma Maturation During Residency Interviews

In the resident selection process, programs seek traits that are predictive of success during residency. In otolaryngology–head and neck surgery (OHNS), academic achievement, Alpha Omega Alpha membership, and team sport participation have been correlated with residency and career success; however, even among these reports, there are contradictory findings and a notable lack of consensus.^1-5  In addition to these factors, various simulations and dexterity tasks have been adopted by many residency programs across specialties. Some studies show mixed results on their predictive power as a residency selection tool,^6,7  while others describe utilizing them to identify outlier candidates who may experience difficulties in a surgical training program.^6-11 

The authors' institution has successfully utilized a microsurgical skills assessment during interviews^6,8  to assess qualities such as dexterity, skill acquisition, and attitude, which may be otherwise underrepresented in written applications and traditional interviews. This activity is used to note outliers in these domains and is considered a part of the global applicant review. With the COVID-19 pandemic, all interviews changed to a virtual format, forcing an adaptation of this practice.

The purpose of this project was to implement a virtual suturing format and determine its feasibility and usefulness to the residency selection committee as a substitute for the prior in-person, microsurgical skills assessment performed during residency interviews.

The present study occurred at a 5-resident-per-year OHNS training program based at a rural tertiary referral center during the 2021-2022 residency interview season.

Prior to the interview dates, the necessary supplies were mailed to all applicants with strict instructions preventing opening prior to their respective interview days (Figure 1, online supplementary data). On the day of the interview, dual-video feed conferencing was set up to see both the candidate's face and the candidate's hands during the exercise (Figure 1). Maturation of a tracheostoma was simulated by instructing candidates to cut a hole in a skin substitute and suture a 3/4-inch Penrose drain circumferentially with half-vertical mattress stitches (Figure 2, online supplementary data). The activity included 5 to 10 minutes for setup and demonstration and 20 minutes to complete the simulation. Applicants were actively assessed and scored by 2 resident assessors in real time during the exercise as is typically performed at the authors' institution using a standardized scoring rubric adapted from prior evaluation tools (online supplementary data).^6,8  Scores were then provided to the selection committee alongside the interview scores from the standard interview rooms composed of faculty members, staff, and additional residents. The standard interview rooms follow a set of themed questions specific to each room with a standardized answer scale developed by the authors to increase objectivity.

After the completion of the residency Match, members of the residency selection committee were surveyed through an electronically disseminated 7-statement author-developed Likert scale survey to further evaluate this exercise in comparison to the microsurgical assessment used previously at the authors' institution. To limit bias, the authors' responses to the survey were not included in the reported survey data.

Findings were reported using descriptive statistics with categorical features summarized with frequency counts and percentages. The study was approved by the Mayo Clinic Institutional Review Board.

The total cost associated with purchasing necessary supplies and mailing them to candidates prior to interview day was $34.78 for each candidate. The packages were shipped at least 1 month in advance of the planned interviews. Over the course of the interview season, all 51 applicants interviewed during the interview cycle received their supplies and completed this exercise successfully without substantial technical issues (Figure 2). After setup and demonstration, the time required to complete the suturing task was estimated to range from 10 to 20 minutes. The survey response rate for the residency selection committee was 100% (16 of 16; online supplementary data). The committee viewed the exercise as a valuable part of the interview day (75%, 12 of 16) and a success overall (87.5%, 14 of 16). Despite differential clinical experience, most of the committee understood the exercise and scoring (75%, 12 of 16). Overall, most surveyed committee members viewed this exercise as a good assessment of dexterity (68.8%, 11 of 16) and handling of stress (93.8%, 15 of 16). The data were considered by almost all committee members in their view of an applicant (93.8%, 15 of 16).

Despite the requirement of virtual interviews secondary to the COVID-19 pandemic, the presented suturing exercise was completed successfully by all interviewees. No technical problems were encountered, and the resultant scores as provided by the resident assessors were utilized in the resident selection process. The residency selection committee viewed this exercise favorably and in line with prior activities performed at the institution.^6,8 

This suture exercise using a dual-video feed enabled assessment of both manual dexterity and candidate attitude during the assigned task at a relatively affordable price when considering the typical costs associated with in-person interviews. Although most of the resident selection committee members used the suturing assessment information in their overall evaluation of the candidates, it was added to a holistic review of the candidate's interview day and application, as done in prior years with the in-person suturing exercise. The residency selection committee used the virtual suturing assessment to identify outliers and facilitate ranking of similarly competitive applicants.

Few reports have described this specific tracheostoma maturation technique as an interview exercise, but novelty may be more important in these motor exercises than the exercise itself. The virtual nature of our simulation exercise appears to function as seamlessly as other described virtual assessments of medical students and residents, such as virtual objective structured clinical examinations.^12-15  This supports continued use of virtual exercises in the recruitment process, particularly as virtual interviewing may continue for the foreseeable future.

This study is limited by the single program and specialty, with small numbers of assessors and applicants, which may impair generalizing to other settings and specialties. This virtual dexterity exercise has not been examined for correlation with performance during residency and thus lacks validity evidence for this purpose. Also, interobserver agreement among the assessors has not been examined, which could affect reliability. The survey questions were not tested; thus, respondents may have interpreted questions differently than intended. Surveys are susceptible to recall bias, which may have affected the selection committee responses in unknown ways.

Future steps in this project include assessing the predictive ability of this suture exercise by comparing scoring on this exercise to later faculty-assessed resident competency, in line with prior work by the authors' institution on this topic.⁶ 

A simple, brief motor exercise completed over real-time telecommunication using a dual-video feed setup was feasible and perceived as helpful to the residency selection committee when assessing OHNS residency candidates during virtual interviews.