Even in the best of times, interview season can be daunting for both program leadership and prospective applicants. In 2020, amid the COVID-19 pandemic, national and specialty level organizations called for virtual-only interview processes for the 2021 graduate medical education (GME) selection period. This marks a landmark change in the way GME selection will occur and may irrevocably change the process going forward. In this article, we explore the recent literature from medical education, social sciences, and business fields to inform residency interview practices, with special attention on digital innovations that may replace or augment traditional interview day processes.
Interviewing: Before and After COVID-19
Residency programs devote an immense amount of time, effort, and resources to residency selection and recruitment. Information gleaned from a typical interview day is one of the most important factors to program directors when determining applicant ranking.1 The interview experience also highly influences applicant decision-making.2,3
The traditional interview process presents many challenges for applicants and residency programs. Program interview dates often do not take into account the interview dates of other programs. This lack of coordination makes medical students' travel planning difficult and often forces applicants to choose between program interviews. Other drawbacks to the current process include the financial burden and loss of time from medical school training.4,5 According to a national survey of more than 1000 graduating medical students, the average number of in-person interviews for applicants was 12.3, and the cost ranged from $1,000 to $5,000 for travel, lodging, and other expenses.6 The financial burden was greater for those applying to more competitive specialties and for those in the couples match.
Pre-COVID-19, some improvements to the interview system were undertaken. Online scheduling services give applicants and programs the ability to schedule, reschedule, and coordinate interviews independently. Some programs allow students to interview if they are participating in a visiting rotation rather than requiring a second visit for the formal interview. Other programs have attempted geographic coordination in which programs in close proximity organize their interview dates in succession. In 1994, the Canadian urology residency training programs designed a “residency fair”: a single site was used for a 1-day in-person opportunity to interview at multiple programs.7 These initiatives also reduce the environmental impact of thousands of miles of travel annually.
Although virtual interviewing holds promise to address many of the problems of the current system, little is known about the effects of a universal virtual interview process for GME. In May 2020, the Coalition for Physician Accountability recommended that all GME interviews be conducted online for the 2020–2021 academic year. These mandated changes present an unusual opportunity for residency selection reform.
Considerations for Virtual Interviews
Rather than adapting existing processes to digital platforms, program directors designing virtual interview processes should determine the goals of the interview process, and then intentionally design virtual experiences to achieve those goals. By performing a local needs assessment, programs can identify areas where previous processes may have gaps. These gaps may occur in gathering, organizing, and sorting information regarding applicants; effectively highlighting program strengths and unique characteristics; or ensuring that the interview process is efficient and equitable. To harness the strengths of technology, specific considerations include incorporating asynchronous longitudinal experiences, increasing interactive elements, selecting a format for formal interviews, and reenvisioning the program “brand.”
Incorporating Asynchronous Longitudinal Processes
Traditional components of the typical interview day have included a tour of the facilities, the program overview, faculty interviews, and a question-and-answer session with the residents that often occurs over a meal.4 Without the constraints of a time-bound in-person interview process, the interview experience may occur asynchronously and longitudinally. The separate elements of the traditional interview day may be spaced and delivered using a variety of media in order to increase knowledge transfer, make the program more memorable, and allow for a more holistic representation of the program and applicant. Program information can be conveyed prior to interviews with accurate, informative electronic manuals, videos of educational offerings, and typical faculty-resident interactions. Digital facility tours can ensure interviewees become familiar with personnel, academic and clinical opportunities, and culture. The program welcome, whether prerecorded or live streamed, should include an orientation to the virtual interview process in addition to any highlights about the program.8 With program orientation occurring in advance, interviews may focus less on delivering information and more on assessing applicant strengths and characteristics. Virtual “second look” visits may reinforce key program features and allow applicants opportunities to ask additional questions prior to rank list formulation. Previously, second look visits have generated controversy due to applicants' perceived pressure to attend one, despite the financial and logistical burden of travel.9–11 However, virtual second looks may provide similar benefits without the same challenges.
Increasing Interactive Elements
Opportunities for applicants and program faculty and residents to engage with one another have played an important role in rank list decision-making for both applicants and program directors, and must be established for the virtual interview process as well.1,3,6 Increasing interactivity in structured and unstructured virtual interview activities can help applicants assess program culture and personality, and help programs assess important applicant characteristics such as maturity, professionalism, and communication skills.
In the traditional format, informal interactions among applicants, residents, and faculty typically took place over meals. Programs can consider hosting virtual social hours with residents or coffee chats with the program director and other program leadership, either concurrently with the formal interview or longitudinally throughout the recruitment season. Breakout functions can allow for smaller group interactions. Like a good dinner party, grouping people with similar interests, when possible, may lead to richer discussions and a heightened experience for everyone. Creative strategies can be employed to support engagement, such as ice breaker activities, trivia games, or progressive dinners where applicants rotate through different breakout sessions. Options to create uniformity of experiences can include theming the event by topic or type of food. Depending on time zones and funding, it may be possible for a program to arrange multiple food deliveries to participants using meal delivery apps. A system that allows participants to RSVP may be prudent to assess level of interest and ensure manageable numbers so that meaningful interaction is possible. Programs can consider incorporating other interactive opportunities for applicants using social media, virtual community of practice platforms (eg, WhatsApp or Slack),12 or blogs13 throughout the interview time period.14
Selecting a Format for Formal Interviews
Digital interview platforms may offer logistical advantages over in-person interviews. Programs may choose to conduct interviews without video, in order to minimize the chance of bias. The digital format may also be harnessed to incorporate alternative strategies that have been used to assess interpersonal or communication skills, such as the multiple mini-interview15–18 or group interviews, or to increase efficiency with parallel tracks of simultaneous interviews with digitally timed transitions. The same strategies recommended for in-person interviews should be considered for the virtual format: explicit written descriptions of the desired traits in an applicant; standardized questions to every applicant; provision of behavior-specific anchors for rating scales for interviewers; use of a scoring rubric to improve interrater and intra-rater scoring; use of multiple observers rather than a single interviewer; training of interviewers in format, scoring, and unethical and illegal question rules; and interviewer blinded to other application data to minimize bias.19,20
Reenvisioning Program Branding
As we discussed in our accompanying article on virtual recruitment, developing and conveying the program brand, or the set of associations that define that program and differentiate it from others, is a crucial component of any selection process.13,14 Although this identity will be conveyed in recruitment activities and media, it is important to consider how the interview experience (virtual or otherwise) also creates and conveys brand identity to applicants. Traditional interview processes may have evolved without a specific intention to convey the brand, and the transition to virtual interviews provides an opportunity for programs to explicitly consider their mission, vision, and values, and to develop a brand identity. Digital or virtual reality tours can overcome barriers of time and space to highlight geographically distant clinical sites, facilities, and aspects of the city and region, such as neighborhoods, restaurants, parks, and landmarks, although these techniques are not without limitations.21
The supplemental material of this article outlines strategies for implementing virtual interview processes and provides an overview of key considerations for virtual interviews.
Recognizing that changes in the application process may heighten anxiety for prospective applicants, programs should provide clear guidance to interviewees. Programs may provide applicants with an overview of the steps needed during the interview experience as well as concrete suggestions for how to navigate the virtual process. A sample applicant checklist is presented in box 1, which may be adapted to provide the same content to program interviewers. Box 2 contains a checklist for programs regarding technology considerations.
box 1 Sample Applicant Checklist
Pre-interview preparation
Send the program coordinator any updates to your application prior to the interview day. Create a brief bio for distribution to interviewers.
Dress code: review program material or contact the program to determine what the dress code is for the interview (eg, business attire, business casual, etc).
Download the platform that will be used for interviews ahead of time, and practice with it to become familiar with the features.
Determine a location to conduct the virtual interview that is quiet, free from visual distractions, and with reliable internet access and adequate light. A few pieces of furniture will improve acoustics. Raise the device you will use to allow the camera to be level with your eyes or higher. Ensure a light source is directed toward you, from behind the camera.
Schedule a practice session to allow for technological questions.
Prepare for the interview as you would for an in-person visit. Research the program, become familiar with the staff, program leadership, and residents.
Interview logistics to clarify in advance of interview
Clarify that video should be on during the interview.
Clarify expectations around use of the mute button, virtual background use, and the chat function while interviewing.
Consider that interviews may be recorded by programs and clarify expectations regarding applicants own recording, taking screenshots, or distributing electronic materials.
Audio/visual: camera and microphone
- ○
Make eye contact with the camera during the interview, unless programs specify an alternative camera location.
- ○
Follow program preferences, if provided, regarding microphone use (internal or external to computer or device), cameras (internal/external; orientation if phone is used), and headphones.
- ○
During the interview
Turn off pager, cell phone, and notifications on your device. Close all applications you will not be using.
Minimize distractions. If possible, ensure that pets and people in the space will not enter the interview space during the interview. Close doors and windows to reduce background noise and arrange window coverings to minimize distractions.
Ensure that you have access to water, tissues, and notes if needed. Try to avoid verbal and physical habits, such as verbal tics and fidgeting with hair, glasses, or swiveling in the chair. Sit upright as you would if you were interviewing in person.
Make sure the username that appears on the interview platform is accurate and professional.
box 2 Sample Virtual Interview Technology Checklist for Programs
General
Send the platform used for interviews ahead of time so the applicant can prepare.
Offer a few opportunities for applicants to attend technology checks with the program staff prior to interview day.
Send a calendar invite with summarized information and links.
Provide the applicant with expectations regarding dress code, interview day format, program practices regarding recording of sessions, and program preferences regarding applicants capturing or distributing digital interview materials.
Audio and visual
Specify camera quality recommendations and microphone expectations.
Provide ground rules regarding the use of the mute button, virtual background use, and the chat function while interviewing.
Browser and computer
Specify browser and application requirements (eg, fully updated version of Google Chrome or Firefox)
Operating system required
RAM required (eg, 4GB+ of RAM)
Hard drive space needed
Software required (eg, Zoom, Skype, Slack, Cisco WebEx, Google Meet, Facebook Messenger, etc)
Internet
Interviewees and interviewers should have a hard-wired internet connection when possible; if relying on wi-fi, make sure the bandwidth is strong, as the audio may warble or drop out completely if weak.
Ensure the internet speed is fast enough to record without stress.
You can run a speed test at https://tools.pingdom.com/.
If you have up/down speeds that are under 1.5 Mbps, this can cause connection issues. You want at least 1.5 Mbps up/down and hopefully 5 Mbps or more.
Potential Disadvantages to Virtual Interviews
Programs may find that communicating culture via virtual interviews is difficult. The digital format may also affect an interviewer's ability to connect with applicants. If aspects of the interview day that are important to applicants are not explicitly included virtual processes, such as portraying interactions among residents, faculty, and staff, applicants will not experience this key aspect of a program. Noncognitive and technical skills may be more difficult to assess over video. “Video fatigue” may become a problem as applicants spend prolonged periods of time in front of device screens. Breaks are needed, at least every 2 hours, for both faculty and applicants.
Conclusions
This transformation to virtual interviews may allow us to reconsider how our present systems perpetuate sociocultural biases. Race, gender, ethnicity, skin color, body habitus, and socioeconomic status have been shown to have profound effects on job selection.22–25 In the current social climate, it is incumbent on program leaders to consider their own processes to minimize bias—both at a personal level for their interviewers, but also at a systemic level within the systems we use. Rather than rinsing and repeating our procedures from years past, programs can view the COVID-19 era as an opportunity to redesign interviews for the better.
Virtual processes may offer logistical advantages in organizing and scheduling interviews, and may reduce time, effort, and financial burden, but will ultimately need to be balanced against the difficult-to-replace personal aspects that in-person experiences offer. Programs should consider incorporating asynchronous longitudinal experiences, increasing interactive elements, selecting a format for formal interviews, and reenvisioning program branding while designing virtual processes that both address gaps of the traditional system and future program and applicant needs in the post-COVID-19 era.
References
Author notes
Editor's Note: The online version of this article contains strategies for implementing virtual interview processes and an overview of key considerations for virtual interviews.