Even in the best of times, interview season can be daunting for both program leadership and prospective applicants. In 2020, amid the COVID-19 pandemic, national and specialty level organizations called for virtual-only interview processes for the 2021 graduate medical education (GME) selection period. This marks a landmark change in the way GME selection will occur and may irrevocably change the process going forward. In this article, we explore the recent literature from medical education, social sciences, and business fields to inform residency interview practices, with special attention on digital innovations that may replace or augment traditional interview day processes.

Residency programs devote an immense amount of time, effort, and resources to residency selection and recruitment. Information gleaned from a typical interview day is one of the most important factors to program directors when determining applicant ranking.1  The interview experience also highly influences applicant decision-making.2,3 

The traditional interview process presents many challenges for applicants and residency programs. Program interview dates often do not take into account the interview dates of other programs. This lack of coordination makes medical students' travel planning difficult and often forces applicants to choose between program interviews. Other drawbacks to the current process include the financial burden and loss of time from medical school training.4,5  According to a national survey of more than 1000 graduating medical students, the average number of in-person interviews for applicants was 12.3, and the cost ranged from $1,000 to $5,000 for travel, lodging, and other expenses.6  The financial burden was greater for those applying to more competitive specialties and for those in the couples match.

Pre-COVID-19, some improvements to the interview system were undertaken. Online scheduling services give applicants and programs the ability to schedule, reschedule, and coordinate interviews independently. Some programs allow students to interview if they are participating in a visiting rotation rather than requiring a second visit for the formal interview. Other programs have attempted geographic coordination in which programs in close proximity organize their interview dates in succession. In 1994, the Canadian urology residency training programs designed a “residency fair”: a single site was used for a 1-day in-person opportunity to interview at multiple programs.7  These initiatives also reduce the environmental impact of thousands of miles of travel annually.

Although virtual interviewing holds promise to address many of the problems of the current system, little is known about the effects of a universal virtual interview process for GME. In May 2020, the Coalition for Physician Accountability recommended that all GME interviews be conducted online for the 2020–2021 academic year. These mandated changes present an unusual opportunity for residency selection reform.

Rather than adapting existing processes to digital platforms, program directors designing virtual interview processes should determine the goals of the interview process, and then intentionally design virtual experiences to achieve those goals. By performing a local needs assessment, programs can identify areas where previous processes may have gaps. These gaps may occur in gathering, organizing, and sorting information regarding applicants; effectively highlighting program strengths and unique characteristics; or ensuring that the interview process is efficient and equitable. To harness the strengths of technology, specific considerations include incorporating asynchronous longitudinal experiences, increasing interactive elements, selecting a format for formal interviews, and reenvisioning the program “brand.”

Incorporating Asynchronous Longitudinal Processes

Traditional components of the typical interview day have included a tour of the facilities, the program overview, faculty interviews, and a question-and-answer session with the residents that often occurs over a meal.4  Without the constraints of a time-bound in-person interview process, the interview experience may occur asynchronously and longitudinally. The separate elements of the traditional interview day may be spaced and delivered using a variety of media in order to increase knowledge transfer, make the program more memorable, and allow for a more holistic representation of the program and applicant. Program information can be conveyed prior to interviews with accurate, informative electronic manuals, videos of educational offerings, and typical faculty-resident interactions. Digital facility tours can ensure interviewees become familiar with personnel, academic and clinical opportunities, and culture. The program welcome, whether prerecorded or live streamed, should include an orientation to the virtual interview process in addition to any highlights about the program.8  With program orientation occurring in advance, interviews may focus less on delivering information and more on assessing applicant strengths and characteristics. Virtual “second look” visits may reinforce key program features and allow applicants opportunities to ask additional questions prior to rank list formulation. Previously, second look visits have generated controversy due to applicants' perceived pressure to attend one, despite the financial and logistical burden of travel.911  However, virtual second looks may provide similar benefits without the same challenges.

Increasing Interactive Elements

Opportunities for applicants and program faculty and residents to engage with one another have played an important role in rank list decision-making for both applicants and program directors, and must be established for the virtual interview process as well.1,3,6  Increasing interactivity in structured and unstructured virtual interview activities can help applicants assess program culture and personality, and help programs assess important applicant characteristics such as maturity, professionalism, and communication skills.

In the traditional format, informal interactions among applicants, residents, and faculty typically took place over meals. Programs can consider hosting virtual social hours with residents or coffee chats with the program director and other program leadership, either concurrently with the formal interview or longitudinally throughout the recruitment season. Breakout functions can allow for smaller group interactions. Like a good dinner party, grouping people with similar interests, when possible, may lead to richer discussions and a heightened experience for everyone. Creative strategies can be employed to support engagement, such as ice breaker activities, trivia games, or progressive dinners where applicants rotate through different breakout sessions. Options to create uniformity of experiences can include theming the event by topic or type of food. Depending on time zones and funding, it may be possible for a program to arrange multiple food deliveries to participants using meal delivery apps. A system that allows participants to RSVP may be prudent to assess level of interest and ensure manageable numbers so that meaningful interaction is possible. Programs can consider incorporating other interactive opportunities for applicants using social media, virtual community of practice platforms (eg, WhatsApp or Slack),12  or blogs13  throughout the interview time period.14 

Selecting a Format for Formal Interviews

Digital interview platforms may offer logistical advantages over in-person interviews. Programs may choose to conduct interviews without video, in order to minimize the chance of bias. The digital format may also be harnessed to incorporate alternative strategies that have been used to assess interpersonal or communication skills, such as the multiple mini-interview1518  or group interviews, or to increase efficiency with parallel tracks of simultaneous interviews with digitally timed transitions. The same strategies recommended for in-person interviews should be considered for the virtual format: explicit written descriptions of the desired traits in an applicant; standardized questions to every applicant; provision of behavior-specific anchors for rating scales for interviewers; use of a scoring rubric to improve interrater and intra-rater scoring; use of multiple observers rather than a single interviewer; training of interviewers in format, scoring, and unethical and illegal question rules; and interviewer blinded to other application data to minimize bias.19,20 

Reenvisioning Program Branding

As we discussed in our accompanying article on virtual recruitment, developing and conveying the program brand, or the set of associations that define that program and differentiate it from others, is a crucial component of any selection process.13,14  Although this identity will be conveyed in recruitment activities and media, it is important to consider how the interview experience (virtual or otherwise) also creates and conveys brand identity to applicants. Traditional interview processes may have evolved without a specific intention to convey the brand, and the transition to virtual interviews provides an opportunity for programs to explicitly consider their mission, vision, and values, and to develop a brand identity. Digital or virtual reality tours can overcome barriers of time and space to highlight geographically distant clinical sites, facilities, and aspects of the city and region, such as neighborhoods, restaurants, parks, and landmarks, although these techniques are not without limitations.21 

The supplemental material of this article outlines strategies for implementing virtual interview processes and provides an overview of key considerations for virtual interviews.

Recognizing that changes in the application process may heighten anxiety for prospective applicants, programs should provide clear guidance to interviewees. Programs may provide applicants with an overview of the steps needed during the interview experience as well as concrete suggestions for how to navigate the virtual process. A sample applicant checklist is presented in box 1, which may be adapted to provide the same content to program interviewers. Box 2 contains a checklist for programs regarding technology considerations.

box 1Sample Applicant Checklist

Pre-interview preparation

  • Send the program coordinator any updates to your application prior to the interview day. Create a brief bio for distribution to interviewers.

  • Dress code: review program material or contact the program to determine what the dress code is for the interview (eg, business attire, business casual, etc).

  • Download the platform that will be used for interviews ahead of time, and practice with it to become familiar with the features.

  • Determine a location to conduct the virtual interview that is quiet, free from visual distractions, and with reliable internet access and adequate light. A few pieces of furniture will improve acoustics. Raise the device you will use to allow the camera to be level with your eyes or higher. Ensure a light source is directed toward you, from behind the camera.

  • Schedule a practice session to allow for technological questions.

  • Prepare for the interview as you would for an in-person visit. Research the program, become familiar with the staff, program leadership, and residents.

Interview logistics to clarify in advance of interview

  • Clarify that video should be on during the interview.

  • Clarify expectations around use of the mute button, virtual background use, and the chat function while interviewing.

  • Consider that interviews may be recorded by programs and clarify expectations regarding applicants own recording, taking screenshots, or distributing electronic materials.

  • Audio/visual: camera and microphone

    • Make eye contact with the camera during the interview, unless programs specify an alternative camera location.

    • Follow program preferences, if provided, regarding microphone use (internal or external to computer or device), cameras (internal/external; orientation if phone is used), and headphones.

During the interview

  • Turn off pager, cell phone, and notifications on your device. Close all applications you will not be using.

  • Minimize distractions. If possible, ensure that pets and people in the space will not enter the interview space during the interview. Close doors and windows to reduce background noise and arrange window coverings to minimize distractions.

  • Ensure that you have access to water, tissues, and notes if needed. Try to avoid verbal and physical habits, such as verbal tics and fidgeting with hair, glasses, or swiveling in the chair. Sit upright as you would if you were interviewing in person.

  • Make sure the username that appears on the interview platform is accurate and professional.

box 2Sample Virtual Interview Technology Checklist for Programs

General

  • Send the platform used for interviews ahead of time so the applicant can prepare.

  • Offer a few opportunities for applicants to attend technology checks with the program staff prior to interview day.

  • Send a calendar invite with summarized information and links.

  • Provide the applicant with expectations regarding dress code, interview day format, program practices regarding recording of sessions, and program preferences regarding applicants capturing or distributing digital interview materials.

Audio and visual

  • Specify camera quality recommendations and microphone expectations.

  • Provide ground rules regarding the use of the mute button, virtual background use, and the chat function while interviewing.

Browser and computer

  • Specify browser and application requirements (eg, fully updated version of Google Chrome or Firefox)

  • Operating system required

  • RAM required (eg, 4GB+ of RAM)

  • Hard drive space needed

  • Software required (eg, Zoom, Skype, Slack, Cisco WebEx, Google Meet, Facebook Messenger, etc)

Internet

  • Interviewees and interviewers should have a hard-wired internet connection when possible; if relying on wi-fi, make sure the bandwidth is strong, as the audio may warble or drop out completely if weak.

  • Ensure the internet speed is fast enough to record without stress.

  • You can run a speed test at https://tools.pingdom.com/.

  • If you have up/down speeds that are under 1.5 Mbps, this can cause connection issues. You want at least 1.5 Mbps up/down and hopefully 5 Mbps or more.

Programs may find that communicating culture via virtual interviews is difficult. The digital format may also affect an interviewer's ability to connect with applicants. If aspects of the interview day that are important to applicants are not explicitly included virtual processes, such as portraying interactions among residents, faculty, and staff, applicants will not experience this key aspect of a program. Noncognitive and technical skills may be more difficult to assess over video. “Video fatigue” may become a problem as applicants spend prolonged periods of time in front of device screens. Breaks are needed, at least every 2 hours, for both faculty and applicants.

This transformation to virtual interviews may allow us to reconsider how our present systems perpetuate sociocultural biases. Race, gender, ethnicity, skin color, body habitus, and socioeconomic status have been shown to have profound effects on job selection.2225  In the current social climate, it is incumbent on program leaders to consider their own processes to minimize bias—both at a personal level for their interviewers, but also at a systemic level within the systems we use. Rather than rinsing and repeating our procedures from years past, programs can view the COVID-19 era as an opportunity to redesign interviews for the better.

Virtual processes may offer logistical advantages in organizing and scheduling interviews, and may reduce time, effort, and financial burden, but will ultimately need to be balanced against the difficult-to-replace personal aspects that in-person experiences offer. Programs should consider incorporating asynchronous longitudinal experiences, increasing interactive elements, selecting a format for formal interviews, and reenvisioning program branding while designing virtual processes that both address gaps of the traditional system and future program and applicant needs in the post-COVID-19 era.

1
National Resident Matching Program
.
Results of the 2018 NRMP Program Director Survey
. ,
2020
.
2
Schmidt
LE,
Cooper
CA,
Guo
WA.
Factors influencing US medical students' decision to pursue surgery
.
J Surg Res
.
2016
;
203
(
1
):
64
74
. doi:.
3
Love
JN,
Howell
JM,
Hegarty
CB,
McLaughlin
SA,
Coates
WC,
Hopson
LR,
et al.
Factors that influence medical student selection of an emergency medicine residency program: implications for training programs
.
Acad Emerg Med
.
2012
;
19
(
4
):
455
460
. doi:.
4
Shah
SK,
Arora
S,
Skipper
B,
Kalishman
S,
Timm
TC,
Smith
AY.
Randomized evaluation of a web based interview process for urology resident selection
.
J Urol
.
2012
;
187
(
4
):
1380
1384
. doi:.
5
Brummond
A,
Sefcik
S,
Halvorsen
AJ,
Chaudhry
S,
Arora
V,
Adams
M,
et al.
Resident recruitment costs: a national survey of internal medicine program directors
.
Am J Med
.
2013
;
126
(
7
):
646
653
. doi:.
6
Association of American Medical Colleges
.
Fried
JG.
Cost of Applying to Residency Questionnaire Report
. ,
2020
.
7
Grober
ED,
Matsumoto
ED,
Jewett
MAS,
Chin
JL.
The Canadian Urology Fair: a model for minimizing the financial and academic costs of the residency selection process
.
Can J Surg
.
2003
;
46
(
6
):
458
462
.
8
Association of American Medical Colleges
.
Virtual Interviews: Tips for Interviewers
. ,
2020
.
9
Nagarkar
PA,
Janis
JE.
Fixing the Match: a survey of resident behaviors
.
Plast Reconstr Surg
.
2013
;
132
(
3
):
711
719
. doi:.
10
Sebesta
EM,
Lipsky
MJ,
Nunez
M,
Cooper
KL,
Badalato
GM.
The National Resident Matching Program code of conduct: what is the perceived degree of compliance during the urology match process?
Urology
.
2018
;
122
:
37
43
. doi:.
11
Nikonow
TN,
Lyon
TD,
Jackman
SV,
Averch
TD.
Survey of applicant experience and cost in the urology Match: opportunities for reform
.
J Urol
.
2015
;
194
(
4
):
1063
1067
. doi:.
12
Sarkisian
SA,
Kagel
A.
The use of Slack for medical residency development and recruiting
.
BMJ Health Care Inform
.
2018
;
25
(
3
):
194
195
. doi:.
13
Shappell
E,
Shakeri
N,
Fant
A,
Branzetti
J,
Gisondi
M,
Babcock
C,
et al.
Branding and recruitment: a primer for residency program leadership
.
J Grad Med Educ
.
2018
;
10
(
3
):
249
252
. doi:.
14
Haas
MRC,
He
S,
Sternberg
K,
Jordan
J,
Deiorio
NM,
Chan
TM,
et al.
Reimagining residency selection: part 1—a practical guide to recruitment in the post-COVID-19 era
.
J Grad Med Educ
.
2020
;
12
(
5
):
xxx
xxx
.
15
Dore
KL,
Kreuger
S,
Ladhani
M,
Rolfson
S,
Kurtz
D,
Kulasegaram
K,
et al.
The reliability and acceptability of the multiple mini-interview as a selection instrument for postgraduate admissions
.
Acad Med
.
2010
;
85
(
10 suppl
):
60
63
. doi:.
16
Eva
KW,
Rosenfeld
J,
Reiter
HI,
Norman
GR.
An admissions OSCE: the multiple mini-interview
.
Med Educ
.
2004
;
38
(
3
):
314
326
. doi:.
17
Joshi
NK,
Yarris
LM,
Doty
CI,
Lin
M.
Social media responses to the Annals of Emergency Medicine residents' perspective article on multiple mini-interviews
.
Ann Emerg Med
.
2014
;
64
(
3
):
320
5
. doi:.
18
Phillips
AW,
Garmel
GM.
Does the multiple mini-interview address stakeholder needs? An applicant's perspective
.
Ann Emerg Med
.
2014
;
64
(
3
):
316
319
. doi:.
19
Stephenson-Famy
A,
Houmard
BS,
Oberoi
S,
Manyak
A,
Chiang
S,
Kim
S.
Use of the interview in resident candidate selection: a review of the literature
.
J Grad Med Educ
.
2015
;
7
(
4
):
539
548
. doi:.
20
Gordon
MJ,
Lincoln
JA.
Family practice resident selection: value of the interview
.
J Fam Pr
.
1976
;
3
(
2
):
175
177
.
21
Zertuche
J-P,
Connors
J,
Scheinman
A,
Kothari
N,
Wong
K.
Using virtual reality as a replacement for hospital tours during residency interviews
.
Med Educ Online
.
2020
;
25
(
1
):
1777066
. doi:.
22
Kutcher
EJ,
Bragger
JD.
Selection interviews of overweight job applicants: can structure reduce the bias?
J Appl Soc Psychol
.
2004
;
34
(
10
):
1993
2022
. doi:.
23
Harrison
MS,
Thomas
KM.
The hidden prejudice in selection: a research investigation on skin color bias
.
J Appl Soc Psychol
.
2009
;
39
(
1
):
134
168
. doi:.
24
Edmond
MB,
Deschenes
JL,
Eckler
M,
Wenzel
RP.
Racial bias in using USMLE Step 1 Scores to grant internal medicine residency interviews
.
Acad Med
.
2001
;
76
(
12
):
1253
1256
. doi:.
25
Desbiens
NA,
Vidaillet
HJ.
Discrimination against international medical graduates in the United States residency program selection process
.
BMC Med Educ
.
2010
;
10
(
1
):
5
. doi:.

Author notes

Editor's Note: The online version of this article contains strategies for implementing virtual interview processes and an overview of key considerations for virtual interviews.

Supplementary data