Residency and Fellowship Program Leaders' Perceptions of Virtual Recruitment and Interviewing

Graduate medical education (GME) rapidly adopted virtual recruitment and interviewing in 2020 in response to guidance from the Association of American Medical Colleges. While more than half of program directors intend to continue with virtual recruitment in some form, many worry about impediments to nuanced assessments of applicants (such as assessment of applicant interpersonal skills and interest in a particular program) as well as the overall success of recruitment with virtual platforms.^1,2  These fears may be particularly pronounced for programs that are smaller or lack the name recognition of a university affiliation. Programs that rely on in-person visits to showcase physical aspects of their program or community may be disadvantaged as well.³  Further research from a greater breadth of training programs is needed to ascertain the trade-offs with virtual formats given their known cost saving benefits. The purpose of this study was to evaluate virtual match perceptions and preferred format for future recruitment from a breadth of diverse residency and fellowship programs.

In 2021, we surveyed residency and fellowship program directors representing 136 Accreditation Council for Graduate Medical Education–accredited academic and community programs affiliated with the University of Pittsburgh Medical Center (UPMC) at 11 separate hospitals across urban and rural areas in Pennsylvania. Surveys assessed perceived match outcomes for the 2021-2022 academic year and plans for recruitment in the upcoming 2022 match cycle.

UPMC committed to virtual recruiting and interviewing for all residency and fellowship positions beginning in the spring of 2020, consistent with recommendations from the Coalition for Physician Accountability.⁴  Specifics of the interview/recruitment process were left to individual program directors. To support programs, the central GME office provided multiple educational sessions about using online interviewing and recruiting. Microsoft Teams was our preferred platform for interviewing. Research questions were included in an annual survey of program directors and coordinators affiliated with the sponsoring institution. The survey was administered in April 2021 following National Resident Matching Program Main Match results but prior to the arrival of matched applicants in the summer of 2021. Questions were to be answered jointly by program director and coordinator and submitted as a single response. The survey included 40 items pertaining to virtual recruitment and interviewing, which were developed through consensus by a group of 7 program directors, the designated institutional official (DIO), and 3 associate DIOs. They were pilot-tested and refined for clarity. Most questions utilized a 5-point Likert scale for agreement (see online supplementary data for the survey instrument).

We used descriptive statistics to report interview day content and structure. Student's t test was used to compare Likert data according to program type (residency vs fellowship) and logistic regression to understand predictors of preference for the upcoming interview season. A sizable portion of our fellowship programs (18%) recruit outside of a dedicated match. Recognizing these programs are unique, they were excluded from our analyses about program perceptions of virtual recruitment and interview success and logistic regression analysis. Hypothesized predictor variables included program type (residency vs fellowship), size of the incoming class, primary teaching site (specialty/referral hospital compared to a community-based hospital), number of faculty who interviewed each applicant, and whether trainees were involved in interviewing. Regression analysis was also used to examine program directors' perceived success in recruiting a diverse class, perceived ability to describe the strengths of programs through virtual formats, and association with future format preferences. All statistical analyses were performed using R, version 4.0.3 (The R Foundation for Statistical Computing). This analysis was considered exempt by the University of Pittsburgh Institutional Review Board.

Out of 136 programs, 129 (94.8%) responded. Program recruitment characteristics are provided in Table 1. Programs' perceptions of virtual interviewing and recruiting are shown in Table 2. Opinions were similar when analyzed according to program type, with the exception of perceptions of faculty time investment (residency mean=2.89, SD=0.85 vs fellowship mean=2.50, SD=0.86, P=.02). Programs noted virtual interviewing and recruiting took slightly less time investment from faculty and current trainees. Virtual interviewing and recruiting were less expensive than in-person recruitment, though we did not track discrete dollar amounts. Programs felt virtual recruiting marginally decreased their ability to describe the strength of their program (mean=2.67, SD=0.78); however, programs did not notice any meaningful change in the diversity or perceived strength of their recruited class (mean=3.10, SD=0.69, and mean=3.09, SD=1.34, respectively).

Overall preferences for continuing virtual recruiting vs in-person recruiting in the future were neutral (mean=3.09, SD=1.34). However, this reflects heterogeneity of opinion. Fourteen percent (14 of 103) had a strong preference for in-person interviewing, 31% (26 of 103) had a slight preference for in-person interviewing, 14% (14 of 103) had no preference, 25% (26 of 103) had a slight preference for virtual interviewing, and 16% (17 of 103) had a strong preference for virtual interviewing and recruitment. This mirrors programs' varying preferences for the ideal structure of the upcoming interview season: 31.1% (32 of 103) of programs preferred to offer both formats and let the applicant choose. Approximately double the number of programs would conduct exclusively virtual recruitment (n=21, 20.4%) rather than exclusively in-person recruitment (n=10, 9.7%), and the remainder would choose a combination (n=40, 38.8%).

Using logistic regression, we analyzed predictors of in-person recruitment and found that community sites were 4 times more likely to prefer in-person vs virtual interviewing and recruiting (P=.013, 95% CI 1.34-12.39). There were no other statistically significant predictors of preference.

The dramatic shift to virtual recruiting and interviewing was unexpected and likely to cause enduring changes in how programs recruit physicians. Few studies have analyzed the impact of fully virtual recruitment across a spectrum of GME programs. Our results demonstrate that while program preference on recruitment methods varied substantially, overall, virtual methods were not perceived to be detrimental to recruitment and interviewing outcomes.

Our data suggest that community and university-affiliated programs have different recruitment needs. Community sites are often desirable for the close trainee and faculty connections and practice settings that closely mirror trainees' ultimate desired setting. However, applicants may have limited exposure to community sites during their medical training, as many medical schools are university-based in urban or suburban settings. Consequently, interview experiences and recruitment events may play a more prominent role in informing and matching applicants into community training programs.

Diversity and inclusion recruitment efforts are not perceived to be adversely affected by the virtual format in our population. Though there has been concern about the virtual format amplifying and potentially introducing new sources of bias,⁵  programs' self-reported measures of matched class diversity in 2021 were comparable to prior years. Our sponsoring institution tracks the percentage of matched trainees who meet Association of American Medical Colleges underrepresented in medicine definitions,⁶  and according to internal unpublished data between 2018 and 2021, this number ranged from 9% to 11%, with 11% of matched applicants meeting criteria from the 2020 recruitment season. Though more research is needed to confirm whether an impact exists on diversity and inclusion, virtual recruitment remains an attractive option to decrease the match process's financial burden on applicants.

Limitations of this study include a single center design and that fellowship programs may be overrepresented in this sample. However, this center represents both academic and community programs as well as a diverse spectrum of residency and fellowship programs, which increases generalizability of our findings.

Our data show that while virtual interviewing is less expensive and may use slightly less faculty time, it is less enthusiastically embraced by community-based programs.