Geographic Reach of Surgery Residency Applicants During In-Person and Virtual Interviews

In 2020 US residency programs rapidly restructured interviews to be virtual-only.^1,2  This change enabled assessment of the effects of virtual interviews on applications from geographic locations farther from residency programs: a program’s geographic reach (GR). Increasing GR, through eliminating the cost and lost time of travel, may affect applicant diversity as well.³  Decreased costs and scheduling flexibility associated with virtual interviews might increase applications from lower socioeconomic backgrounds.⁴ 

Multiple strategies to increase diversity in surgical specialties have been implemented, including outreach to applicants through underrepresented in medicine (UIM)-focused clerkships, UIM-focused mentorship networks, revision of screening criteria and interview structure to minimize bias, and analysis of recruitment data to keep track of projected recruitment goals.^5-7  Despite these efforts, studies show no change in recruitment of UIM applicants in US surgical specialties over the last decade.^8,9  Since the COVID-19 pandemic shift to virtual interviews, studies have examined increases in interviews for programs and applicants, but little has been published regarding effects on GR, diversity, and lower-income locations of applicants.¹⁰  Given the importance of increasing diversity and backgrounds for many specialties, these additional effects need to be considered.^8,9,11 

We hypothesized that virtual interviews would increase surgery residency program GR, with greater increases for UIM applicants and those from areas identified as lower income.

This was a retrospective cohort study of the 2019 and 2020 general surgery residency application cycles at the Beth Israel Deaconess Medical Center (BIDMC). The BIDMC categorical surgery residency is a large, 45 categorical resident, urban, university-based program in northeastern United States. All categorical surgery residency applicants, except international medical graduates, were included. The following data were obtained from application files: age, gender (male or female), race/ethnicity (self-identified), United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge (CK) scores, and applicant’s zip code. UIM was defined as Black, Hispanic/Latinx, Native Hawaiian or Pacific Islander, American Indian or Alaskan Native, and Afro-Caribbean applicants.¹²  Applicants report their current and permanent address. We assumed that current address referred to their living arrangement while attending medical school, and permanent address referred to their long-term home. GR was defined as the distance between an applicant’s current address and BIDMC, where in-person interviews were conducted. Applicants’ current zip codes were used to calculate distance “as the crow flies” to BIDMC using the Haversine formula, which calculates the great-circle distance between 2 points on a sphere based on their longitudes and latitudes. In the absence of data on applicants’ socioeconomic status, we used census data as a proxy. To identify applicants potentially with greater financial burden, each applicant’s current and permanent address was entered into the Federal Financial Institutions Examination Council (FFIEC) Geocoding Mapping System¹³  to obtain the following 2020 Census Socioeconomic Data: income level tract, percentage of racial and ethnic minority groups, and percentage of population below the poverty line within their tract. Per FFIEC, tracts are geographic entities within counties. Income level tract is classified as upper, middle, moderate, and low by comparing income within a tract to expected income in the Metropolitan Division, therefore adjusting for cost of living (definitions provided in online supplementary data). Data for current and permanent address were both tested as metrics of applicants’ socioeconomic status. Application cohort was recorded as 2019 (pre-COVID-19 pandemic, in-person interviews) or 2020 (COVID-19 pandemic, virtual interviews).

Continuous variables were described using mean and standard deviation, and categorical variables were presented as percentages. t test, ANOVA, chi-square test, and Pearson’s correlation were used for univariable comparisons of program geographic reach. Univariable predictors of GR were considered for multivariate testing if P≤.20. Application cycle was included a priori as the main predictor. Upper- and middle-income tracts for the current address were merged into a single category, given significant overlap of GR data. USMLE Step 1 scores were included a priori to control for academic achievement and were kept constant in the model. Step 2 CK were discarded given missing data.

Backward elimination was used to build a multivariable regression model predicting the impact of variables on program geographic reach. The multivariable analysis was constructed using 2 interaction terms to test the hypothesis that the relationship between interview type and applicant geographic reach was different in: (1) applicants from UIM backgrounds, and (2) applicants living in communities with poverty levels ≥7%, which constituted the 25th percentile. Bootstrapping was used given violation of the normality assumption. Statistical significance was determined for P≤.05. Stata 16 software for Windows (Stata Corp) was used.

The BIDMC Institutional Review Board approved this study.

A total of 1365 medical students met inclusion criteria, 667 in 2019 for in-person and 698 in 2020 for virtual interviews (Table 1). Comparison of applicant characteristics by interview year are shown in Table 1. In-person and virtual interview groups were comparable in demographics: age, gender, race/ethnicity, UIM status, and Step 1 and 2 CK scores (P>.05 for all). Genders were equally represented in both groups (females n=340, 50.97% vs n=363, 47.99%, P=.70). Half of applicants were White (n=337, 54.01% vs n=336, 48.14%, P=.52). There were 115 of 608 (18.9%) and 119 of 647 (18.4%) identified UIM applicants in the in-person and virtual interview groups, respectively. For both current and permanent address, in-person and virtual interview groups were comparable in income tracts and percentage of residents living under poverty conditions (P>.05 for all). For permanent address, applicants during virtual interviews had a higher percentage of minority residents (n=36.70, 24.74%) as compared to applicants during in-person interviews (n=33.65; 23.89%; P=.02). For current address, there was no difference in the percentage of minority residents among interview groups (P=.86).

Univariable predictors of program geographic reach are summarized in Table 2. Overall, the mean GR was 880.47 miles (SD=802.08) for in-person and 876.01 miles (SD=830.49) for virtual interview groups (P=.92). Applicant age and gender did not correlate with GR. Applicants from UIM backgrounds had a greater GR (ie, applied from further locations) as compared to non-UIM applicants (mean 1101.25 miles, SD=985.63 vs mean 826.99 miles, SD=757.29, P<.001). Having lower USMLE Step 1 scores correlated to applying from more distant location (r=-0.054%, P=.047).

When utilizing FFIEC data for current address, income tracts were not associated with GR. Applicants in low-income tracts had mean GR of 717.66 miles (SD=725.54), applicants in moderate income tracts had mean GR of 835.17 miles (SD=816.87), applicants in middle income tracts had mean GR of 922.72 miles (SD=820.20), and applicants in upper income tracts had mean GR of 900.37 miles (SD=828.68). Applicants living in communities with a greater percentage of minority residents applied from more distant locations (r=19.23%, P<.001). On the other hand, applicants living in communities with a greater percentage of people living under poverty conditions applied from closer locations (r=-8.16%, P=.003).

When utilizing FFIEC data for permanent address, applicants living in communities with a greater percentage of racial and ethnic minority groups had greater GR (r=19.74%, P<.001); conversely, FFIEC income tract (P=.70) and percentage of people living under poverty conditions were not significant (r=0.7%, P=.80).

The multivariable prediction model of program GR is presented in Table 3. There was no difference in GR among applicants from in-person and virtual interviews (P=.83). UIM status was not associated with GR (P=.08). Applicants from communities of lower socioeconomic status applied more locally: applicants from moderate and low income FFIEC tracts had 184.11 fewer miles (95% CI -322.68, -49.63) and 308.52 fewer miles (95% CI -501.28, -105.18) in GR as compared to applicants from middle and upper tracts (P=.007 and P=.003, respectively). The percentage of racial and ethnic minority groups within the applicants’ tract for both current (7.71, 95% CI 4.97-10.63) and permanent (2.72, 95% CI 0.26-5.011) address were independently associated with greater geographic reach (ie, applying more broadly). The percentage of people living under the poverty line in their current address (-7.99, 95% CI -11.88, -4.17, P≤.001) was independently associated with lower GR (ie, applying more locally). USMLE Step 1 score was controlled for but was not independently associated with applicant geographic reach (P=.70).

Multivariate analysis with interaction between GR and UIM, and GR and poverty levels is summarized in Table 4. The effect of interview type on how far applicants were willing to apply from was different for UIM and non-UIM applicants. UIM status had no association with GR for in-person interviews (P=.98), but virtual interviews were associated with an increased GR for UIM applicants as compared to non-UIM applicants (235.17, 95% CI 28.87-441.47, P=.02).

During in-person interviews, applicants living in communities with poverty levels ≥7% had less GR as compared to those living in communities with poverty levels <7% (-332.45, 95% CI -492.10, -172.79, P<.001), an effect that was not observed during virtual interviews (P=.25).

Despite many changes during the COVID-19 pandemic, this comparison of applicants to a large US surgery residency program found that applicants pre- and post-virtual interview application cycles were similar overall in demographic and socioeconomic characteristics. However, virtual interviewing was associated with increased GR for UIM applicants but had no impact on non-UIM applicants. On the other hand, in-person interviewing was associated with decreased GR for applicants from communities with higher poverty levels, an effect that was not observed during virtual interviews. These findings suggest that offering virtual interviews to surgery residency applicants may be an additional strategy to enhance program diversity.

In 2015, an Association of American Medical Colleges Organization of Student Representatives questionnaire about the cost of applying to residency reported that students funded their residency applications through personal savings, family savings, or loans.¹⁴  More than half of survey respondents reported that financial considerations influenced their decisions to attend interviews by, for example, limiting the number of applications, restricting applications to programs within a driving distance, and cancelling interviews due to lack of funds. Studies have shown that US graduates self-identifying as non-White have been less likely to receive an interview invitation and match in a surgery position after controlling for other factors, including USMLE scores.¹⁵  Applicants from UIM backgrounds tend to have greater medical school debt.¹⁶  It has been postulated that the financial burden of attending in-person interviews, greater for lower income applicant and some UIM applicants, could be reduced by virtual interviews.⁴  Optimizing virtual interviews and other virtual recruiting strategies may enhance contact between programs and applicants from UIM and lower income backgrounds.

We found that socioeconomic variables obtained from the FFIEC census data had a similar effect on GR for current and permanent addresses. It is possible for these estimates to be consistent because medical students of similar socioeconomic backgrounds might cluster to attend similar medical schools and live in similar neighborhoods.

This study is limited by use of a single large residency program and specialty located in the US Northeast, which may limit generalizing to other locales and specialties, particularly those more or less competitive in terms of application to position ratios.^16,17  As the US northeast is a small region with a high density of medical schools and residency programs, travel miles are likely to be much higher in the rest of the United States, particularly less urban areas. Type of school (public vs private) was not accounted for in the analysis. Data have shown that private medical school graduates are slightly less likely to have debt, but when present, their debt levels tend to be higher than public school graduates.¹⁶  This factor was not accounted for in our analyses. Application of study findings is further limited as only applications were studied and not interview offers, interviews conducted, ranking decisions, and Match results.

Travel in miles is a surrogate for but not equivalent to costs of travel or time required to travel, which are the key variables, although more difficult to study. Current location data was obtained from Electronic Residency Application Service application files and may not have reflected actual location at the time of interviews. Metrics of applicants’ socioeconomic status such as student debt or credit scores would have been ideal, but these were not available. Instead, socioeconomic status metrics obtained from census data were used as proxies. The data used is from the very first year of virtual interviews and may not reflect the following years, when programs and applicants have more experience with this modality as well as in the absence of an ongoing pandemic. Multiple studies have demonstrated an increase in the number of applicants since programs transitioned to virtual interviews across specialties, which was not observed in this study.¹⁸  Changes in GR after multiple virtual interview cycles for UIM and low-income applicants, as well as other groups remains unknown. Data from additional application cycles, both in-person and virtual, would further validate our inferences, but these were not available for research purposes.

Next steps include examining data from other locations and specialties. It is important to determine if virtual interviews produce more balanced diversity among specialties and programs, (ie, matched students and graduated residents).

In this early comparison of in-person vs virtual interviews at a large surgery residency program, there was no difference in overall GR or the proportion of applicants who were from UIM backgrounds or from communities with higher poverty levels. Virtual interviews for surgery residency during the COVID-19 pandemic were associated with increased geographic reach for applicants from UIM and from lower socioeconomic backgrounds after controlling for performance on the USMLE Step 1.

The online version of this article contains Federal Financial Institutions Examination Council definitions.

Funding: The authors report no external funding source for this study.

Conflict of interest: The authors declare they have no competing interests.

This work was previously presented at the Congress of the American College of Surgeons, October 16-20, 2022, San Diego, California, USA.