The Effect of Implicit Bias on the OB/GYN Residency Application Process

Implicit bias refers to the unintentional and unconscious mental associations all individuals make about social identity groups, such as race or gender.¹  Although unconscious, implicit bias is informed by and reinforces societal trends, often favoring the socially dominant.²  Previous studies have demonstrated the discrepancies in employment attainment and career advancement between White and racially minoritized people.^3-6 

Like the general population, medical professionals also exhibit the same level of implicit bias.⁷  This is highlighted at the medical school level with discrepancies in increased admissions rates, more positive feedback, and better performance evaluations, all favoring White and male students.^8-10  Few studies, however, have explored how implicit racial bias affects residency selection.¹¹  The American College of Obstetrics and Gynecology has reported that, despite the steady increase of Black medical students, the rates of racially minoritized obstetrics and gynecology (OB/GYN) residents have decreased over the past few years.¹²  Recent literature has supported the notion that a racially and ethnically diverse health care workforce can effectively address the disparities that exist in maternal health outcomes.¹³  Understanding the contribution that implicit bias has on the racial and ethnic makeup of physician trainees emboldens the argument for investment in policies and interventions that encourage intentionality in building diversity in health care.¹⁴ 

In this study, we sought to determine whether implicit bias influences the selection into OB/GYN residency of Black applicants, defined as underrepresented in medicine,¹⁵  and Asian applicants, who, though not underrepresented, face racial bias in medical education, with a lack of research about their experiences.¹⁶  We hypothesize that when there is a clearly superior applicant, such as the top-tier applicant, implicit bias would play less of a role,³  but when applicants have similar qualifications, participants would prefer White applicants due to implicit bias.

Participants were OB/GYN residents, fellows, and attending physicians from 7 randomly selected large academic institutions’ OB/GYN residency programs, based on online availability of emails on their institutions’ websites, in the year 2021. Institutions were selected starting in the Northeastern United States, moving westward to capture as much geographical diversity as possible. After reaching at least 100 participants, no further participants were recruited. Physicians received an email invitation to the questionnaire sent from the study team. The email invitation first directed them to a consent form, informing them that they could opt out of the study at any time or not respond to any questions if they felt uncomfortable. A power calculation was done using a pilot study by analyzing the difference of the rankings of applicants by 20 applicants selected from a sample of convenience not included in the data presented here. The same questionnaire and same methods of data analysis were employed, which found that 100 participants (50 in each arm) were needed in total to sufficiently power the study.

A 3-part questionnaire was electronically created on Qualtrics XM 2020 (Qualtrics). The questionnaire was created by a fellowship program director, a fellow involved with recruitment, and a medical student, and was reviewed by a medical education researcher. Participants were randomized to 1 of 2 questionnaire forms, A and B. The first part of both forms included demographic data, such as academic position (resident, fellow, attending), age ranges, gender, race/ethnicity, where they spent their formative years, and where they were currently practicing.

In the second part of the questionnaire, participants were shown 4 applicant profiles. These profiles contained both objective and subjective measures that are found on the Electronic Residency Application Service (ERAS), including name; date of birth; race and ethnicity; gender; United States Medical Licensing Examination Step 1, Step 2 Clinical Knowledge (CK) scores; honors; undergraduate institution attended; medical school attended; summaries of grades, letters of recommendation, personal statement, other experiences; and hobbies. The 4 applicants were stratified with academic institutions attended, USMLE scores, and summary of their grades into top-, mid-, and low-tiers. The tiers of the academic institutions were distinguished by rankings of undergraduate and medical institutions on US News & World Report.^17,18  The USMLE scores were sorted using means and standard deviations of average OB/GYN applicants based on Charting Outcomes in the Match, 2020.¹⁹ 

Each participant was shown a top-tier applicant, 2 mid-tier applicants, and a low-tier applicant. The mid-tier applicants were differentiated by one demonstrating resilience after losing her mother to ovarian cancer during medical school. Between forms A and B, the names and races were changed, but all other attributes remained the same. Applicants’ photos were not included to minimize the effect of appearance-based bias; because research shows that physical attractiveness can affect applicants’ rankings, also called “pretty privilege.”²⁰  In form A, the participants saw a White top-tier applicant, an Asian mid-tier applicant with resilience, a White mid-tier applicant without a story demonstrating resilience, and a White low-tier applicant. In form B, the participants saw a Black top-tier applicant, a White mid-tier applicant with resilience, an Asian mid-tier applicant without a story demonstrating resilience, and a White low-tier applicant. A summary of the race, gender, and names of applicants in each form can be found in the online supplementary data. The top- and mid-tier applicants were women while the low-tier applicant was a man to attempt to elucidate whether gender would affect the rankings. The participants were then asked if they wanted each applicant as a colleague and asked to rank them from 1 (most favorable) to 4 (least favorable).

In the third section, participants were asked to rank factors that affected their ranking of the previous section’s applicants. These factors were selected based on a National Resident Matching Program survey of OB/GYN program directors.²¹  The factors were academic achievements and scores, dedication to the field, resilience, fit for program, research and other extracurricular activities, and letters of recommendation. Participants ranked these factors from 1 (most impactful) to 6 (least impactful).

Forms A and B of the questionnaire can be found in the online supplementary data.

The primary outcome of this study was the ranking of the applicants from most (1) to least (4) favorable. The secondary outcomes were the rankings of the factors that impacted the participants’ ranking.

Baseline characteristics of study participants were compared with the Pearson’s chi-square tests or Fisher’s exact tests as appropriate. We examined whether rankings between the 2 forms differed significantly using a between-subjects one-way analysis of variance (ANOVA) test. This analysis facilitated prediction of means using generalized estimated equation (GEE) modeling to account for intra-respondent variability. Pairwise analyses were then conducted using the ANOVA model to examine differences between different applicants regarding predicted average ranking. We calculated an inter-item correlation coefficient to assess construct and content validity among survey responses. Significance was identified by P<.05. SAS 9.4 (SAS Institute) and R (R Core Team) were used for analyses.

Institutional review board approval was received through the Albert Einstein College of Medicine Institutional Review Board.

One hundred ninety-one people were invited, and 103 participants ranked the 4 applicants. Fifty-one participants were randomized to form A, and 52 to form B. No significant differences were found in the gender, age range, race, ethnicity, and academic positions between the 2 groups’ participants. The distribution of participants who completed each form and their demographics can be found in Table 1.

The GEE model facilitated projected mean scores of each applicant given all the data included (Table 2). Mean ranking of the White top-tier applicant was 1.1, while that of the Black top-tier applicant was 1.5. Among mid-tier applicants, the White applicant with resilience had a mean ranking of 2.0, White applicant without resilience ranked 2.4, the Asian applicant with resilience ranked an average of 2.5, while the Asian applicant without resilience ranked an average of 2.6. The low-tier White applicant ranked an average of 4.0. We calculated an inter-item correlation coefficient of 0.56 between applicant profile and applicant ranking by respondent.

Of the 51 participants who completed form A, 46 (90.2%) ranked the White top-tier applicant as the highest applicant, 3 (5.9%) as second highest, 2 (3.9%) as third highest, and none as the lowest (Figure 1). In form B, the Black top-tier applicant was ranked by 34 of 52 (65.4%) as the highest applicant, 12 (23.1%) as the second highest, 5 (9.6%) as the third highest, and 1 (1.9%) as the lowest (Figure 1). Table 3 displays results of the pairwise ANOVA test. Compared to the White top-tier applicant, the Black top-tier applicant was significantly less likely to be ranked as the top candidate (-3.3, 95% CI -3.5, -3.1). This association did not differ depending on respondents’ race or position.

In form A, 20 (39.2%) ranked the Asian applicant with resilience as the more preferred applicant, while 31 (60.8%) preferred the White applicant without resilience (Figure 2). In form B, 36 (69.2%) ranked the White applicant with resilience as the more preferred applicant, and 16 (30.8%) ranked the Asian applicant without resilience as their preferred applicant.

The pairwise analysis of mid-tier applicants showed a significant difference in the rankings of the different applicants with and without resilience. White applicants without resilience were significantly more likely to rank higher than Asian applicants without resilience (3.2; 95% CI 3.0, 3.4). The Asian applicant with resilience was significantly more likely to be ranked highly compared to the Asian applicant without resilience (3.0; 95% CI 2.8, 3.2). The White applicant with resilience was significantly more likely to be ranked higher than the Asian applicant without resilience (0.6, 95% CI 0.5, 0.7). No aspects of the respondents’ demographics or level of training were correlated with significant differences in the ranking of mid-tier applicants.

When asked to rank most important factors in their ranking, 34 (33%) selected academic achievement, 25 (24.3%) selected fit for program, 20 (19.4%) selected dedication to the field, 11 (10.8%) selected resilience, 2 (1.9%) selected research and other experiences, and 6 (5.8%) selected letters of recommendation as most important. There was no significant difference between these rankings (see online supplementary data).

Participants, regardless of their own demographics, ranked the top-tier White applicant higher more frequently than the top-tier Black applicant. Both mid-tier White applicants, with and without resilience, were more likely to be preferred over Asian applicants, including the applicant demonstrating resilience. No difference was found between factors in informing participants’ decisions. We found a moderately strong inter-item correlation between applicant profiles and ranking, suggesting both construct and content validity regarding the response process.

These results suggest the existence of a racial implicit bias favoring White applicants, consistent with results from previous studies conducted in the United States.³  These results are also consistent with previous studies in medicine, showing a preference of White students at all levels of medical education.^8,10,22-24  Studies have demonstrated that White race, when controlling for all other factors, can be used as an independent predictor of higher final clerkship grades.^10,23  Black, Latinx, and Asian students are less likely to receive honors, with Asian students more likely to report lower overall grades.^23,24  Even the Medical Student Performance Evaluation, intended to be a standardized summary of the student’s performance, falls short of being objective since White applicants often receive more positive summary words when adjusted for all other demographics.¹⁰ 

Implicit bias deters the selection of racially minoritized applicants into medical school.⁸  Those accepted face possible biases in their evaluations, a factor used in selecting students into residency.²³  Even if they receive evaluations comparable to their White peers, our results suggest that there is still bias against their selection into OB/GYN residency. These disparities ultimately result in fewer racially minoritized residents and lead to fewer racially minoritized OB/GYN providers, reducing the physician-patient concordance shown to be linked to improved patient outcomes.²⁵ 

One limitation in this study is the selection bias from participants who were more curious about this topic on both extremes, those who believe that implicit bias is an issue in medical education that needs to be addressed and those who believe that implicit bias does not exist within medical education, responding to the study. Participants might not be reflective of a typical residency program selection committee, where program directors and faculty usually make final decisions about applicant rankings, while trainees might give minor input. Inclusion of trainees aimed to elucidate generational differences. While the questionnaire was created with instructions written to be as neutral as possible, there is a possibility of participants recognizing the questionnaire’s implicit bias assessment. Since there is a trend toward recruiting more racially minoritized applicants to OB/GYN, some participants may be influenced by social desirability bias,¹  which would shift the responses toward the null hypothesis, so the actual impact of implicit bias could be stronger than our quantified effect estimates. Also, those in selection committees may have had implicit bias training, which would mitigate findings of this study. Another limitation was our inability to make an in-person analysis; we could not give one participant both forms of the survey without the possibility of participants realizing that we only varied the names and the races of the applicants and figuring out the study objective. Also, most of our respondents were within the 25 to 40 age range (76.6%) and female (86.0%). Previous studies of health care professionals have demonstrated that younger age and being a woman are associated with lower levels of implicit bias, so our results may not reveal the actual extent of implicit bias.²⁶  To validate these results, future studies could replicate this study in other fields of medicine with a more balanced gender ratio than exists in OB/GYN.²⁷  Between the 2 groups, more attendings received form B (50.0% vs 36.5% in form A, P=.19); while this difference was not significant, it could explain why the Black top-tier applicant was ranked lower and the White mid-tier applicant was preferred.

Future directions could investigate if there is a similar trend in other specialties in a way more consistent with their application processes, such as only surveying program directors or having the number of profiles be proportional to the actual applicant race breakdown.

This study suggests an implicit bias against top-tier Black and mid-tier Asian applicants that is consistent with the implicit bias in medical education reported in previous studies.

The online supplementary data contains further data from the study, the survey used in the study, and a visual abstract.

Funding: The authors report no external funding source for this study.

Conflict of interest: The authors declare they have no competing interests.

This work was previously presented at the Society of Maternal-Fetal Medicine’s 42nd Pregnancy Meeting, January 31-February 5, 2022, Orlando, Florida, USA.