The Residency Match: Interview Experiences, Postinterview Communication, and Associated Distress

The goal of the National Resident Matching Program (NRMP) is “to match healthcare professionals to graduate medical education and advanced training programs through a process that is fair, efficient, transparent, and reliable.”¹  The Institutional, Program, and Applicant Match Participation Agreement (MPA) and the code of conduct (COC) were established to prohibit undesirable interview behaviors and forbid coercion of applicants to declare a program's ranking.^2,3  However, applicants remain free to volunteer such information and programs can do the same, often placing both in an uncomfortable position where assurances may be offered and statements misinterpreted.

The dissolution of the “exploding offer” and increased regulation suggest that significant improvements in fairness and transparency have been made.⁴  However, there remains evidence of interview behavior that violates the spirit of the Match and of ongoing gamesmanship in the postinterview period, with a number of applicants reporting exposure to interview questions about other interview locations,⁵  rank preferences, marital status, and plans to have children,^6–8  and discomfort when programs contact them following the interview.⁹  Besides adding a statement explicitly reminding programs that applicants should not be asked about other interviews, the NRMP does not appear to have adopted any specific policy changes or interventions in response to these data.⁵ 

In this context, we hypothesized that (1) the behaviors continue to take place across a variety of specialties; (2) the behaviors cause distress; and (3) postinterview communication is an aspect of the Match process in which the applicant may prefer change. The purpose of this survey was to assess interview behaviors and to obtain applicants' perspectives on potential policy changes that could help improve the Match process and minimize associated distress. To accomplish this goal, we surveyed applicants to several specialties at a single, large institution.

An anonymous, 31-question survey was created to assess applicant experiences among all programs at which they interviewed across a variety of specialties during the 2015–2016 cycle (provided as online supplemental material). The survey questions drew from prior publications on this topic^6,7  and assessed demographic information, inquiries that may have arisen during the interview that violate the NRMP COC, postinterview behavior on behalf of applicants and programs, policies used during postinterview communication, and applicant opinions about these experiences. Questions were initially piloted for clarity among 4 individuals (2 current residents, 1 program coordinator, and 1 program director [PD]); no further testing was done. Question formatting varied from yes/no, true/false, multiple choice, and numeric responses, so the absolute exposure rate to certain interview experiences as well as frequency could be estimated.

All residency PDs at our institution were invited to take part in this project, with programs from the following specialties agreeing to participate: dermatology, emergency medicine, internal medicine, neurological surgery, physical medicine and rehabilitation, radiation oncology, urology, and vascular surgery. Because a portion of applicants apply to more than 1 specialty, respondents were asked to select which of the aforementioned specialties was their preferred choice and to answer only questions related to those specialty-specific interviews. Applicants to these programs during this match cycle were solicited to participate in the survey by e-mail with an introductory cover letter describing the background and rationale for the work. The survey was generated using SelectSurvey.NET (ClassApps, Kansas City, MO). The survey opened on March 7, 2016, after the rank order list (ROL) deadline, to encompass the entire scope of postinterview communication experienced at any program and to minimize any potential concern applicants may have had about their responses affecting program rankings. Two reminder e-mails were sent prior to survey closure on April 6, 2016. Applicants were informed that their decision to proceed implied consent to participate in our study. A software setting preventing transmission of identifying information ensured anonymity, and applicants were not asked whether they had interviewed at our institution. Duplicate responses from the same Internet protocol address were prohibited, and no awards or incentives were offered.

The survey was approved by the Cleveland Clinic Institutional Review Board.

The data were analyzed and tabulated to report descriptive results. A χ² test was utilized for comparisons between groups when the responses were categorical. An unpaired t test was used when the responses were continuous. Statistical significance was assumed at the .05 level. All statistical analysis was performed using SAS version 9.4 (SAS Institute Inc, Cary, NC).

Complete surveys were returned by 2079 of 6693 applicants, for an overall response rate of 31%. Demographic information can be found in table 1.

Applicants' recollection of program behavior is summarized in table 2. A total of 72% of applicants (1497 of 2079) were asked at least once about other interviews, with applicants reporting that this occurred at 25% of interviews overall. A total of 38% (790 of 2079) were asked at least once about marital status, occurring at a mean 14% of interviews. A total of 15% of applicants (312 of 2079) were asked at least once how highly they would rank a program. All of these questions arose at similar rates for men and women (P > .05). A total of 17% (353 of 2079) stated they were asked at least once about children or plans to have children, occurring at 5% of interviews overall. Women were asked about childbearing 50% more frequently than men (22% versus 14%, P < .0001).

Overall, 91% of applicants (1885 of 2079) engaged in postinterview communication, including written thank-you notes and e-mails, at 1 or more of the programs at which they interviewed, of which 77% of applicants (1451 of 1885) initiated this contact. A total of 22% of applicants (460 of 2079) stated that thank-you notes were written solely due to fear of otherwise being viewed unfavorably. A total of 70% of all applicants (1451 of 2079) informed their top program that they had ranked it highly. Of this subset, 78% (1126 of 1451) felt this action would improve match success, although 70% (1022 of 1451) reported a sense of distress in making this proclamation. Ultimately, 20% of applicants (418 of 2079) reported changing their rank list based on their engagement in postinterview communication.

A total of 10% of applicants were told they were ranked to match by at least 1 program. All applicants were asked whether they would want to know if a program had ranked them to match prior to the ROL deadline, to which 64% (1326 of 2079) answered affirmatively. However, 61% of respondents (1261 of 2079) stated that knowledge of this information would cause distress because of pressure to affirm mutual interest. Overall, 20% of programs (mean of 2.2 programs of the average 10.8 programs at which applicants interviewed) explicitly discouraged postinterview communication. This rate was highest for radiation oncology programs (38%, 4.4 of 11.6), followed by internal medicine (27%, 2.8 of 10.5), dermatology (22%, 1.9 of 8.6), urology (13%, 1.7 of 13.4), emergency medicine (11%, 1.2 of 11.1), neurological surgery (6%, 0.9 of 14.7), physical medicine and rehabilitation (6%, 0.6 of 10.4), and vascular surgery (2%; 0.4 of 16.2; P < .001).

A total of 71% of respondents (1480 of 2079) stated they would feel relieved if postinterview communication was explicitly discouraged by programs (figure 1). A total of 51% (1059 of 2079) said they would prefer programs to prohibit candidates from notifying them of their high rank on the candidate ROL, to help avoid manipulation of the match process (figure 2). Applicants were asked whether a complete ban on postinterview communication would be preferred, and 30% (633 of 2079) responded yes, whereas 50% (1030 of 2079) stated that postinterview communication may continue as long as no NRMP rules are violated.

In a large sample of residency candidates participating in the 2015–2016 Match, this survey demonstrates the high frequency of questions candidates face that may breach the NRMP COC during the interview process. Engagement in postinterview communication, including the writing of thank-you notes and candidates' voluntary proclamation of a program's high rank, differs in that these activities are not prohibited by the Match Participation Agreement or the COC. Our findings show that these steps are often taken due to candidates' perception of their importance, and yet a large majority expressed a preference that these actions also be actively discouraged. To our knowledge, this is the first multidisciplinary survey to report on candidate preferences for change in the postinterview period.

Our findings are in line with previous studies reporting on the rates of interview violation questions. For example, in a large multidisciplinary survey, 66% of applicants reported being asked at least 1 potentially illegal question, with specialty-specific surveys in urology, radiation oncology, and dermatology showing exposure to these types of questions exceeding 90%.^7,10–12  On the other hand, an emergency medicine survey showed a much lower rate (30%) for such illegal queries.¹³ 

The only question in our survey that manifested a demographic bias was regarding plans to have children, with women reporting that they were 50% more likely to be asked this question than men. Data in the indexed literature regarding gender-specific exposures to certain questions are limited, but a study of urology applicants also showed women as more likely (62% versus 25%, P < .001) to be asked about plans to have children.¹¹  These findings suggest there may be a bias among faculty interviewers that requires attention.

In terms of postinterview communication, previous studies also reveal findings similar to ours, with approximately 60% to 70% of applicants engaging in postinterview communication despite evidence that engaging in this behavior bears no association with Match success.^14,15  This may not be surprising, as prior studies have revealed that at least half of applicants exaggerate their interest in programs in order to improve their chances of matching favorably, with less than 10% of PDs actually believing students' proclaimed interest.^7,16  Our findings suggest that many students continue to express interest or proclaim high rank to programs because of their perception that it may improve their match success; however, the fact that 51% of our survey respondents would prefer programs to prohibit candidates from notifying them of their high rank (in contrast to the 27% who clearly stated they would not prefer this) suggests that this form of postinterview communication may not only be disingenuous at times, but may also be distressing.

Our study has some limitations. It was intentionally disseminated after the ROL deadline, and the intervening time may subject responses to recall bias. The response rate was 31%, and the survey may have attracted participants who had negative perceptions of interview behavior or postinterview communication. We surveyed candidates applying to a single institution, and findings may not be representative of the entire 2015–2016 applicant pool. The survey questions were not tested beyond the 4-person pilot, and respondents may have interpreted questions differently than intended. Results may also have been influenced by the relatively higher number of internal medicine candidates who participated.

Areas for future study should entail querying PDs on training offered to faculty interviewers to help minimize interview violation questions; such a study would provide specific data and a different perspective on this issue. Additionally, it may be valuable to obtain PD feedback on whether changes in the postinterview communication policy would be productive. Lastly, in terms of the applicants themselves, the creation of an opt-out (rather than opt-in) postinterview survey for all NRMP participants in which applicants can confidentially report their experiences would allow the NRMP to document these issues on a larger scale.

Our single institution study results indicate that potential match violation questions are experienced by a majority of applicants, but they may stem from a minority of programs at which applicants interview. Candidates report significant distress associated with postinterview communication and would prefer programs to actively discourage such behavior.