There has been an increase in the number of applications medical students have submitted for the National Residency Matching Program (NRMP). These additional applications are associated with significant costs and may contribute to match inefficiency.
We explored if match rates improved in years when an increased number of applications were submitted.
We analyzed yearly published data from the NRMP and the Electronic Residency Application Service for 13 specialties. A generalized linear model was used to assess the relationship between the annual match rate and the mean number of applications submitted per applicant, while controlling for the number of positions available and the number of applicants in the given year.
Over the last 13 years there has been an increase in the mean number of applications submitted per applicant (P < .001). For the 13 assessed medical specialties, there was no statistically significant relationship between the mean number of applications per applicant per year submitted to the NRMP, and the annual match rate (odds ratios near 1.00 and nonsignificant, P values > .05).
There was no improvement in the match rate in years when medical students submitted an increased number of applications. Therefore, it would appear that the applicants do not benefit from the larger number of applications submitted. Further study is required to assess the cost and benefit of these additional applications.
Medical students are submitting a larger number of applications. The effectiveness and impact on the match have not been studied.
A study of aggregated data suggests applicants do not benefit when all submit a large number of applications.
Aggregate data preclude analysis if certain types of applicants benefit from current practices; data apply to US senior applicants only.
The current high number of applications submitted has no utility for applicants and adds financial and opportunity costs for applicants and programs.
Matching into residency has become increasingly competitive.1–5 As a result, medical student applicants are submitting an increasing number of applications to the National Residency Matching Program (NRMP). These additional applications can have a deleterious effect on the match in 3 ways: (1) they create an added cost for applicants, (2) they result in an additional time commitment for program directors who need to review these applications, and (3) they cause congestion in the application review process; therefore, program directors may overlook preferred applicants as a result. There is no study investigating whether these additional applications are associated with a benefit to match participants.
Game theory is the economic study of decision making. The prisoner's dilemma game illustrates the counterintuitive phenomenon of all members of a group being worse off when each member of the group acts in his or her self-interest (figure 1).6,7 Applying the prisoner's dilemma model to the residency match in aggregate, we hypothesize that applicants may be better off if they could agree to submit a finite number of applications rather than trying to out-apply each other. We have previously published on the possibility of a prisoner's dilemma existing in the urology match, suggesting that the urology match could do without such a large number of applications, and that participants would be served by a reasonably set application limit.8
In this study, we investigated whether other medical specialty matching markets and applicants are benefited by a higher number of applications submitted. We also investigated if there is a related improvement in the match rate in years when larger numbers of applications were submitted.
Data were obtained from the NRMP9 and Electronic Residency Application Service (ERAS)10 websites. On the NRMP website, a robust amount of match data is published, beginning with 1984. For each specialty, the NRMP reports for each year contain the number of programs, the number of positions offered, the number of unfilled programs, the number of US senior applicants, and the number of US senior matches. Data from the ERAS website contain tables citing the yearly mean number of applications submitted in each specialty, and the number of applicants submitting those applications.11
The match rate was calculated with the following formula: Match Rate = No. of US Seniors Matched/No. of US Senior Applicants. All data necessary for match rate calculations were obtained from the NRMP website in the annually released “Results and Data” packets for years 2008 through 2013.12–17 From each packet, the following information was extracted for 13 specialties: (1) positions offered, (2) number of US senior applicants, and (3) number of US senior matches. Analysis was limited to 2008 and later, because starting with 2008 the data show applicants who ranked multiple specialties.
For specialties offering both postgraduate year (PGY)-1 and PGY-2 positions (anesthesiology, dermatology, emergency medicine, psychiatry, radiology-diagnostic), the values from PGY-1 and PGY-2 positions were combined for final analyses, with the assumption that overall trends in match rate should optimally be reflected in the sum, rather than analyzed separately. In addition, trending for 6 years mitigated year-to-year variability.
To test if the annual average number of applications was related to the match rate, a generalized linear model with a logit link was used. The response was the yearly number of US seniors matched over the number of US senior applicants, and the predictor was the mean number of applications per student during that year (derived from ERAS data).11 In addition, the number of positions offered, and the number of applicants, were included in the model as covariates. The effect of the mean number of applications was measured by an odds ratio (OR), and the α level for statistical significance was set at .05.
To estimate the cost savings effect of a residency application limit, a model was created using these parameters: the number of US applicants for the 2013 match, the mean number of submitted applications per specialty for the 2013 match (US applicants), the cited ERAS application costs,18 and a hypothetical application limit of 20 applications per applicant (table 1). Cost savings of a match with an application limit of 20 for the 13 studied specialties was calculated by the following formula: Cost savings = Cost of Match (No. of Applicants 2013, No. of Applications 20) – Cost of Match (No. of Applicants 2013, No. of Applications 20).
To estimate the time saving effect of a residency application limit for program directors, we created a model using these parameters: the number of US applicants for the 2013 match, the mean number of applications per specialty for the 2013 match (US seniors), a hypothetical application limit of 20 applications per applicant, and an 8-minute application review time per application (table 1). As there were no data available that determined the mean amount of time program directors spend reviewing an application, we assumed an application review time of 8 minutes, on the basis of how long it would take a program director to read through a student's application. Time savings of a match with an application limit of 20 for the 13 studied specialties was calculated by the following formula: Time Savings = Time Saved (No. of Applicants 2013, No. of Applications 20) – Time Saved (No. of Applicants 2013, No. of Applications 20).
The mean number of applications submitted by each applicant has significantly increased over the last 13 years (P < .001; figure 2). The specialty with the highest rate of increase in mean applications submitted was orthopedic surgery (1.85 applications per year), and the specialty with the lowest rate of increase in mean applications submitted per year was pediatrics (0.44 applications per year).
table 2 shows the results from the generalized liner model by specialty and for all specialties combined. For every specialty, the effect of the mean number of applications was small (odds ratios near 1.00) and nonsignificant (P values > .05). The number of positions offered significantly increased the odds of a match for dermatology (P = .001) and emergency medicine (P = .05), while the number of applicants significantly decreased the odds of a match for dermatology (P = .02), emergency medicine (P < .001), otolaryngology (P < .001), and diagnostic radiology (P = .001). When all specialties were combined in a single analysis, with specialty included as a stratified variable, there still was no effect for the mean number of applications (OR = 1.00; 95% CI 0.99–1.01; P = .61). The number of positions offered increased the odds (P < .001), and the number of applicants decreased the odds (P < .001).
If a hypothetical application limit of 20 was created for the 2013 match (for the 13 specialties studied), each applicant on average would save $174 from a total application cost of $355. With 27 027 applicants, the total savings would be $4,696,979. The amount of time saved per program director, with a hypothetical application limit of 20, would be 11 hours and 19 minutes. For the 2936 program directors, in the specialties analyzed, the time savings would be 33 250 hours.
Applicants individually have an incentive to submit a large number of applications. However, when every applicant submits a large number of applications, these applications in aggregate may “cancel” each other out, leaving participants with added costs yet without an improvement in the match rate.
The reason why medical students are not benefitted by submitting an unlimited number of applications can be explained by game theory and the prisoner's dilemma model. Consider 2 highly qualified students, A and B, applying for a residency position in orthopedics. If medical student A and medical student B submit 20 applications, they have an equal chance of matching. If medical student A submits 40 applications while medical student B submits 20 applications, medical student A will have an increased chance of matching over medical student B (and vice versa). If medical student A and medical student B each submit 40 applications, they have the same chance of matching, yet they incur the added cost of the 20 additional applications. Using this model, it would appear that these medical students would be served by an application limit.
The idea of limiting the number of residency applications is not new. We surmise that the tiered ERAS application pricing system was used to discourage the overzealous submission of applications. ERAS charges $92 for the initial 10 applications, $9 each for applications 11 to 20, $15 each for applications 21 to 30, and $25 each for applications 31 or more.18 However, it is unlikely that most medical students will let a $25 application fee stand in the way. Thus, the current pricing system seems to be a poor deterrent. Although increasing the ERAS prices for submitting additional applications would deter applicants from overapplying, wealthy applicants would be able to out-apply poorer applicants, and individual resources would become a factor in match success. A more effective and equitable method for limiting the high number of applications would be to impose an application limit.
The match system algorithm20 was designed by recent Nobel Prize winners Roth and Shapley to facilitate the creation of matches that optimize the outcome for both applicants and programs. In this context, an evaluation of the current application system is important. For medical students, these additional applications add significant costs to the application process.21 For program directors who already have limited time,22 the added applications add a significant time commitment. Most important, the high number of applications may cause congestion and cause program directors to overlook preferred applicants as a result of the large number of applications. Anecdotally, this phenomenon is best illustrated when we hear of top-notch applicants not receiving interview offers from less competitive programs, by program directors noting that, given the large number of applications, they are forced to interview applicants they perceive are interested in their program as opposed to the best applicants.
Our study has limitations, including the use of only aggregate data. We thus do not know if certain applicants (ie, American Osteopathic Association, higher US Medical Licensing Examination scores) are benefitted by additional applications, nor do we know which applicants (ie, poor, strong, average) are submitting a higher number of applications. Our data pertain only to US allopathic graduates and should not be generalized to other groups. Additionally, data are not available that can be used to assess the effect that additional applications have on rank lists.
The current system is not efficient, and in certain specialties (otolaryngology and orthopedic surgery), applicants are on average applying to nearly half of the available residency programs. This suggests an ineffective utilization of the match. Having said that, our study does not offer sufficient information to suggest an application limit to the match, nor do we know the level at which such a limit should be set. It may, however, serve as the groundwork for further research to assess the utility of an application limit.
Despite an increase in the number of applications submitted, there is no associated increase in the match rate, suggesting that applicants may be done a disservice by a system where they can submit an unlimited number of applications. Research is needed to assess if these additional applications have led to an improvement in applicants' rank list outcomes, and to assess how these additional applications have affected applicants of varying quality.
Steven J. Weissbart, MD, is Chief Resident, Department of Urology, Icahn School of Medicine at Mount Sinai; Soo Jeong Kim, BA, is a Medical Student, Icahn School of Medicine at Mount Sinai; Richard S. Feinn, PhD, is Assistant Professor of Medical Sciences, Department of Medical Science, Frank H. Netter, MD, School of Medicine, Quinnipiac University; and Jeffrey A. Stock, MD, is Associate Clinical Professor of Urology, Department of Urology, Icahn School of Medicine at Mount Sinai.
Funding: The authors report no external funding source for this study.
Conflict of interest: The authors declare they have no competing interests.