With the development of standardized online residency applications that can be easily submitted to many programs, the large number of submissions has led to unintended consequences for programs and candidates. Candidates feel obligated to interview as widely as possible to increase their chances of matching, and programs sift through hundreds of applications, many of which may be from applicants not likely to rank the program highly in the National Resident Matching Program (NRMP). In this issue of the Journal for Graduate Medical Education, Whipple and colleagues argue that programs are therefore forced to rely on “easy” metrics (ie, board scores, grades, and publications) to filter candidates, despite strong evidence that these metrics do not correlate with future success and may be prone to unconscious biases.1 They speculate that if programs were able to focus on fewer applications—with a more holistic approach that considers candidates' personal statements and letters of recommendation—programs could better identify trainees who would do well in their programs. By allowing programs to focus on a smaller number of applicants, programs could examine all components of the application more thoroughly and enhance the overall review process.
To accomplish this, the authors propose to allow candidates to show their hand by signaling their preferences when applying. A similar strategy is used by PhD graduates in economics seeking their first academic jobs.2 Using mathematical modeling, the authors predict that program directors would extend interviews to a different subset of candidates who not only have stronger “holistic metrics,” but also are more likely to rank the program highly. Thus, even when all applicants are incentivized to apply to many programs, candidates with weaker “easy” metrics but stronger “holistic” metrics would be more likely to receive invitations without hurting the candidates with stronger “easy” metrics.
Program directors among us can relate to the challenge of evaluating large numbers of applications. Although absolute cutoffs are discouraged, surveys show that US Medical Licensing Examination scores are among the most important factors programs consider when deciding who to interview, probably because it is an easy filtering metric.3 However, everyone seems to have differing opinions on candidate characteristics most important for success. Whipple and colleagues make the assumption that a more complete, holistic evaluation of applications will help program directors make better decisions. Although it seems obvious that a more thorough reading of an application will improve the assessment of a candidate, this assumption is not tested in the article.
We think the larger problem is that candidates are already attending too many interviews. Three years ago, senior residents applying for pediatric surgery fellowships in our program reported each interviewing at 17 to 21 programs and spending over $10,000 on travel and lodging, not to mention the days missed from training. While they were strong candidates, the field is very competitive and they wanted to leave nothing to chance. Conversely, for less competitive fellowships, the pressure is on programs to interview excessively, as many as 15 or more applicants for a single position. We believe that the concept of indicating preferences up front may also be used to address these problems.
Our group recently proposed instituting a match for the selection of candidates to interview.4 An “interview match” would allow programs to rank-order candidates into groups or tiers regarding who they would like to interview. Candidates would, in turn, do the same with respect to programs. These novel rank lists enable candidates and programs to signal interest in each other without revealing their preferences publicly. A matching algorithm would then be used to fill the set number of interview slots at each program. We also suggest that the number of interview slots be limited, but an alternative strategy could be to have programs modify the limit, depending on how competitive a program.
Both of these strategies rely on signaling mechanisms to indicate interest prior to an interview and place additional importance on the preinterview application review process. In the proposal by Whipple et al, the signaling method is somehow explicit in the application. In ours, the signaling is 2-way between the programs and candidates, conveyed by their respective interview match rank lists. Theoretically, the authors' proposal would reduce some of the workload burdens in reviewing large numbers of applications and lead to programs spending more time thoroughly reviewing a smaller number of applications. It has not been proven whether more thoroughly reviewing entire applications is beneficial. Maybe reducing the number of interviews would not only reduce the costs of excessive interviews but also lead to higher quality, more informative interviews. The strategy by Whipple and colleagues could be complementary to ours; applicants would first signal their preferences to help programs screen and rank applications, and then an interview match would be conducted based on their respective ranking of each other.
Content of applications is critical to the final ranking of candidates in both approaches. Neither strategy would be helpful if interview performance was the only criteria important for the final ranking of candidates. The relative importance of the interview probably varies by specialty and program. We would argue that the more important an interview is, the more interviews may be needed for the best resident-program match.
Ultimately, to make the matching process more efficient, real data are required to understand the decision-making processes of programs and applicants. To accomplish this, organizations that run residency matches, such as the NRMP, should share data such as the number of submitted applications and ranking lists of programs and candidates. Without this data, many assumptions need to be made on how changes in the interview selection process can impact the matching behaviors of both programs and candidates.
