Immediate Impact of Participation in the Electronic Residency Application Service on a Fellowship Program

In a 1990 nationwide survey,1 graduate medical education program directors who participated in the National Residency Matching Program supported the concept of a centralized residency application service with electronic transmission of data as an alternative to the then current manual system. Further investigations under the auspices of the Association of American Medical Colleges2,3 verified that the majority of residency program directors for both obstetrics-gynecology and family medicine supported the initiation and use of an online, computerized residency application process via a centralized Electronic Residency Application Service (ERAS).

Graduating medical students applying for first-year residency positions in obstetrics-gynecology in 1995–1996 took part in a large-scale pilot test of ERAS in 150 participating programs. The pilot study was intended to identify potential problems before extending the electronic process to other residency programs. Findings indicated that disadvantages mainly related to technical issues. However, in general the advantages outweighed the disadvantages because application materials arrived in a well-organized, uniform, and complete format, and filtering mechanisms could be used by programs to streamline the application process.4 

Widespread implementation of ERAS by obstetrics-gynecology programs occurred in 1996–1997, followed by adoption in 1997–1998 by family medicine, radiology, emergency medicine, orthopedic surgery, and transitional-year programs. In 1999, internal medicine programs first participated in ERAS, and fellowship programs followed in 2003. Currently, 34 fellowship training programs participate in ERAS, of which 14 are subspecialties of internal medicine that include cardiovascular medicine (since 2006) and interventional cardiology (since 2008).5 

A report comparing applications made to one emergency medicine residency program before and after participation in ERAS (1996–1999)6 indicated that the number of applications increased but the quality of the applicants remained unchanged. Similarly, a survey7 of all 407 internal medicine residency programs (in the United States, Canada, and Puerto Rico) after the 1999 match that first used ERAS found that 86% of responding program directors (81% response rate) viewed the electronic system as easier to manage. The impact of ERAS on the number of applications varied: 48% of programs reported an increase, 32% a decrease, and 20% no change compared to the previous year. In addition, 47% of program directors rated the overall quality of applicants in the first ERAS match as “similar” to that of the match immediately prior to ERAS participation; 38% rated it “better” and 15% “worse.” More recently, fellowship program directors participating in ERAS have provided initial positive reports and encouraged participation of other programs.8 

Initiation of ERAS for cardiology fellowships was generally perceived to result in an increase in the number of applications received by individual programs. This, in turn, raised concerns regarding the administrative burden of such a change and its potential effect on the quality of applications received by cardiology fellowship programs. This study aimed to objectively evaluate the immediate impact of the newly administered ERAS on the cardiology fellowship application process.

The cardiology fellowship program at Geisinger Medical Center (GMC) received a total of 1824 applications in 4 consecutive years, from 2004 to 2007. Before ERAS participation (2004 and 2005), the program received hard-copy, printed applications through the US Postal Service. Beginning in 2006, the program participated in ERAS and received applications electronically. We reviewed all applications submitted from 2004 to 2007 and extracted selected applicants' demographic and educational data from all printed and electronic applications, as appropriate for the study time period. The GMC Institutional Review Board reviewed and approved the study prior to initiation.

Specific extracted information included the variables of applicants' age, gender, type of medical school (allopathic versus osteopathic), location of medical school, years since medical school graduation, geographic location of internal medicine residency, citizenship status, research experience, and 3-digit scores on the United States Medical Licensing Examination (USMLE). We defined a “complete application” as one that contained all the requested demographic and educational information about a candidate, the candidate's personal statement, at least 3 letters of recommendation, pertinent licensing examination scores, and medical school transcripts.

Student t tests for continuous variables and χ² tests for categorical variables compared applicants' demographic and educational characteristics, obtained from paper applications submitted in 2004–2005 (pre-ERAS) versus those obtained electronically from applications submitted to the program in 2006–2007 (post-ERAS). Tests of statistical significance for differences between the 2 time periods used a 2-sided nominal value of P < .05.

In the 2-year period prior to participation in ERAS, our program received a total of 732 applications (353 in 2004 and 379 in 2005), of which 577 (79%) were complete. In contrast, the immediate 2-year period after ERAS yielded a total of 1092 applications (524 in 2006 and 568 in 2007); 983 (90%) of these applications were complete. Comparison of periods before and after ERAS yielded a 49% increase in the total number of applications and a 70% increase in the number of complete applications (P < .05 for both) to our program. In the same time frame, the nationwide pool of applications to all cardiology fellowship training programs grew by just 6%, from an aggregate number of 2302 in 2004 and 2005 (pre-ERAS) to 2448 in 2006 and 2007 (post-ERAS).9 Benchmarked by these national data, our program attracted 32% (732 of 2302) to 45% (1092 of 2448) of the nationwide cardiology fellowship applications in the pre-ERAS and post-ERAS periods, respectively.

Analysis of applicant characteristics before and after ERAS participation revealed 2 other statistically significant differences. First, the number of applications from female candidates more than doubled, from 81 to 186, corresponding to a proportional increase of 6% (from 11% to 17%, P < .05). Second, candidates from a wider geographic area submitted applications, as measured by the mean distance from their internal medicine residency institution to GMC (from 420 ± 454 miles to 585 ± 559 miles; P < .05). Among all applications before ERAS, in 2004 the mean geographic distance from GMC was 392 ± 403 miles (range 0–2754 miles), and in 2005 it was 451 ± 502 miles (range 0–2754 miles). The mean geographic distances among all applications after ERAS participation were 484 ± 473 miles (range 0–2826 miles) in 2006 and 584 ± 639 miles (range 0–2826 miles) in 2007.

Analysis of data yielded, however, no significant differences among applicants to our program before and after ERAS in relation to the following: mean age (32 ± 4 years versus 32 ± 4 years); proportion of US citizens or permanent residents (63% versus 66%); proportion of allopathic (versus osteopathic) medical school graduates (93% versus 96%); mean years since medical school graduation (6.9 ± 4.1 versus 7.1 ± 4.3); and, where applicable, mean 3-digit USMLE scores for Steps 1, 2, and 3. Finally, the percentage of US medical school graduates did not differ (20% versus 17%), and the distribution of countries where respective applicants attended non–US medical schools remained basically unchanged before and after ERAS participation.

The present study describes the experience of a single cardiology fellowship program with the newly initiated ERAS application process in this subspecialty. It indicates that a significant increase occurred in both the total number of applications and the proportion of those applications containing all the required information. These observations are consistent with the general perception of cardiology program directors regarding the immediate impact of ERAS on the application process. The findings also replicate, in part, earlier observations made by an emergency department residency program.4 The reason for such an increase in the total number of applications submitted to a single program is not fully understood. It is speculated that the ease of preparation of a single application may encourage candidates to apply to a larger number of training programs and to a wider geographic area. In fact, based on the national data, the proportion of all cardiology fellowship candidates who have applied to our program increased from 32% to 45% after participation in ERAS. In 2006 and 2007, our program received an average of 546 applications for 4 positions, or 137 applications per position. During this period, the average national candidate pool was 1224 applicants.9 

We also noted an increase in the proportion of women applying to our program. In the absence of national data, the reason(s) for an increase in the number and percentage of female applicants remains uncertain.

The increased number of female applicants and the wider geographic area from which applications are filed may serve to foster diversity among trainees and provide a larger pool of applicants to choose from. This is especially true in view of the higher proportion of applications with complete information, without adversely affecting the quality of candidates; this is evidenced by the similar educational characteristics of the candidates (including USMLE scores, location of medical school, and years since medical school graduation). The latter finding is consistent with and extends the observations made in prior reports.4–6 However, the administrative burden of this sudden increase in the number of applicants, the need for educating program staff in the skills of handling the electronic application process, and the influence of the electronic application process on the final matching of the applicant to various programs need careful evaluation. In the future, it may become necessary to adopt strategies that help limit the number of programs that each applicant can apply to in order to alleviate some of the administrative burden of the training programs. In addition, training programs may develop predetermined selection criteria in order to encourage applications most suited to their training environment. Finally, direct verification of candidates' educational experiences by adding a questionnaire to the universal application may eliminate many of the uncertainties (such as meaningful research experiences) regarding the submitted applications.

In summary, we objectively investigated the immediate impact of participation in ERAS by performing a systematic examination and comparison of nearly 2000 applications submitted over a 4-year period. The data-extraction process we have followed is easy to use and reproducible. However, the data presents a snapshot of the ERAS application process for a single cardiology fellowship program at one institution. It is obvious that a larger sample of similar programs and institutions would provide more insight about the impact of ERAS. In addition, further studies are needed to evaluate the impact of available ERAS filtering mechanisms on the ease of streamlining the application process for individual cardiology fellowship programs.