Characteristics and Outcomes of an Innovative Train-in-Place Residency Program

Work-related injuries and illnesses, for which workers' compensation is the major payer, pose a significant burden to the United States. In 2013, approximately 3 million nonfatal workplace injuries and illnesses were reported in US private industry; more than half resulted in job transfer, time off work, or returning to work on modified duty assignment.^1,2  Workers' compensation costs ($62 billion in 2012³) are in the range of cost estimates for asthma, mental health, and cancer.⁴  Integrated workers' compensation management, for which occupational and environmental medicine (OEM) specialists are uniquely qualified, reduces costs and improves return-to-work outcomes.^2,5 

OEM is a division of preventive medicine that blends population health management and patient care, focusing on the care of injured and ill workers. Although OEM has a high satisfaction index⁶  and a robust market of available positions,^7,8  many physicians enter OEM midcareer, often unaware of its existence until after residency^9–11  and practice in another field.

OEM faces funding challenges. Programs have traditionally relied on funding from the National Institute for Occupational Safety and Health (NIOSH), the Health Resources Services Administration, the military, the Occupational Physicians Scholarship Fund, and other private sources. As these sources have dwindled, the number of Accreditation Council for Graduate Medical Education (ACGME)–approved OEM residency programs has declined from 38 a decade ago to 26, and one-third of positions are unfilled.¹²  The number of board-certified OEM physicians also has declined from 121 in 2003 to 76 in 2015.¹³  The American College of Occupational and Environmental Medicine projects a net workforce reduction of 891 physicians (33%) from 2015 to 2025 due to retirement, using a conservative assumption of 764 newly certified OEM physicians.¹⁴ 

OEM residency–trained physicians report their training is highly relevant to their practice.¹⁵  They are afforded more professional opportunities due to greater skill diversity, including population management, clinical, research, and leadership skills, than those not trained in the specialty.⁷  In contrast, physicians who make a midcareer shift to OEM without added residency training often feel hampered in the new professional role.⁷  At the same time, individuals interested in added training may be unable to terminate their current employment or take a leave of absence to initiate it.¹⁰ 

Our program was created in response to the Institute of Medicine's 2000 call to develop new routes to OEM certification, and focused on meeting the challenge of training midcareer physicians. Medicine lacks opportunities to change specialty following completion of residency,¹⁶  and training often is concentrated in larger metropolitan areas that may not be readily accessible to all.^17,18  Innovative methods, such as train-in-place education opportunities,¹⁹  allow physicians to train where they practice and live.

Our residency program uses a combination of supervised training in the community settings in which the residents in the program already are employed and intensive training at a tertiary institution.

The novel train-in-place track in our program has been in existence since 1997. Qualified applicants have completed at least 1 clinical year and have a Master of Public Health or equivalent degree, or a plan for completing such a degree before graduation, in keeping with ACGME requirements.²⁰  Residents in the train-in-place track work full-time as OEM physicians at an approved clinical training site, which provides for the range of required experiences for meaningful training and supervision by an American Board of Preventive Medicine (ABPM)–certified physician. Site adequacy is documented prior to admission. Program electives include the Occupational Safety and Health Administration, NIOSH, the Environmental Protection Agency, the Agency for Toxic Substances and Disease Registry, and corporate settings.

There are 2 interrelated components of the program: the applied component at the clinical training site and a didactic component that consists of monthly 3-day sessions at the University of Pennsylvania hospitals. There are 12 sessions during the first year¹⁹  and 5 during the second year. Residents are educated to achieve ACGME Milestones and competencies²⁰  and American College of Occupational and Environmental Medicine²¹  and program competencies (Table 1), and succeed on the ABPM–occupational medicine (OM) Examination. Special emphasis is placed on organizational health care management skills with strategy development and marketing, as OEM physicians may function in corporate and public sector positions^5,22 ; interpersonal and communication skills to effectively interact with labor, management, and industrial hygiene representatives; and population management and critical thinking skills to prepare graduates to address the population health needs of workers.^21,23  Faculty are ABPM diplomates and experts in their subject areas.

Residents develop an individualized educational plan in conjunction with faculty where they detail their plan for competency achievement. Elements of resident assessments considered include prior skills, experience, interests, results from assessment tools such as the annual In-service Examination, self-assessment against program competencies (n ∼ 150), the ACGME Milestones (since 2013), and evaluation of rotations and resources.

Didactic education is divided into 10 subject areas (Table 1), with 5 noncredit courses and several experiential learning experiences. First-year subject area rotations (2 months in length) provide approximately 30 hours of direct faculty contact. Each session culminates in an experiential mentored project presented to faculty and peers, which incorporates rotation-specific competencies (Table 2). Second-year subject area rotations (1 month in length) provide approximately 21 faculty contact hours (Table 1). The critical review of the literature course, designed to foster critical literature review and analytical skills, is taught using a standard format for article critique.²⁴  Along with the occupational epidemiology course, this lays the foundation for the research methods course where residents design and execute a research project during their second year. Required deliverables are an abstract presentation at a scientific meeting and a manuscript.²⁵  The clinical toxicology and clinical review series complete the course offerings. Residents present at monthly grand rounds using a structured format, which is offered via a web conference application to facilitate participation of remote faculty. Teaching sessions maximize discussion, interaction, and skill development.

Clinical training site faculty supervise residents, complete semiannual evaluations and chart reviews utilizing standard templates, help residents identify projects and gain stakeholder support, and provide formative feedback. Faculty conduct mentoring and auditing site visits (quarterly the first year and 3 times the second year), during which they shadow, mentor, and advise residents, provide 1-on-1 teaching, and meet with the site supervisor to formally discuss resident progress.

The Clinical Competency Committee uses assessment tools and a resident portfolio to determine milestone achievement. Assessments include direct observation of the residents' clinical skills and project presentations by faculty, chart reviews, resident self-evaluations, 360-degree evaluations including patient ratings, resident awards and achievements, educational plans, monthly logs of clinical training activities, and site visit reports. Formal feedback to residents is provided semiannually, with informal feedback given throughout.

Graduation requires satisfactory achievement of milestones and competencies. The Program Evaluation Committee (a subcommittee of the Residency Advisory Committee subcommittee) evaluates the program. Residents evaluate the program, faculty, and peers.

Three independent measures of program quality are used: resident self-assessments and whether the training met their needs, faculty evaluations of residents, and resident performance on the ABPM Examination.

A Diversity and Inclusion Committee, created in 2008, comprising residents and graduates seeks to increase diversity among residents. Initiatives include 1-day observerships for residents and medical students²⁵  and presentations at meetings.²⁶ 

The University of Pennsylvania Institutional Review Board declared this research exempt upon review.

Of 110 residents accepted into the external program between 1997 and 2015, 109 completed the program. Ten external track residents are currently in training. Prior to entering the program, 46% (50 of 109) were diplomates of other American Board of Medical Specialties: family practice (n = 24), internal medicine (n = 22), emergency medicine (n = 2), pediatrics (n = 1), and surgery (n = 1). Six matriculants had PhD degrees, and 106 (97%) had master's degrees. The establishment of the Diversity and Inclusion Committee helped increase trainees who were underrepresented minorities from 6% (4 of 69) to 18% (8 of 44).

The mean resident self-assessment score against program competencies completed before the program started was 2.98 of 5, and rose to 4.69 of 5 at program completion. Resident evaluation whether the program met their needs was 4.79 of 5 (N = 89; graduates between 2002–2015). Despite average or below scores on the national In-service Examination compared to those of other residents (Table 3), program residents scored, on average, above the mean of OEM residency–trained ABPM-OM examinees (Table 4). As the largest civilian OEM residency program in the United States, graduates of the train-in-place program have comprised 7.5% of new ABPM-OM diplomates over the past decade (2005–2015).

Training sites for current and past residents are distributed throughout all regions of the United States: Northeast (43 of 125, 34%), Midwest (24, 19%), Southeast (24, 19%), West (29, 23%), Southwest (3, 2%), and Hawaii and Puerto Rico (2, 2%). Most sites are located outside the 25 largest standard metropolitan statistical areas (86 of 125, 69%). More than 90% of graduates continue to practice at these locations for more than 5 years after graduation. Graduates work in academic, industry, governmental, and hospital-based and group clinical settings.

Successful implementation of this innovative train-in-place program has resulted in the training of a significant number of new board-certified OEM physicians. The acceptance of our model is evidenced by having all available funded positions filled each year, with a majority of residents enrolled in the external track. Graduates mentor and return to teach residents, and 19% (21 of 109) of external track alumni have served as site supervisors.

Although a significant time commitment is required by faculty to provide direct contact and personal interaction for external track residents, faculty enjoy the interaction and association with an academic medical center, resulting in a mutually beneficial relationship. Residents express satisfaction with the training and report competence on program completion based on their self-assessments. Training physicians in their communities renders graduates with superior skills for analyzing and meeting community needs. Most have remained in the field.

In comparison to traditional programs, superior performance on the ABPM-OM Examination, juxtaposed with comparable or slightly lower In-service Examination scores, emphasizes the importance of training in achieving these results. Geographic, ethnic, and cultural diversity of a critical mass of peers enhances trainee experience. Our model may allow a higher output of graduating residents per unit of grant dollars spent, as resident training sites remain a source of employment.

This intervention has limitations, including a lack of detailed assessment on performance measures after graduation. Published data on this measure, which would allow comparison of the train-in-place model to traditional programs, are limited. The ABPM-OM board pass rate is used as a surrogate. Another limitation is that we report data from a single institution, and the findings may not generalize to other institutions, particularly those with smaller faculty and resident numbers.

Next steps should include comparing long-term outcomes data to traditional programs, and determining the feasibility of implementing this model at other institutions.

Our novel train-in-place model overcomes barriers to access to training, resulting in increased numbers of OEM physicians prepared to meet the health care needs of the US workforce. This model has potential for replication.