Costs and Benefits of Initial Certification for Emergency Medicine Residency Graduates

To become certified by the American Board of Emergency Medicine (ABEM), physicians undergo rigorous residency training followed by secure multiple-choice-question and oral examinations. Though the value of initial certification is generally not contested, some physicians in other specialties have questioned the value of maintaining or renewing certification.^1,2  Board-certified physicians have been shown to deliver improved quality of care and have fewer medical malpractice complaints.^3–13  Prior studies^14,15  have looked at preparation for the ABEM recertification examination and the career and learning benefits of staying certified. However, the value of initially achieving certification in most specialties, specifically emergency medicine, has been poorly defined.

This study sought to understand the perceived value of board certification by describing examination preparation costs and the benefits of becoming board certified.

In this descriptive, cross-sectional study, a survey was administered to physicians taking the 2018 spring and fall ABEM Oral Certification Examinations (OCEs). The spring OCE was administered May 5–7, 2018, and the fall OCE was administered October 13–15, 2018. The anonymous, voluntary survey was available to physicians during the OCE administration in paper-and-pencil format, which physicians dropped into an on-site sealed box. The de-identified survey responses were manually loaded into a secure database at ABEM headquarters. Response frequencies were calculated (table).

This study focused on costs entirely under the control of applicants, specifically study courses and preparation materials, in contrast to nondiscretionary costs such as examination fees and travel arrangements. Survey instructions deterred respondents from monetizing their preparation time when estimating costs. This approach was similar to that of a prior study,¹⁴  which identified examination costs as the costs of attending in-person preparation courses as well as using online examination preparation courses or written and online preparation question banks and testing manuals. Respondents also were asked about the time they spent studying for the examination after residency.

Validity evidence for survey questions was derived from expert panel development, psychometric review, and prior use, as well as refinement of a similar postexamination survey. Specific questions about examination preparation and benefits have been used for the past 5 years. There was a variable number of nonresponses to individual questions, and frequencies were calculated based on the number of actual responses for each question. Questions about potential benefits could receive multiple affirmative responses. The best way to determine if physicians perceived no benefit was to have them check a negative response option (ie, “I did not derive any benefit from preparing for and taking this examination”).

This study was reviewed and received an exception for human research by the University of Pennsylvania Institutional Review Board.

Of the 2016 physicians who took the 2018 OCEs, 38% (599 of 1565) spent more than 30 hours in preparation, and another 28% (438) spent between 21 and 30 hours. Regarding the expenses excluding preparation time, 21% (328) spent nothing to prepare for the examination, 50% (776) spent less than $1,000, and 2% (38) spent more than $3,000. Nearly 1 in 5 test takers took an in-person examination preparation course.

Most physicians (80%, 1254) reported a learning benefit to preparing for and taking the OCE: 49% of physicians (765) reported that preparing for the examination reinforced their knowledge of emergency medicine, 44% (689) reported that it increased their knowledge, and 30% (473) reported that it made them a better physician. No learning benefit was reported by 20% (311) of respondents. The majority of physicians (92%, 1442) reported that becoming ABEM certified provided career benefits, the most common being more career opportunities (69%, 1076), followed by other physicians viewing them more positively (61%, 962) and receiving a higher salary (26%, 411). No benefit from ABEM certification was reported by 8% (123) of respondents.

Physicians spent a substantial amount of time preparing for their certifying examinations. Most physicians spent less than $1,000 on preparation materials and activities. Physicians identified the primary benefits as having greater career opportunities as well as reinforcing and learning new medical knowledge.

The value of initial certification remains undetermined. Discussions about the cost of certification¹⁶  and benefits of board certification¹⁵  have often been based on isolated analyses. Similar investigations^14,15  to the current study explored the time, cost, and benefit to continuing certification in emergency medicine. In those studies, one-third of physicians took an in-person board prep course, and 70% spent over 35 hours studying for the examination.^14,15  A learning benefit was reported by 90% of physicians and a career benefit by 91%, the most common response being more job opportunities.¹⁵ 

The perceived value of board certification is likely to be influenced by examination fees, cost of test preparation, and amount of time spent preparing for and taking the examination, as well as any learning and career benefits. The ABEM's examination fee was the same for all physicians: $420 application fee, $960 written examination (qualifying examination) fee, and $1,225 oral certification examination fee (total $2,605). As the oral certification examination is held only in Chicago and lasts 5 hours, costs for travel and, if needed, a place to stay overnight varied greatly for physicians. This variability may have affected whether physicians purchased preparation materials or participated in preparation activities. The fluctuating costs of travel may also have affected physicians' perceived benefits from certification in ways that this survey could not assess. The cost that is most influenced by physician choice is for examination preparation activities and materials. Most physicians surveyed (71%, 1104 of 1565) spent nothing to less than $1,000 in preparation costs, while just 2% (38 of 1565) spent over $3,000.

Physicians spent a considerable amount of time studying for the examination, which represents an expenditure of personal resources. One model used for determining maintenance of certification costs monetized a physician's time dedicated to maintenance-related activities.¹⁷  Using this approach, 90% of recertification costs were attributed to monetizing a physician's time.

Direct financial benefits are difficult to determine. Fewer than 1 in 3 physicians reported a financial benefit to obtaining ABEM certification; however, salary surveys have reported a financial benefit.^18,19  Based on self-reported compensation data,¹⁸  it is estimated that in aggregate, ABEM-certified physicians receive an additional $260 million in annual total compensation compared with noncertified physicians. Most respondents (80%, 1254 of 1565) reported learning benefits from preparing for and taking the examination, while 20% (311 of 1565) did not. Few physicians reported that they would receive no career benefit from becoming board certified.

Among physicians who claimed a career benefit, more career opportunities received the highest frequency of responses, which was also the most frequently reported benefit by physicians seeking to stay ABEM certified. These findings were anticipated in that nearly all job postings for emergency department positions require board eligibility or board certification.

This study is limited by self-reports of cost, with the possibility of recall bias and social desirability bias (tendency to give answers satisfactory to the questioner), especially because the survey was administered during the examination. That respondents had just finished or were in the midst of an intense assessment experience may also have influenced their answers. Survey validity evidence was limited; thus, respondents may not have interpreted questions as intended. An ideal survey would have avoided negatively worded response options such as “I did not derive . . .” Travel costs were not determined, and it is likely that, for some participants, travel costs exceeded participation costs, with unknown effects on perceived value of certification. Finally, surveys were not linked to examination performance. A prior study¹⁵  found that higher-performing physicians perceived greater benefit to the initial examination and maintaining ABEM certification.

There are opportunities for further exploration, such as determining the average and range of travel and lodging costs. In addition, although value is subjective, physician interviews about perceptions of the value of initial board certification might provide a better assessment of perceived benefits.

Initial certification requires a considerable investment of time and money. Physicians seeking initial ABEM certification found both learning and professional benefits, with the most frequently reported being the reinforcement of medical knowledge and greater career opportunities.

Ultimately, the value of board certification is best defined by those who use certification as a proxy for professional capability. As certification boards seek to enhance the value of certification, physician perception is a critical index to measure. Future surveys can be improved to meet best practice standards to better capture this.