Evidence for Economic Evaluations of Athletic Trainer Services

Studies that met the following criteria were included in the review: economic evaluation studies on AT services, original studies written in English, and studies published in the last 10 years (2011 to present). Because we aimed to identify evidence for economic evaluations that considered cost and benefit simultaneously, we excluded studies that examined either only the costs or only the benefits of AT services.

We performed a key word search in 5 databases: MEDLINE (FirstSearch), PubMed, Web of Science, SPORTDiscus, and Google Scholar. We used the following search terms, aiming at including a comprehensive list of economic evaluation studies: [economic evaluation OR cost benefit analysis OR cost-benefit analysis OR cost effectiveness analysis OR cost-effectiveness analysis OR cost utility analysis OR cost-utility analysis OR cost analysis OR return on investment OR return-on-investment OR incremental cost effectiveness ratio OR incremental cost-effectiveness ratio] AND [athletic train*]. We used athletic trainer OR athletic training in place of athletic train* if the truncated term was not applicable for that database.

We also conducted a search of references cited in the full-text articles identified in the database search. The same eligibility requirements were used for articles identified via cited references as for full-text articles from the database search. We also searched the reference lists of articles that were selected from the cited references search until no additional articles met eligibility criteria. In addition, we conducted a generic web search to identify any informal sources that might meet the eligibility criteria.

We extracted data using a predefined checklist of major components of economic evaluation based on the recommendation of the Panel on Cost-Effectiveness in Health and Medicine²⁰  including study perspective, study population, time horizon, comparison groups, economic evaluation used, cost and outcome measures, major findings, if a sensitivity analysis was performed, and optimal strategy findings. We also extracted data regarding athletic training practice setting, hoping to identify the value of AT services in a variety of settings (eg, high school, military).

The Figure illustrates the study selection process following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement.²¹  The initial search yielded 51 articles. Nine duplicate articles were removed. We screened the 42 remaining articles by title and abstract and excluded 37 because they either were not athletic training related or were not economic evaluations. We assessed the 5 remaining articles with full texts for eligibility and excluded 2 because the authors either did not report original results or did not assess athletic training services. The remaining 3 articles from the initial search were included.

In addition to the initial search, we identified 5 articles in our reference list search, 3 of which were excluded after full-text review because either no economic evaluation was performed or cost measures were not reported. We found no usable articles during the generic web search. Thus, our review consisted of 5 articles.^17,22–25 

Details extracted from the included articles are shown in Table 1. The practice settings of the ATs in the articles were 2 high school settings,^17,25  2 military settings,^22,23  and a community outreach program that consisted of high school, professional, semiprofessional, collegiate club team, and youth club team settings.²⁴  Regarding the study perspective, only Li et al¹⁷  clearly indicated a community perspective. The other 4 studies did not describe the perspective of their economic evaluation. Data sources varied by setting. Fisher et al,²²  Hirschhorn et al,²³  and Hambleton et al²⁵  used primary data collected from an internal source specifically for the AT intervention used. Slone et al²⁴  used an existing internal aggregate business analysis and differentiated between hospital and physician group billing when calculating revenue from AT referrals. Only Li et al¹⁷  used publicly available data. Specifically, they used state-level medical claims data from both public and commercial insurance sources. This allowed Li et al¹⁷  to examine the different economic effects of AT services by insurance status. All studies had timeframes of multiple years except for 1 that used data from a single academic year.²⁵ 

Comparison groups consisted of AT access versus no AT access^17,23–25  or AT access in addition to the regular standard of care versus only the regular standard of care.²²  Overall, a lack of consistent economic evaluation methods in the literature existed. Authors of the studies used a variety of outcome measures for the effectiveness or benefits of athletic training services. This was primarily due to their focus on ATs' effects on different aspects of health care and in different practice settings. Examples include the potential effects of AT services on health care affordability (eg, cost savings^22,25  or savings in claim payments¹⁷), health care utilization (eg, new referrals,²⁴  treatment provided,²⁵  or revenue from community outreach²⁴), and prevention of adverse events (eg, reduction in workplace injuries and missed training days^22,23). Investigators also measured cost based on different data sources, including the Bureau of Labor Statistics for average salaries^17,25  and internal sources for salary, supplies, and AT program costs.^22,24  Finally, researchers used different measures to report their findings. Li et al¹⁷  conducted a cost-benefit analysis and calculated the return on investment (ROI) as benefits associated with the cost of AT services. Slone et al²⁴  performed an internal quality assurance and economic viability audit, whereas Hirschhorn et al²³  carried out a secondary ROI analysis. Authors of the other studies^22,25  reported cost savings. Although sensitivity analysis is an important component in economic evaluation,²⁰  only Li et al¹⁷  used this method with 95% CIs in potential savings in claims payments and AT salaries as their sensitivity variables.

Although the authors used various methods to assess the value of AT services, most did support the use of AT services as economically viable. Slone et al²⁴  found that their AT outreach program generated referrals, billable patient encounters, and positive revenue serving various community sports programs. Hambleton et al²⁵  identified modest cost savings and suggested ATs could positively influence school and family insurance premiums. In the military setting, AT services were shown to be beneficial, generating a return of $9.48 for every $1 spent by reducing soldier attrition²³  and resulting in significant cost savings via reduced recruit attrition and missed training days.²² 

However, the findings may differ when taking insurance status into account. Among the studies we reviewed, only Li et al¹⁷  examined the economic effects of AT services by insurance type. Li et al¹⁷  conducted a cost-benefit analysis using medical claims data and specifically focused on insurance status. Their findings were mixed in that AT services were associated with reduced payments for patients who had commercial insurance but with increased payments for those who had Medicaid. Li et al¹⁷  shed light on the importance of assessing ATs' effects on health care when considering individuals' insurance coverage.

Economic research is essential to advance the athletic training profession and was prioritized in the research agenda by the Athletic Training Strategic Alliance.¹⁸  Despite the importance of economic evaluation in many areas of clinical and population health,^26–28  literature on economic evaluation of AT services remains limited. Our systematic review identified 5 articles on this topic.^17,22–25  Although secondary schools are a major employer of AT services,²⁹  we located only 2 economic evaluation studies in the secondary school setting.^17,25  Several professional health care organizations have endorsed ATs as coordinators of athletic care and pushed for AT availability in every high school athletics program to ensure the safety of young athletes.^30,31  High school ATs are employed via a variety of practice models, including hospital and clinical outreach programs and direct employment by schools. Li et al¹⁷  suggested that these different models may influence the cost-benefit of AT services. Understanding the economic effect of AT services and various AT practice models is critical to informing policy regarding the most cost-effective way to expand AT availability in secondary school settings.

We will summarize several major challenges and key areas for improvement regarding the status quo of economic evaluation of AT services. Furthermore, we will make specific suggestions to remedy the current limitations and improve future research, aiming to advance our understanding of the value of AT services. We aligned our recommendations and examples with the SSVM created by the NATA Secondary School Athletic Trainers' Committee, the Committee on Revenue, and the Committee on Professional Advancement, which also aimed to “educate [ATs] on the importance of building a positive perception in the value of their services.”¹⁹ 

First, the biggest challenge to understanding the value of AT services from the current literature is not only the limited number of economic evaluation studies but also the lack of comparability among these studies. Comparability is essential for economic evaluation studies to support decision making among alternatives. The lack of comparability across economic evaluation studies usually results from unclear perspectives, inconsistent cost and outcome measures, and a lack of standard methods for handling uncertainty,³²  which lead to difficulties comparing the current research on AT services. It is important for future authors to use a standard set of analytical and reporting practices to conduct economic evaluations.

To address this challenge and to improve comparability of future research, we adapted general recommendations for conducting and reporting economic evaluation studies in health and medicine^20,26  and provided specific examples for applying these standards in studies of AT services (Table 2). Authors should explicitly describe the perspective in their study. This is critical in economic evaluation because perspective determines viewpoint and standing, such as what costs and outcomes are included, how long a period is analyzed, and what criteria are used for interpreting the results. For instance, studies from a societal perspective should include all costs and outcomes.^28,34  In the case of AT studies, this means evaluating the effects on all stakeholders influenced by AT services, such as athletes, parents, and other caregivers; schools and employers; physicians and hospitals; and public and private insurance programs. Although the societal perspective is widely recommended by guidelines,^26,28,32,35  it is often difficult to implement and easy to omit important elements.^26,36  Therefore, other, narrower perspectives can also be used to address specific questions and inform decision making in specific contexts.²⁶  This results in various types of outcomes and cost measures. Specifically for AT services, examples of effectiveness measures from a health care system's perspective may include the number of clinical encounters, number of emergency department visits, or number of hospitalizations,¹³  whereas measures from a consumer's (eg, athlete's or school's) perspective may include days of missing school or lost game days.¹⁹  Regarding cost measures, it is recommended to apply the concept of opportunity cost, which is “the value the resource could have produced if it were spent in its best available alternative use.”²⁰  Examples of cost measures can be investment in hiring ATs (eg, AT salaries and benefits) from an employer's perspective and may include transportation cost and productivity loss from a parent's perspective. The variety and complexity of effect measures highlight the importance of transparency in study perspectives.

The length of time horizon also depends on the study perspective. Although the time horizon should be long enough to track all relevant costs and outcomes, depending on the perspective, a longer time horizon may involve more elements of uncertainty.³⁴  To address uncertainty, authors should conduct and report sensitivity analyses.^20,26  All economic evaluations of AT services should include 1-way sensitivity analysis by adjusting key factors (eg, costs, benefits, disease prevalence, treatment success rates) one at a time.²⁰  This is a simple but powerful tool to identify aspects for which uncertainty could have a greater influence on the study findings and decision making. Researchers can also use multivariate sensitivity analysis, which changes the values of multiple factors simultaneously, to assess the overall uncertainty and test the robustness of the findings.²⁰ 

Second, we identify data limitations as a big challenge to economic evaluation studies of AT services. For instance, one of the most important outcomes of AT services is to prevent or decrease adverse events, such as preventable injuries, referrals, absenteeism in schools, and concerns of parents and schools. However, these avoided events related to AT practice are often not tracked by large data sources such as health records and medical claims and are not readily available for economic research. Authors often analyze limited data sources from a single location or case study, which involve only a few ATs or a short study period.^24,25  Such data limitations tend to not only reduce the generalizability of a study's findings but also limit the comparability across studies. In another example, the NATA's SSVM model¹⁹  suggests that AT services can generate value by preventing unnecessary diagnostic testing because of provider training and skills in evaluation and communication. Unfortunately, these AT services that improve diagnosis in a cost-effective way are often not fully reflected in data, and therefore, the value of AT services tends to be underestimated.

Addressing the data limitations in AT studies requires tremendous input to build up an AT-specific data infrastructure that can identify, track, and reflect the effects of AT services accurately, comprehensively, and longitudinally. Future investigators performing secondary data analyses should consider using data that provide more detailed information on AT-relevant medical services and include broader populations across years.^13,17  More importantly, it is imperative for all ATs to proactively collect data on their work and contribute to the AT-specific data infrastructure. As the NATA guidelines^19,37  pointed out, ATs should apply best practices to data collection and standard documentation of their practices. Data logs and statistics include injury evaluations, communication logs, treatment logs, hours worked, and notes for coaches, parents, and nurses.¹⁹  It is crucial to recognize that, although tracking and documenting these data may cost time and add to an AT's workload, only if this AT-specific data is documented and available for economic research can the value of the athletic training profession be better measured and articulated. This data collection can also have short-term benefits for ATs by demonstrating their value to relevant stakeholders in their individual practice settings. The aim of advancing evidence of AT value through improved data infrastructure will not be achieved by any single stakeholder. It demands multidisciplinary collaboration among clinicians, researchers, educators, and public and private payers, who will all eventually benefit from improved understanding of the cost-effectiveness of AT practices.

Finally, we advocate for expanding future economic evaluations to more AT settings domestically and globally. All studies included in our review were from the United States. To ensure that our search process did not exclude international studies that may have used the athletic training equivalent term in other countries (ie, “athletic therapist”), we repeated the search process using the terms athletic therapy OR athletic therapist and athletic therap*, but no additional articles met our inclusion criteria. In the last 20 years, efforts have been made to establish athletic training and athletic therapist education programs and professional bodies outside of North America.³⁸  Research on economic evaluation of AT services could provide useful information to help guide the expansion of AT services internationally.

Economic evaluation could also inform and support decision making to expand AT services into nontraditional or emerging practice settings. For example, the military is considered an emerging setting for ATs.³⁹  We found 2 studies^22,23  that supported the use of AT services in the military. As NATA suggested, “regardless of setting, it is critical that [ATs] be able to quantify and articulate their worth and value.”¹⁹  Economic evaluation can be a useful tool for understanding the ROI when investing in AT services and providing evidence to expand into new practice settings.

In summary, despite some challenges in and limitations of the current economic evaluation studies on AT services, in this review, we still shed light on the important influences ATs can have in health care and the various settings in which they can improve the value of care. It is important that standard analytical and reporting practices be used to increase comparability and generalizability across studies and to better support policy and decision making to invest in AT services. Future investigators should consider patients' insurance status when studying AT services.^13,17  It may help to answer whether ATs can improve the value of care for underinsured and underserved populations because of their unique position in health care systems. We also advocate for multidisciplinary collaboration to improve data infrastructure for AT practices, which is imperative to improve economic evaluation studies of AT services and to advance the athletic training profession.

The NATA has prioritized research to demonstrate the value of AT services.^18,19  With AT employment projected to grow 23% by 2030,⁴⁰  economic evaluation research can support policy and decision making to increase investment and expand AT services. Although studies in this area were lacking, our findings suggest that AT services have positive economic effects. Research is needed to improve the evidence via standard methods and collection of more AT-specific data.