Improving Concussion Education: Do Athletic Trainers’ Opinions Match Expert Consensus?

This study was approved with exempt status by the University of Georgia Institutional Review Board. Between September and November 2022, the National Athletic Trainers’ Association (NATA) successfully distributed an electronic survey via email to a random selection of 9861 ATs registered as working in the SS or collegiate setting in the United States. Additional reminders were distributed every other week for a total of up to 6 emails. The email included a link to an electronic survey in which an informed consent was presented to all participants before beginning the survey questions. Participants were excluded if they did not consent, reported not working in the SS or collegiate setting, or were under the age of 18.

We created an online survey on Qualtrics, which was ultimately hosted and shared by the NATA. First, we collected demographic information (eg, age, sex, gender identity, professional setting) and then asked participants about their awareness of the NCAA-DoD Mind Matters Challenge consensus statement on improving concussion education. We first asked, “Are you aware of the consensus statement on concussion education from the NCAA-DoD Mind Matters Challenge?” Those who responded yes were asked if they had read the statement (“Have you read the consensus statement on concussion education from the NCAA-DoD Mind Matters Challenge?”). Next, participants separately rated the utility and feasibility of each of the consensus’s final 17 recommendations. The participants were then presented with a recommendation and asked to respond to the questions “Is it useful?” and “Is it feasible?” on a Likert scale ranging from 1 to 9, anchored as no (1) and yes (9), with 9 representing the strongest utility or feasibility. This question format was repeated for all recommendations. Utility was defined as, “If this practice were implemented would it meaningfully improve concussion disclosure?” Feasible was defined as, “Is this something that could reasonably be implemented at your institution?” The questions were presented in the same order to all participants. The wording for all recommendations can be found in Kroshus et al’s¹²  Supplementary Table 5 (see Supplemental Appendix, available online at https://dx.doi.org/10.4085/1062-6050-0486.23.S1).

We grouped participants based on their practice setting (ie, SS or collegiate) and ran descriptive statistics for the participants’ demographic information (eg, age, sex, gender identity) collectively and by setting (Table 1). To address our first aim, we summed the number of ATs in each setting who were aware of the NCAA-DoD Mind Matters Challenge consensus statement and had read the article. We ran a χ² analysis and calculated an odds ratio (OR) to compare rates between settings. To address our second aim, we compared participants’ mean utility and feasibility ratings for each recommendation to a ≥7.00 threshold to determine if AT ratings met or exceeded the minimum standards used in the consensus process.¹²  We also calculated median scores due to the nonparametric nature of the response data. To address our third aim, we used Mann-Whitney U analysis to compare between settings for each recommendation’s feasibility and utility ratings and calculated Pearson r effect sizes. Effect sizes were interpreted as low (<0.30), moderate (0.30–0.50), and large (>0.50).¹⁹  Data were analyzed using IBM SPSS Statistics (Version 27.0.00; IBM Corp) with an a priori α = .05.

We received responses from ATs representing Washington, DC, and all states except Alaska. Five hundred seventy-seven individuals consented to participate (5.9%), 515 (89.3%) of whom completed the demographics (age = 40.4 ± 12.8 years; 56.5% female sex; 53.2% female identifying) and at least the first question associated with aim 1 (Table 1). According to the Board of Certification (BOC) 2022 Annual Report, our sample’s demographics are like that of the certified AT population (57.2% female sex; 55.0% female identifying).²⁰  Our sample was slightly older; the BOC reports 68% of ATs are under 39 years.²⁰  Our uneven sample of AT settings (60.3% practice in the SS setting) is like the BOC report, in which 60.6% of ATs in our target groups (collegiate or SS) practice in the SS setting.²⁰  We also captured ATs who reported working with either collegiate or SS populations but reported a different primary work setting, such as hospital outreach or also serving primary school students. These individuals were excluded from aim 3, the setting-specific analysis (n = 5). The SS and collegiate ATs did not significantly differ in age (P = .082), sex (P = .772), or gender identity (P = .270). The question regarding gender identity included cisgender male, cisgender female, transgender male, transgender female, nonbinary, and options such as not listed, with a free-text option to allow for self-described gender identity. We asked collegiate setting ATs if the school represented a collegiate military program (eg, a Federal Service Academy); however, we did not have a representative sample to analyze this setting separately from other collegiate settings (n = 8).

One-third (33.4%) of participants were aware of the NCAA-DoD Mind Matters Challenge consensus statement, and nearly two-thirds (65.7%) of those aware had read the article. Collegiate ATs were more than twice as likely to be aware of the consensus statement than SS ATs (OR = 2.16, 95% CI = 1.48, 3.15; P < .001). Of all participants who were aware of the article, collegiate ATs were almost 2.5 times more likely to have read it (OR = 2.41, 95% CI = 1.25, 4.64; P = .008).

When evaluating aim 2, the ATs’ mean utility ratings were >7.00 for all recommendations, exceeding the minimum threshold set by the expert panel (Table 2). The overall participant utility and feasibility mean and median ratings for each recommendation are described in Table 2.

While participants unanimously agreed with the experts about utility, participants disagreed about recommendation feasibility for some items. The following 4 items—all requiring active collaboration with organizational stakeholders—were deemed unfeasible by virtue of a mean feasibility rating <7.00 (Table 2). Those 4 recommendations are as follows:

• Recommendation 6: “Actively collaborate with organizational stakeholders (including coaches/commanders, primary health care providers, athletes/service members, military chain of command) to select concussion education approaches that are engaging, interactive and that foster discussion” (mean = 6.70).

• Recommendation 14: “Provide opportunity for team members and coaches/leaders in the military chain of command to discuss and establish team values that are supportive of concussion symptom disclosure” (mean = 6.84).

• Recommendation 15: “Actively collaborate with organizational stakeholders (including coaches/leaders in the military chain of command, primary health care providers, athletes/service members) to identify and address organizational barriers to concussion symptom disclosure” (mean = 6.73).

• Recommendation 16: “Evaluate the effectiveness of institutionally selected concussion education approaches in changing athlete/service member concussion symptom disclosure behavior” (mean = 6.66).¹² 

Secondary school and collegiate ATs mostly agreed on the recommendations’ utility and feasibility. No significant differences in utility for 16 recommendations and no differences in feasibility for 10 were found. Mean group utility ratings and group differences are described in Table 3, and median utility ratings are described in the Supplemental Table.

No significant differences were found between groups for the recommendations that fell below the minimum threshold in aim 2 (P value range = .181–.453, r range = 0.04–0.07), although the collegiate ATs’ mean feasibility rating did exceed the minimum threshold for recommendation 14, to “provide opportunity for team members and coaches/leaders in the military chain of command to discuss and establish team values that are supportive of concussion symptom disclosure” (collegiate mean = 7.01, SS mean = 6.74; P = .181; r = 0.07).¹²  The only recommendation for which SS ATs reported significantly higher ratings than collegiate ATs was recommendation 11, to “provide easily accessible information to parents/guardians about how to support athlete/service member concussion symptom disclosure,” for both utility (P = .007) and feasibility (P = .002) but with small effect sizes (r = 0.14 and 0.16, respectively).¹² 

The collegiate ATs’ feasibility ratings were significantly higher than SS ATs for the following recommendations regarding educational content and delivery methods but with low effect sizes (Table 3):

• Recommendation 1: “Provide athletes/service members with education that addresses the potential dilemma individuals face when deciding to disclose a concussion (eg, tradeoffs, concerns about what might happen next, knowing how to report)” (P = .001; r = 0.16).

• Recommendation 2: “Provide athletes/service members with education that addresses short-term benefits of early concussion symptom disclosure (eg, athletic, academic, occupational)” (P = .006; r = 0.13).

• Recommendation 3: “Provide athletes/service members with education that addresses what is known about possible long-term manifestations of concussion and head injury” (P = .014; r = 0.12).

• Recommendation 5: “Provide athletes/service members with education that addresses site-specific information regarding institutional concussion resources and policies (eg, steps to take if an individual suspects they have a concussion)” (P = .022; r = 0.11).

• Recommendation 8: “Integrate messaging about the importance of complete concussion symptom disclosure throughout the recovery process” (P = .002; r = 0.16).

• Recommendation 13: “Provide athletes/service members with education that addresses the role they can play in encouraging peers to disclose possible concussion symptoms (eg, share evidence-based bystander education programming)” (P = .019; r = 0.12).¹² 

Despite low awareness of the NCAA-DoD Mind Matters Challenge consensus statement, our study generally showed that ATs practicing in the SS or collegiate settings found the 17 expert recommendations to be useful and feasible. Athletic trainers’ ratings met the utility thresholds for all 17 recommendations; however, 4 recommendations did not meet feasibility thresholds for both groups. Those 4 recommendations shared a common theme of requiring collaboration with stakeholders (eg, with identifying and addressing barriers to care-seeking, planning education). Some differences based on the institution’s academic level (SS or collegiate) existed. Athletic trainers in the SS rated the usefulness and feasibility of educating parents more highly than collegiate ATs. Collegiate ATs, however, had higher feasibility scores for recommendations about providing robust and ongoing education to the athletes than SS ATs.

Two years after the NCAA-DoD Mind Matters Challenge consensus statement was published, only one-third of participants were aware of the consensus statement on concussion education, and two-thirds of those who were aware of the statement read the article. We expected that a minority of ATs would be aware of and have read the consensus statement based on previous findings, in which only 28% of ATs thoughtfully read a research article in the previous 2 months.¹⁵  Evidence-based medicine improves patient outcomes, but a widespread implementation lag exists for translating research to clinical practice across the medical field.^13,21  As hypothesized, employing systematic dissemination practices outside of traditional research journals (eg, infographics, regional and local conferences, symposia) may be necessary to increase ATs’ familiarity with the NCAA-DoD Mind Matters Challenge consensus statement on improving concussion education. Such efforts should occur in partnership with ATs to understand their communication needs and preferences and should also address whether further guidance or support is needed for recommendation implementation.

As hypothesized for aim 2, ATs’ utility ratings met inclusion thresholds for all 17 recommendations. It is encouraging that the expert panel identified recommendations that ATs agree are useful across settings, since a common implementation barrier is when best practices are misaligned with clinicians’ needs or philosophies.¹³  Providing recommendations that the clinicians feel are useful will likely aid in their implementation.

Feasibility ratings also exceeded the threshold for 13 of the 17 recommendations. Feasibility ratings were lower than utility ratings for every recommendation, likely signaling that ATs support these initiatives but question whether they have the support or capacity necessary to carry them out. Consistent with the Capability, Opportunity, Motivation, Behavior behavior change theory, implementation barriers can arise at multiple levels. Prior researchers have found that 2 common obstacles that health care providers face in implementing evidence-based practice are organizational barriers (eg, the social and physical opportunity for gaining stakeholder buy-in) and lacking time or resources (eg, physical or psychological capability).¹³  Researchers have demonstrated ATs generally support evidence-based practice, but only 6.6% change their practice based on professional organizations’ position statements.¹⁵  It is critical to patient outcomes that ATs have sufficient time and resources to stay current with research and do their jobs properly; adequate staffing and organizational support may address this gap between expert-determined best practices that clinicians believe are useful and their capacity to carry them out.² 

As expected, ATs had lower feasibility ratings for recommendations requiring buy-in from other athletic or military stakeholders (eg, coaches or leaders in the military command, athletic directors), including recommendations involving identifying organizational barriers to concussion reporting, selecting educational interventions, and evaluating their effectiveness. Authors of previous studies have demonstrated stakeholder disengagement, in which athletes want their coaches to be more involved.¹⁴  We did not anticipate ATs rating discussing and developing team values as unfeasible. In practice, coaches may be responsible for leading team-level discussions, and ATs may be anticipating coach resistance to participating in this role. Literature that provides strategies for increasing stakeholder buy-in for ATs in the implementation of evidence-based guidelines is absent. However, literature from other fields like sport leadership or dissemination research may provide useful insight to identify opportunities and methods to improve institutional culture and foster value-clarifying discussions to increase recommendation feasibility and thus implementation.^22,23  The expert panel also suggested that organizations create a “multidisciplinary implementation team” in which different members of the organization can champion a movement for a more collaborative approach to improving concussion reporting.¹²  More broadly, further research is needed to understand barriers and facilitators to recommendation implementation across stakeholders (eg, coaches, ATs, administration). This information could inform adaptable strategies to assist all athletic department members with providing comprehensive education and employing appropriate organizational processes to support athletes in seeking care after a possible concussion.

Athletic trainers may also face barriers when changing their practice in a way that aligns with expert recommendations given conflicting demands in clinical practice. At the time of publication, the NATA alone provided 25 position statements on best practices for injury prevention and management, and other sports medicine associations and athletic organizations’ governing bodies provide many more. Athletic trainers have <10% compliance with best practices for life-threatening conditions (eg, heat illness, emergency action plans, and lightning safety), demonstrating that expert recommendations may not align with the realities of providing clinical care or are too difficult to locate and synthesize.^24–27  Sports medicine organizations’ position statements (eg, NATA, American Medical Society for Sports Medicine) and expert organizations (eg, Concussion in Sports Group) may be suitable for obtaining information on current scientific knowledge about concussions and best practices. Future researchers should explore if similar barriers to our findings exist between different consensus statements and ways to improve access and implementation.

In this study, we characterized some challenges ATs face with implementation of concussion education and identified opportunities for improving consensus processes. It is noteworthy that, while all panelists involved in developing these recommendations were familiar with athletics, military, or both settings, few were Board-certified ATs, and none were employed in a full-time clinical athletic training role while serving on the panel.¹²  The expert panelists’ general lack of clinical practice as ATs in the collegiate or SS setting may explain some of the gap between expert-assumed feasibility challenges and those reported by the practicing ATs in the present study for the NCAA-DoD Mind Matters Challenge recommendations. Non-AT stakeholders can also advocate for increasing resources and consequently feasibility in following the expert recommendations.

Our data showed that collegiate ATs were at least twice as likely to be aware of the NCAA-DoD Mind Matters Challenge consensus statement or to have read it than SS ATs. Considering SS ATs were not the target population of our research, additional dissemination toward ATs in the SS and other noncollegiate athletic settings is warranted to encourage equity in reaping the benefits provided by following these concussion education recommendations.

The sole utility discrepancy between settings was related to providing education to parents. While parents tend to be more physically present with SS athletes, collegiate athletes are nonetheless influenced by their parents when it comes to concussion care seeking and are an important group to engage.^28–30  This discrepancy was small and expected given the increased need for parental involvement with providing care to minors in the SS setting. Athletic trainers in the collegiate setting may also underappreciate the role parents or guardians may have in influencing concussion reporting behavior in collegiate athletes.

Our results show little difference in feasibility regardless of setting, demonstrating the experts’ ability to anticipate and overcome another common implementation barrier for best-practice statements: being designed too specifically for a single setting or circumstance.¹³  Varying resource levels and organizational structures exist across NCAA member institutions, which were considered in recommendation generation.¹²  Three of the 4 recommendations below the feasibility threshold in aim 2 remained below the cutoff in aim 3, the setting-specific analysis. We were largely incorrect in our hypothesis that the SS ATs would have more difficulty with organizational and team-level strategies (ie, recommendations 13–17). Secondary school ATs reported it was unfeasible to provide an opportunity for discussion about team values (recommendation 14), while collegiate ATs narrowly rated it as feasible, but no statistically significant difference between groups was found. Secondary school ATs are more likely to provide athletic training services to all sports rather than a single team, which may make it more difficult for them to provide an opportunity for discussions about team values. We expected lower feasibility for working with SS stakeholders to select educational strategies than college or university ATs, which was not supported by the data and may be the result of a widespread lack of stakeholder buy-in within the athletics and military communities regarding delivering concussion education.

Each group identified concerns with different recommendations. Secondary school ATs had higher feasibility than collegiate ATs for 1 of the 17 recommendations, which asked about the provision of “easily accessible information to parents/guardians about how to support athlete/service member concussion symptom disclosure.”¹²  We anticipated this feasibility difference given that, at the SS level, parents or guardians are presumed to reside with their dependent athlete and have higher levels of contact with a local educational institution than those of collegiate athletes.

We did not anticipate that the SS ATs would report low feasibility for most recommendations related to educational content and their ability to discuss honest symptom disclosure throughout recovery. Athletic trainers in the SS settings are commonly understaffed, providing athletic training services to an average of 515 athletes per school, and may find it more difficult to include all of the core educational content listed in the consensus statement.¹⁷  If they are not employed on a full-time basis or do not interact with their athletes daily after a concussion, they may not have the physical opportunity to implement some of the recommendations.

Secondary school interscholastic organizations, school boards, state legislation, or all of these may dictate minimum content for concussion education, and ATs can supplement such efforts with the expert recommendations’ content. Athletic trainers with limited time or access to provide formal education may, however, share educational material through posters or use informal time they have with athletes to have conversations about the topics recommended in the consensus statement (eg, the dilemma of reporting, short- and long-term concerns about concussion, athletes’ misperceptions). These differences in SS ATs’ ability to provide comprehensive, equitable education to their athletes is a call to action for schools to employ full-time ATs, improve staffing ratios, and provide sufficient resources to allow for easier provision of evidence-based care, including these recommendations, for increasing care seeking after concussion.

Athletic trainers can use a myriad of educational strategies to reach their athletes and other stakeholders, but as the experts acknowledged, structured support systems to help ATs navigate the challenges of delivering education and implementing these recommendations are needed, especially due to limited efficacy data.¹²  A centralized resource with access to educational intervention examples that target site-specific needs and resources for ATs is needed. Education should be delivered in different ways and multiple times throughout the year but should also remain appropriate for the circumstances.¹²  The recommendations were made for collegiate institutions and military service academies and may not be reasonable expectations for other practice settings. For instance, an AT who is not present at the school every day may not have the opportunity to provide education throughout recovery (recommendation 8).

This study had some limitations affecting its generalizability and applicability. By limiting distribution to members of the NATA, we may have excluded lower-resourced ATs who cannot afford or choose not to maintain membership. This survey was also distributed only to members reporting employment in the SS or collegiate setting to the NATA; ATs who work in outreach positions or on a per-diem basis or schools without an AT altogether may have different ratings. Due to anonymous distribution, we may have received multiple responses from the same institution. We may have suffered a response bias in which ATs who are more interested in providing concussion education were willing to participate in this survey study and may have provided higher utility and feasibility ratings. The ATs also may have considered the recommendations as individual items rather than in the context of performing them collectively in addition to their other clinical obligations and may feel differently if it was considered more holistically. It is also noteworthy that the recommendations were designed specifically for the college or university setting.

Few ATs practicing in a clinical setting reported having seen or read the NCAA-DoD Mind Matters Challenge consensus statement on improving education to support care seeking after a possible concussion. Athletic trainers have a primary role in providing concussion-related health education, so further dissemination efforts to this professional community are necessary. Athletic trainers generally agreed with the utility and feasibility of implementing the expert panels’ 17 recommendations for providing robust education; however, ATs deemed 4 recommendations unfeasible, all of which involved non-AT stakeholders taking an active role in delivering and evaluating interventions. Some setting-specific differences existed between SS and collegiate ATs, but none were significant enough to affect a recommendations’ overall utility or feasibility meeting the inclusion threshold. More research is required to identify solutions and improve implementation.

Supplemental Appendix. National College Athletic Association and Department of Defense Mind Matters Challenge Recommendations on Improving Concussion Education.

Supplemental Table. Recommendation Utility and Feasibility Median Ratings by Professional Setting.

Found at DOI: https://dx.doi.org/10.4085/1062-6050-0486.23.S1