Athletic trainers (ATs) are heavily involved in concussion assessment and return-to-play (RTP) decision making. Despite ATs' crucial role, few researchers have directly examined ATs' knowledge of concussions or whether concussion knowledge or clinical experience affects clinical concussion-management practices.
To determine the overall concussion knowledge of ATs and whether concussion knowledge and clinical experience affect concussion-assessment and -management practices.
Cross-sectional study.
Online survey.
Random convenience sample of 8725 (15.0% response rate [1307/8725]; certified, 14.8 ± 10.6 years) ATs surveyed from the National Athletic Trainers' Association membership.
The survey collected demographics, concussion-assessment and -management tools used, and concussion knowledge (patient-clinician scenarios, signs and symptoms recognition). We used multiple logistic regression models to determine the odds ratios (ORs) for using assessment and management tools based on signs and symptoms recognition and years of clinical experience.
The ATs correctly identified 78.0% ± 15.1% of concussion signs and symptoms. Approximately 46% (357/770) of ATs indicated an athlete could RTP if the athlete stated he or she had a “bell rung.” Every additional year of clinical experience decreased the odds of using standardized sideline-assessment tools by 3% (OR = 0.97, 95% Confidence Interval [CI] = 0.95, 0.99). The odds of using standardized sideline tools (OR = 0.98, 95% CI = 0.96, 0.99) and symptom checklists (OR = 0.98, 95% CI = 0.97, 0.99) for RTP assessment were significantly decreased for each additional year of clinical experience. No other tools used for RTP assessment were influenced by signs and symptoms recognition (P ≥ .136) or clinical experience (P ≥ .158).
The ATs with greater clinical experience had lower odds of using concussion-assessment and -management tools. Athletic trainers should frequently review and implement current consensus guidelines into clinical practice to improve concussion recognition and prevent improper management.
Athletic trainers had adequate concussion knowledge and familiarity with concussion-guideline publications.
Years of clinical experience may influence concussion assessment and management, with decreased optimal tool usage over time.
Athletic trainers should aim to frequently review current concussion guidelines in order to improve patient outcomes and their overall health care practice.
Certified athletic trainers (ATs) are among the first health care providers conducting concussion assessments and carrying out return-to-play (RTP) procedures. Athletic trainers are employed across all levels of sport and in many other job settings, with almost 68% of secondary schools in the United States having at least part-time AT access.1,2 Athletic trainers play a vital role in proper concussion management because they are heavily involved in diagnosis, management, and RTP decision making.3,4 Thus, it is imperative that ATs possess knowledge of concussions and remain up to date on concussion-assessment and -management techniques.
Concussion knowledge is critical for all health care providers. Failing to recognize concussion signs and symptoms or not understanding acute concussion effects may lead to a misdiagnosis or delayed diagnosis, a strong contributor to prolonged recovery.5,6 Symptom checklists have consistently been among the strongest diagnostic tools for concussion assessment7–9 ; thus, we suggest that the ability to recognize concussion signs and symptoms is essential for health care professionals. Previous researchers who examined concussion knowledge and practice gaps in other health care providers identified a variety of shortcomings. Only 26% of physicians used standardized concussion tools,10 medical residents failed to recognize up to 65% of common concussion symptoms,11 and chiropractors scored, on average, 27% ± 22% on a concussion-knowledge and -recognition quiz.12 Only Naftel et al13 assessed ATs' concussion knowledge, but the generalizability of their results to the entire athletic training profession was limited due to the small sample of 55 Alabama ATs. Despite investigations of other health care professionals' concussion knowledge, no authors have thoroughly addressed ATs' concussion knowledge or the influence of other clinicians' attributes on assessment and management.
How clinical experience influences concussion assessment and management is unclear. With ever-evolving recommendations for concussion management, it is important to understand how individuals integrate evidence-based medicine into practice. Previous researchers14 demonstrated a negative, though weak (ρ = −0.10), correlation between evidence-based practice knowledge and ATs' years of clinical experience. Similarly, a study15 of physical therapists indicated that individuals with <5 years of clinical experience were more than 24 times more likely to have learned about evidence-based practice than those with >15 years of experience. From this work,14,15 we can infer that those with fewer years of clinical experience are more familiar with and more likely to implement evidence-based practice than those with more experience. Concussion-management practices have evolved rapidly over the past 2 decades, receiving growing attention in athletic training education programs16 and potentially resulting in concussion-knowledge and -practice differences according to years of clinical experience.
Only 1 group has assessed the relationship between years of clinical experience and concussion-tool use. Kelly et al17 surveyed 610 National Collegiate Athletic Association (NCAA) Division I ATs on concussion-tool use at baseline, acute, and recovery time points and did not observe any practice differences based on years of clinical experience. These findings are valuable, but they provide limited insight into concussion practices because of the specific sample of Division I ATs, the 2011 data collection, and recently published surveys of AT practices.3,4,18 Therefore, the purpose of our study was to examine (1) the level of concussion knowledge among ATs and (2) the influences of concussion knowledge and years of clinical experience on concussion assessment and management among ATs. We hypothesized that (1) ATs would have a generally high level of concussion knowledge and (2) those with greater knowledge and fewer years of clinical experience would be more likely to assess more concussions annually and perform recommended assessment and management techniques.
METHODS
Participants
A sample of 8777 ATs from the National Athletic Trainers' Association (NATA) membership survey database were invited to participate. Of the 8777 e-mails sent, 52 were invalid e-mail addresses, which resulted in an initial survey sample of 8725. The survey was distributed in February 2018 and at that time, NATA-certified and student certified members totaled 33 410 of the 53 166 board-certified ATs in the United States (L. Northup, Board of Certification, written communication, July 2018).19 Participants from all potential work settings and districts in the United States (excluding individuals from Guam) were randomly sampled. The inclusion criterion was being either an NATA-certified or certified student member. Individuals were excluded if they were not ATs, did not consent to survey participation, or did not complete the primary concussion-knowledge measure (signs and symptoms recognition) survey component. Informed consent was provided by all participants before they initiated the online survey module. The study protocol was deemed exempt by our institutional review board.
Instrumentation
We created an online survey (Qualtrics Inc, Provo, UT) that asked participants about their demographic characteristics as well as concussion-assessment and -management tools used; it also allowed us to determine their concussion knowledge through specific patient-clinician scenarios and signs and symptoms recognition.20 The survey was part of another study aimed at examining the practice behaviors of ATs and is described in detail elsewhere.4 The survey took 15 minutes, on average, to complete, and all participant responses remained anonymous. This survey was modified from previously published surveys of concussion practices among ATs3,21,22 and consisted of numerous item-response types. The survey was validated by 4 content experts and piloted among 17 ATs to ensure function and clarity of the items.4
Survey display logic was used so that a series of items related to the initial item was to be answered only if a specific response was selected. For example, if individuals indicated that they used the Balance Error Scoring System (BESS)23 in practice, then future items inquiring about the BESS would appear. If a specific response for certain items was not selected, then the related follow-up items were not displayed. Due to the logical display functions used, the survey's item total could range from 23 to 52. Participants did not have to answer every item and were allowed to skip items.
The survey first asked participants about their demographic information and which methods they used to diagnose, manage, and determine RTP after concussion. The survey then assessed participants' concussion knowledge through patient-clinician scenarios (eg, “If a player reported postconcussion symptoms but appeared normal on standardized methods of concussion [eg, SAC, BESS, neuropsychological testing], would the athlete be allowed to return to play at your institution?” [SAC, Standardized Assessment of Concussion]) and signs and symptoms recognition. The latter section was based on a reliable, valid, and published survey (Table 1)20 and consisted of 20 items (8 true, 12 false) that asked ATs to indicate whether the listed sign or symptom was associated with a concussion. To ensure that the survey was reliable among ATs, we determined test-retest reliability using intraclass correlation coefficients (ICCs) for signs and symptoms recognition among the 17 ATs to whom the pilot instrument was administered. The second survey administration was completed 14.9 ± 2.2 days from the initial survey, and signs and symptoms recognition displayed high test-retest reliability (ICC [2,1] = 0.78, 95% confidence interval [CI] = 0.49, 0.92).24
Procedures
A cover letter, informed consent, and Web link directing participants to the survey was sent via e-mail from the NATA. The NATA sent follow-up e-mails at 3 and 6 weeks after the original communication between February and March 2018. Participants received follow-up e-mails even if they had already completed the survey; however, survey software features prevented individuals from completing the survey more than once. The survey was available online for 8 weeks total.
Statistical Analysis
During data analysis, we observed numerous cases in which ATs labeled all 20 item responses to the signs and symptoms recognition as false. Due to the survey design, if an AT was presented with the signs and symptoms recognition tool, the automatic response was false, and it was not possible to determine whether surveys in which all signs and symptoms were indicated as false were actual responses or were survey generated. To ensure that only signs and symptoms recognition scores entered by ATs were included, we removed ATs from analyses when the signs and symptoms recognition items did not have at least 1 of the 20 items indicated as true (n = 167). Participants were allowed to continue to the next survey item without responding to the prior item or to discontinue the survey at any time, resulting in items having different response rates.
Descriptive statistics were calculated for AT demographics, patient-clinician scenarios, and signs and symptoms recognition item responses. Signs and symptoms recognition responses were converted into composite scores by summing the number of correct signs and symptoms items and dividing by 20, resulting in a percentage-correct score.25 Signs and symptoms recognition was the main measure of concussion knowledge in our analyses. Numerous multiple logistic regression models were calculated to determine the odds (with corresponding 95% CIs) of using a specific assessment or management tool on the basis of the signs and symptoms recognition score (measure of concussion knowledge) and reported years of clinical experience. Multiple logistic regression model assumptions were assessed with no violations found. All multiple logistic regression 95% CI values not including 1.0 were considered statistically significant. All analyses were completed using The R Project for Statistical Programming (version 3.4.3; Murray Hills, NJ)26 with a priori α = .05.
RESULTS
A total of 1331/8275 ATs started the survey. Twenty-four individuals were excluded for either not consenting to participate (n = 12) or not being an AT (n = 12), resulting in an overall response rate of 15% (1307/8725), which was similar to that of previous AT survey studies.27,28 Approximately 63% (818/1307) completed more than 90% of their survey items. Of the 1307, a total of 773 ATs completed the signs and symptoms recognition section (demographics provided in Table 2). Only ATs practicing clinically who completed the signs and symptoms recognition section were included in the multiple logistic regression models (n = 633).
Concussion-Knowledge Factors Among ATs
The average signs and symptoms recognition score was 78.0% ± 15.1% (median = 85.0%, range = 40.0%–100%; Table 1). Approximately 46% of ATs stated an athlete would be allowed to RTP at their institution after having had his or her “bell rung” (357/770). The remaining patient-clinician scenario results are presented in Table 3. The ATs were most familiar with the 2014 NATA position statement on concussion29 (95.0%; 734/773), 2016 Berlin “Consensus Statement on Concussion in Sport”30 (59.9%; 463/773), and 2017 NCAA consensus on best practices of concussion diagnosis and management31 (56.4%; 436/773).
Influence of Concussion Knowledge and Clinical Experience on Concussion Assessment and Management
All ORs with corresponding 95% CIs are presented in Table 4. When the ATs assessed patients for concussion, the odds of using a clinical examination, signs and symptoms checklist, computerized neurocognitive test, and any balance assessment were not influenced by signs and symptoms recognition (P ≥ .136) or clinical experience (P ≥ .158). The odds of using standardized concussion sideline tools were affected by clinical experience (OR = 0.97, 95% CI = 0.95, 0.99), indicating that for every additional year of clinical experience, the odds of using sideline tools decreased by 3%. The use of a 2-domain or 3-domain multidimensional assessment battery (ie, using 2 or all 3 recommended measures of symptom recognition, balance, and neurocognitive testing) was not influenced by signs and symptoms recognition (P ≥ .695) or clinical experience (P ≥ .262).
When determining an athlete's readiness to RTP, the odds of using standardized concussion sideline tools (OR = 0.98, 95% CI = 0.96, 0.99) and a symptom checklist (OR = 0.98, 95% CI = 0.97, 0.99) were affected by clinical experience, with the odds of using standardized tools or symptom checklists decreasing by 2% for every additional year of clinical experience. The odds of using a clinical examination, signs and symptoms checklist, computerized neurocognitive test, and balance assessments were not influenced by signs and symptoms recognition (P ≥ .155) or clinical experience (P ≥ .213; Table 4).
DISCUSSION
This study is the first to assess ATs' overall concussion knowledge and evaluate how signs and symptoms recognition, a measure of concussion knowledge,20 and clinical experience influence concussion health care practices. Overall, ATs displayed adequate concussion knowledge as measured by acceptable signs and symptoms recognition, correct responses to patient-clinician scenarios, and familiarity with concussion-guideline publications. Despite these findings, the odds of using standardized sideline tools and graded symptom checklists with more years of clinical experience suggest overall improvements in evidence-based concussion management could be made.
Athletic Trainers' Concussion Knowledge
For concussion knowledge, ATs scored 78% on average in recognizing concussion signs and symptoms. Whereas direct comparisons to previous signs and symptoms recognition by ATs is not possible, the same survey we used was used to assess high school and collegiate athletes. The ATs correctly recognized slightly more signs and symptoms than high school (75%)32 and collegiate athletes (74%).25 This was a surprising and somewhat concerning finding because ATs are often tasked with diagnosing and managing patients with concussions. The graded symptom checklist, the tool from which the signs and symptoms recognition survey was derived, has consistently been cited as the strongest diagnostic tool for concussion,7–9 further suggesting the need for a thorough understanding of signs and symptoms among ATs. However, ATs taking the survey may have misinterpreted the specific directions (Table 1) to identify strictly concussion signs and symptoms and also identified signs and symptoms of more severe traumatic brain injury. This speculation may explain why symptoms indicative of intracranial bleeding, such as abnormal taste or smell, bleeding from the ear, and numbness or tingling of the arms, were correctly identified by only 44.6% to 61.8% of our sample (Table 1).
The majority of responses to the patient-clinician scenarios (Table 3) aligned with recommended clinical practice, suggesting that most surveyed ATs would recommend proper RTP decisions. Still, an alarming result was that 46.4% of ATs would allow athletes to RTP if their clinical examination revealed no abnormalities and no concussion signs or symptoms, but the player stated, “I'm fine, I just got my bell rung” (Table 3, item 3). Though a large percentage of ATs indicated an athlete would be allowed to RTP in this scenario, it is possible that respondents misinterpreted the statement and assumed standardized concussion-assessment tools such as the SAC or BESS were also being used. Regardless, idiomatic terms such as bell-ringer, ding, and cobwebs cleaned are often used in sport cultures to describe a transient period of altered mental status after an impact, followed by the rapid disappearance of the initial signs and symptoms. Several position statements29,33 specifically stated that athletes who described their injury in these terms should be removed from play and not allowed to return the same day. Health care providers should not use informal terms to describe a concussive event because such terms minimize the severity of the injury. However, a patient's use of these terms should be documented and the athlete removed from play for further assessment.
When asked whether an athlete would be allowed to RTP, ATs more frequently responded yes than in identical scenarios asking whether the AT would feel comfortable with the athlete returning to play (Table 3). These differences in item questions were emphasized to ATs in the online survey, making it unclear why ATs would feel less comfortable with what would be allowed for RTP at their institution. It is possible that certain stakeholders (eg, player, parent, coaches) previously identified in these survey data4 as being involved in the RTP decision could negatively influence an AT's decision; however, this is theoretical given that we did not assess why these differences occurred. Future researchers should explore why ATs' behaviors differ from expert consensus guidelines to identify the barriers to appropriate health care and improve athlete safety.
Athletic trainers' concussion management has improved over the last 15 years.3,4,21 This improvement may be credited to familiarity with recent consensus and position statements. Approximately 95% of the ATs in our sample were familiar with the 2014 NATA position statement on concussion,29 and 60% were familiar with the consensus statement of the 5th International Conference on Concussion in Sport.30 Only 69% and 49% were familiar with the previous 2004 NATA position statement on concussion33 and consensus statement of the 3rd International Conference on Concussion in Sport in 2013 (http://www.concussiontreatment.com/images/Zurich_Statement_2009.pdf), respectively.3 Athletic trainers should continue to use these freely available guidelines set forth by experts in the field.29,30
Effect of Clinical Experience and Concussion Knowledge on Practice
We are among the first to examine how clinical experience and signs and symptoms recognition, a measure of concussion knowledge, influence concussion-assessment and -management practices among ATs. Based on the findings, we suggest that the use of assessment and management tools was not significantly affected by signs and symptoms recognition, and the majority of ATs were not influenced by years of clinical experience (Table 4). However, for every additional year of clinical AT experience, the odds of using standardized sideline tools at initial assessment decreased by 3%, standardized sideline tools at RTP decreased by 2%, and symptom checklist at RTP decreased by 2%. Though these decreased odds may initially appear negligible, given the average years of clinical experience for the sample (12.6 years), the odds of using standardized sideline tools at initial assessment and RTP decreased by approximately 25% and 38%, respectively. The odds of using computerized neurocognitive testing at both assessment and RTP, however, were not significantly influenced; this may indicate that ATs with more clinical experience have been selectively choosing which recommended assessments to implement postconcussion; computerized neurocognitive testing and standardized assessment tools both became publicly available during the early 2000s.34,35 Furthermore, it is possible that consistent changes in sideline evaluation tools led to disparities among clinical ATs.
Our findings differ slightly from those of previous researchers17 who evaluated Division I collegiate ATs. Kelly et al17 did not observe any significant associations between the number of assessment tools used at baseline, acute evaluation, or RTP and years of clinical experience. Our results are similar except that the use of standardized sideline tools and a signs and symptoms checklist appeared to be influenced by years of experience. This discrepancy may be due to several factors: (1) our direct inquiry about specific tools used versus theirs about the total number of tools used, (2) our survey's assessment of ATs across all work settings versus their assessment of a Division I collegiate AT sample, and (3) differences in statistical models and survey design. Regardless of the differences between studies, it is important to consider how rapidly best practices in health care and specifically in concussion management change. Athletic trainers should continue to focus on reviewing current best practices and implementing multidimensional batteries (ie, signs and symptoms checklists, balance, neurocognitive testing) to ensure the highest diagnostic accuracy in their assessments.7–9
Limitations
The survey was subject to the response bias inherent in survey-based research. We assumed that participants completely understood each item and provided accurate and honest responses. To address any potential confusion about the survey questions, participants were instructed that items could be left unanswered and the research team's contact information was supplied. The survey response rate was low (15%), although it was comparable with that of recent surveys of ATs.27,28 Last, the conservative method of excluding responses in which all signs and symptoms recognition responses were false is a potential limitation; however, we believe this approach was more appropriate because it ensured that only responses entered by participants were analyzed.
CONCLUSIONS
Athletic trainers surveyed from across the nation displayed acceptable concussion knowledge as evidenced by appropriate signs and symptoms recognition, responses to patient-clinician scenarios, and familiarity with guidelines. We identified several areas for concussion-knowledge improvement, along with a need to better understand why ATs were more likely to allow individuals to RTP than to feel comfortable about the RTP when presented with patient-clinician scenarios. Signs and symptoms recognition, a measure of concussion knowledge, did not influence concussion-assessment or -management practices. Clinical experience affected concussion assessment and management, with the odds of using standardized sideline tools and symptom checklists decreasing slightly for each year of clinical experience. Athletic trainers should aim to frequently review current peer-reviewed literature and consensus guidelines29,30 to maximize positive patient outcomes.
