Version 5 of the Sport Concussion Assessment Tool (SCAT5) was released in 2017 with an additional 10-word list option in the memory section and additional instructions for completing the symptom scale.
To provide reference scores for high school rugby union players on the SCAT5, including immediate memory using the 10-word list, and examine how age, sex, and concussion history affected performance.
Cross-sectional study.
Calgary, Alberta high schools.
High school rugby union players (ages 15–18 years) participating in a 2018 season cohort study (n = 380, males = 210, females = 170).
Sport Concussion Assessment Tool 5 scores, including total number of symptoms (of 20), symptom severity (of 132), 10-word immediate memory (of 30), delayed memory (of 10), modified Standardized Assessment of Concussion (of 50), and balance examination (of 30).
The median number of symptoms reported at baseline ranged from 5 to 8 across sex and age stratifications. Median symptom severity was lowest in males with no concussion history (7; range, 0–28) and highest in females with a concussion history (13, range = 0–45). Median total scores on immediate memory were 2–3 (range = 0–4) for males and 21 (range = 9–29) for females. Median total scores were 3 (range = 0–4) on digits backward and 7 (range = 0–20) on delayed memory (all groups). Based on simultaneous quantile (q) regression at 0.50 and 0.75, adjusted for age and concussion history, being female was associated with a higher total symptoms score (q0.75 βfemale = 2.85; 99% confidence interval [CI] = 0.33, 5.37), higher total symptom severity score (q0.75 βfemale = 8.00; 99% CI = 2.83, 13.17), and lower number of errors on the balance examination (q0.75 βfemale = −3.00; 99% CI = −4.85, −1.15). Age and concussion history were not associated with any summary measures.
The 10-word list option in the memory section reduced the likelihood of a ceiling effect. A player's sex may be an important consideration when interpreting the SCAT5 after concussion.
Despite changes to the instructions for the baseline symptoms assessment, the majority of high school rugby players reported symptoms at baseline.
Addition of the 10-word list to version 5 of the Sport Concussion Assessment Tool eliminated a ceiling effect for memory scores in high school rugby players.
This study informs the use and interpretation of the Sport Concussion Assessment Tool, version 5, in Alberta high school rugby players and indicates that reporting of symptoms is normal at baseline in this age group.
Concussions are a commonly occurring injury in athletes, especially those in collision sports.1 According to the 5th International Consensus Conference on Concussion in Sport (Berlin 2016),2 a sport-related concussion (SRC) was defined as a brain injury resulting from an impulsive force transmitted to the head. An estimated 3.8 million SRCs are sustained in North America annually,3 and 21% to 27% of injuries sustained in high school sports are SRCs.4
The Sport Concussion Assessment Tool (SCAT) was developed for use as part of the sideline and clinical assessment of concussion. It has been modified over the years with input from expert panels, systematic reviews, and conference attendees. In 2017, version 5 was released from the 5th International Consensus Conference on Concussion in Sport.5 The majority of the elements of the SCAT5 are similar to those of the SCAT3.5 Research suggested that the 5-word lists on the previous version (SCAT3) were being memorized and rehearsed by athletes.5 For example, Snedden et al4 reported that just over half of high school athletes administered the SCAT3 were able to recall all the words for the immediate memory test, and roughly a third were able to recall all 5 words for the delayed memory tests. To address this concern, a 10-word list option was added to the SCAT5 word-recall task to decrease the ceiling effect and preserve the original intent of this component.5 Additional modifications that took place between the SCAT3 and SCAT5 were detailed instructions regarding the symptom checklist, new return-to-school and -sport progressions, and a rapid neurologic screen.5
Current evidence is conflicting regarding the association between participant demographic characteristics and baseline reference scores on previous versions of the SCAT.6 For example, some investigators reported differences in symptom reporting, cognitive scores, and balance based on age, sex, and history of concussion, whereas others found no clinical differences.4,7–12 Furthermore, although many components on the SCAT3 remained unchanged, no reference data currently exist for the 10-word recall list in high school rugby players. Therefore, our objective was to provide SCAT5 baseline reference scores for rugby union players, including scores for immediate and delayed memory using the 10-word list, and to examine the associations between age, sex, and previous concussion history and SCAT5 scores in high school rugby union players.
METHODS
This cross-sectional study was a substudy of a prospective cohort investigation examining the risk of injury and concussion in high school rugby union players (ages 15–18 years). Trained research assistants administered the SCAT5 electronically at schools in the Calgary (Alberta, Canada) area as part of the preseason assessment. Participants were included in the study only if they were cleared for competition in high school rugby and had not sustained a concussion in the 4 weeks before testing. We chose the 4-week exclusion criterion to decrease the likelihood of capturing players still experiencing concussion symptoms despite clearance based on the fifth consensus statement on concussion in sport,2 which suggested the expected duration of symptoms in children with SRC was up to 4 weeks. Ethics approval was obtained from an institutional ethics board and all participating school boards. Parent or participant (in cases of mature minors who were 14 years of age or older and whom the school administrator and research team had assessed as capable of making a decision about their research involvement) consent and participant assent were required before the study.
Procedures
Seventeen rugby union teams from 6 high schools consented to participate in the cohort study. One high school hired external support personnel to administer their baseline assessments, and its athletes' data were not included in this substudy. Therefore, a convenience sample of 5 senior boys' teams (ages 15–18 years), 3 junior boys' teams (ages 15–17 years), 4 senior girls' teams (ages 15–18 years), and 2 junior girls' teams (ages 15–17 years) from 5 schools in 3 of the city's quadrants was eligible. All components of the SCAT5 were administered at the schools to groups of 3 to 6 students in available classrooms, gyms, hallways, or sport clinics via an iPad (Apple, Cupertino, CA) version of the SCAT5 created using Research Electronic Data Capture.13 The instructions for the SCAT5 provided by the Concussion in Sport Group were followed.5 Participants also completed a preseason assessment form that collected demographic information and medical history. To determine whether a participant had sustained a prior concussion, 2 questions were asked. The question from the preseason assessment form asked, “Have you ever had a concussion (either diagnosed or not) or been knocked out or had your bell rung?” whereas the question on the SCAT5 asked, “How many diagnosed concussions has the athlete had in the past?” A participant who responded that he or she had a concussion or suspected concussion to either question was considered to have had a history of concussion. All components of the neurologic screen were performed except passive cervical range of motion. The total time to administer the test was approximately 8 to 15 minutes per participant.
Analyses
Data were analyzed using Stata (version 14; StataCorp, LLC, College Station, TX). A descriptive analysis of all components of the SCAT5 was performed. Performance on each SCAT5 component (ie, total number of symptoms, symptom severity, percentage of normal, cognitive screen, neurologic screen, balance examination) was stratified based on prior concussion status, sex, and age. Continuous summary scores were explored graphically for normality. When the data were skewed, we provided the median and range of scores. To allow for comparison with previous studies,4,10,14 the means, standard deviations, medians, and ranges, as well as proportion reporting no symptoms, were calculated to summarize outcomes related to the total symptom score and total symptom severity score. Potential group differences (history versus no history of concussion) in the proportion of rugby players reporting specific symptoms were examined graphically for males and females separately. Specific symptoms with greater than a 10% difference between groups reporting the symptom were highlighted with 95% confidence intervals (CIs). The median and range, percentage correct, and/or percentage testing negative were calculated and reported for the cognitive and neurologic screening sections of the SCAT5. Reference ranges are presented based on the sample's distribution. Specifically, percentile cutoff values were at cut points previously used for professional hockey15 and rugby11 players (below and above average cutoffs at the 25th and 75th percentile ranks, unusually low or high scores at the 10th and 90th percentile ranks, and the extremely low or high scores as close as possible to the second and 98th percentile ranks). Similar to previous studies, the classifications were in the direction of what would be considered a worse score for the specific outcome measure. For example, the extremely high cutoff for symptom score would be at the 98th percentile rank and extremely low total Standardized Assessment of Concussion (SAC) score would be at the second percentile rank. Simultaneous quantile regression models with bootstrapped standard errors (1000 repetitions) at the 0.50 and 0.25/0.75 quantiles (median and below/above average cutoffs, respectively) were used to assess the associations between SCAT5 summary outcome scores (ie, total number of symptoms, symptom severity, modified SAC score, total number of errors on the modified Balance Error Scoring System) and age, sex, and history of concussion. Associations at the 0.90/0.10 and 0.98/0.02 quantiles were not examined because of small sample sizes at the extreme ranges. Alpha was set a priori at .05 but adjusted for multiple comparisons based on the number of independent tests (P = .05/4 = 0.01).
RESULTS
A total of 388 student-athletes (males = 212, age range = 15–18 years; females = 176, age range = 15–18 years) from 14 high school teams completed SCAT5 baseline testing at the beginning of the rugby season. Six players were excluded from the analysis because they had sustained a concussion within the previous 4 weeks, and 2 players were excluded because they had other musculoskeletal injuries at the time of testing that precluded them from participating in rugby. Table 1 provides descriptive information for included participants (n = 380) stratified by sex. The reports of concussion history on the preseason baseline questionnaire and SCAT5 were concordant in 361 of the 380 participants (95%). A total of 248 of 380 rugby players (65.26%) reported no history of concussion, and 132 of 380 (34.74%) stated they had at least 1 previous concussion. A greater proportion of females than males gave a history of diagnosed depression, anxiety, or other psychiatric disorder. A greater proportion of males than females described an attention-deficit disorder or attention-deficit/hyperactivity disorder diagnosis. Approximately 61% of males and 56% of females reported prior experience participating in rugby.
Symptom Scale
A summary of the total number of symptoms and symptom severity reported on the SCAT5, stratified by sex, concussion history, and age, is provided in Table 2. One male did not complete a rating for the symptom of trouble falling asleep, and thus, the total number of symptoms and symptom severity score were removed for that participant. The median number of symptoms endorsed by participants at the time of baseline testing ranged from 5 to 8 across sex, concussion-history, and age stratification. The top symptoms reported by males at baseline were fatigue (124 of 210, 59%), difficulty concentrating (122 of 210, 58%), trouble falling asleep (97 of 209, 46%), and neck pain (96 of 210, 46%). The top symptoms reported by females at baseline were similarly fatigue (118 of 170, 69%), difficulty concentrating (113 of 170, 66%), trouble falling asleep (88 of 170, 52%), and, in contrast to males, feeling nervous or anxious (99 of 170, 58%). The proportions of participants reporting specific symptoms and the symptom severity scores are shown in Figures 1 and 2. Participants primarily rated their symptoms as mild. When compared with males without a history of concussion, males with a history of concussion had at least a 10% greater proportion reporting headache (difference = 10.3%; 95% CI = −3.2%, 23.9%), sensitivity to light (difference = 15.4%; 95% CI = 3.5%, 27.2%), feeling slowed down (difference = 11.2%; 95% CI = −1.2%, 23.5%), drowsiness (difference = 12.4%; 95% CI = −0.7%, 25.5%), and irritability (difference = 17.5%; 95% CI = 4.5%, 30.4%). For females, some symptoms were reported more often by participants with no prior concussion, including headache (difference = 10.1%; 95% CI = −6.1%, 26.3%), dizziness (difference = 15.4%; 95% CI = 2.4%, 28.4%), and irritability (difference = 10.9%; 95% CI = −5.3%, 27.1%), and 1 symptom was reported more often by females with a history of concussion (feeling like in a fog; difference = 10.9%; 95% CI = −3.4, 25.2%).
Percentage of males reporting specific symptoms and baseline and severity scores stratified by history of concussion. Lines visible within the mild and moderate categories represent the separations between symptom severity ratings of 1 and 2 and between symptom severity ratings of 3 and 4, respectively. Abbreviations: NC, no history of previous concussion; PC, history of concussion.
Percentage of males reporting specific symptoms and baseline and severity scores stratified by history of concussion. Lines visible within the mild and moderate categories represent the separations between symptom severity ratings of 1 and 2 and between symptom severity ratings of 3 and 4, respectively. Abbreviations: NC, no history of previous concussion; PC, history of concussion.
Percentage of females reporting specific symptoms and baseline and severity scores stratified by history of concussion. Lines visible within the mild and moderate categories represent the separations between symptom severity ratings of 1 and 2 and between symptom severity ratings of 3 and 4, respectively. Abbreviations: NC, no history of concussion; PC, history of concussion.
Percentage of females reporting specific symptoms and baseline and severity scores stratified by history of concussion. Lines visible within the mild and moderate categories represent the separations between symptom severity ratings of 1 and 2 and between symptom severity ratings of 3 and 4, respectively. Abbreviations: NC, no history of concussion; PC, history of concussion.
At the time of baseline testing, the median percentage-of-normal score for male participants was 95% (range = 60%–100%) for players with no history of concussion and 90% (range = 40%–100%) for players with a history of concussion. The median percentage-of-normal score for females with or without a history of concussion was 90% (range = 50%–100%). Only 74 of 248 players (29.84%) without a history of concussion and 29 of 132 players (21.97%) with a history of concussion reported feeling 100% of normal.
Cognitive and Neurologic Screening
The scores for the cognitive screen, neurologic screen, and balance examination are summarized in Table 3. Of the 380 participants, 289 (76.05%) answered all 5 SCAT orientation questions correctly and 98.42% answered at least 4 questions correctly. The most common error occurred on the question that asked the date, with 77 of 380 participants (20.36%) answering incorrectly. The median score on the orientation section was 5 across all groups. Median total scores on the immediate memory score (of 30) were similar for males and females with or without a history of concussion and across age groups. The median total score for digits backward (of 4) was 2–3 (range = 0–4), and for delayed memory (of 10) was 7 (range = 0–10) for all groups. Of interest, on the neurologic screen, 12.38% of males and 11.18% of females reported double vision with eye movement, and 9.05% of males and 9.41% of females were unable to perform a tandem-gait task. Median total errors made during the modified Balance Error Scoring System were slightly higher for males (6; range = 0–20) than for females (3; range = 0–13).
Factors Associated With Median and Cutoff SCAT5 Scores
The reference ranges on the SCAT5 measures based on cutoffs at the 0.75, 0.90, and 0.98 quantiles are shown in Table 4. Based on the quantile regression analysis, participants' sex was significantly associated with the total number of symptoms at the 0.75 quantile (βfemale = 2.85; 99% CI = 0.33, 5.37; P = .004), symptom severity at the 0.75 quantile (βfemale = 8.00; 99% CI = 2.83, 13.17; P < .001), and the total number of errors on the balance test at the 0.75 quantile (βfemale = −3.00; 99% CI = −4.85, −1.15; P < .001) when controlling for age and history of concussion. Age and concussion history were not significantly associated with any of the summary SCAT5 outcome scores (see the Supplemental Table, available online at http://dx.doi.org/10.4085/1062-6050-123-19.S1, for all regression coefficients from the quantile regression models). An additional stratification was added to Table 4 to reflect the findings from the quantile regression analysis.
DISCUSSION
Our study provided reference values for uninjured rugby union players on the SCAT5. New information regarding reference scores on the 10-word immediate memory and delayed word-recall memory tasks for high school rugby players is presented. Female high school rugby players were likely to report more symptoms and have higher symptom severity scores during baseline testing than male rugby players. Females also had a lower number of total errors on the balance examination.
Despite changes to the wording of the instructions on the SCAT5 symptom scale, the majority of uninjured high school rugby athletes still reported symptoms as part of their baseline assessment. The high number of symptoms in this sample highlights the importance of clinical assessment to determine the cause of the symptoms rather than relying on a presumption of an asymptomatic score to measure recovery. The median symptom score for males (6; range = 0–22) was much higher than the median symptom score reported by male professional rugby players (0; range = 0–22),11 potentially suggesting that level of play and age are important factors in addition to sport. Although we found no significant associations between age and performance measures of the SCAT5 total symptom scores or total symptom severity scores, this is likely due to the small age range of a high school–only sample. These results are consistent with those of other studies4,9 that have examined the effect of age on previous versions of the SCAT in a high school sample. Previous researchers12,16 observed an association between age and performance on earlier versions of the SCAT when comparing the differences across larger age ranges.
Headache, dizziness, and irritability were reported by more females without a history of concussion than by females with a history of concussion. One possible explanation for this finding is that some of the females without a history of concussion were tested at the time of menstruation or before and were experiencing premenstrual syndrome. However, information regarding stage of the menstrual cycle was not requested as part of this study, and thus, this explanation cannot be confirmed.
An interesting result was that only 27.11% of participants reported feeling 100% normal at the time of testing. To our knowledge, we are the first to report a summary statistic for this outcome. The low number of participants feeling 100% normal could provide insight into the challenges associated with the feasibility of obtaining scores that represent a true baseline value. Some of the reasons for not feeling 100% normal described by participants on the SCAT suggested that the variable stressors associated with being a high school student-athlete may have contributed to this finding. It is also possible that athletes who were healthy may not have understood this question. Further research connecting this subjective assessment of feeling normal to scores on the SCAT5, both at baseline and postconcussion, is warranted to better inform the utility of this question.
Our findings indicated that the addition of the 10-word list to version 5 of the SCAT eliminated a ceiling effect in high school athletes. In this sample, no athlete was able to attain a perfect score of 30 on the immediate memory test, and only 4% attained a perfect 10 on the delayed recall test. This is in contrast to a previous report17 on prior versions of the SCAT with 5 words demonstrating that 50% of high school athletes achieved a perfect score of 15 on immediate recall and one-third remembered all 5 words on delayed recall. The high school rugby athletes in this study performed similarly to the collegiate athletes in a recent study18 who scored an average of 20.57 of 30 (range = 9–29) on immediate recall and 6.59 of 10 (range = 1–10) on delayed recall.
One strength of this study was the use of quantile regression to understand how sex, concussion history, and age were simultaneously associated with concussion. This statistical model allows the examination of confounders across the distribution of outcome measures and is robust against nonnormality. Earlier authors tried to address the limitations associated with the typically skewed SCAT outcome measures using univariate nonparametric statistics, which can increase the risk of a type 1 error.19
This study did have some limitations that warrant consideration. First, the data were from a convenience sample of high school students participating on rugby union teams in Alberta, Canada. Although the sport is growing in popularity in Canada, many athletes are not exposed to rugby until high school. This was evident in our sample: 39% of males and 44% of females had no prior rugby experience, which may partly explain the lower proportion of athletes with previous concussions than in other high school rugby samples. Therefore, the generalizability to rugby players in other countries where rugby is started at an earlier age may be affected. Second, although the procedures outlined in this study were consistent with the current practice of baseline testing, the time of day when the test was done may have influenced the findings. Most of the testing occurred in the afternoon, after a full day of school. This may have resulted in more participants reporting fatigue on the symptom scale. Third, whether English was the athlete's first language may have also affected outcomes. Previous research18 in collegiate athletes demonstrated statistical differences in immediate and delayed memory between native English speakers and nonnative speakers. We did not collect data on, and thus, could not control for the native languages of the participants. Fourth, a potential source of measurement bias was the testing environment. Although attempts were made to minimize distractions and separate athletes being tested concurrently in larger groups (group size between 3 and 6 athletes), some distractions did occur. These distractions may have resulted in decreased scores on the memory and concentration components of the SCAT5. Finally, we explored potential associations between reference scores on the SCAT5 and sex, previous history of concussion, and age. History of concussion was self-reported and no concussion definition was provided. Hence, it is possible that the history of concussion was underreported or overreported. Furthermore, the SCAT5 scores may have been altered by other confounding factors.6 Premorbid conditions including attention-deficit/hyperactivity disorder, learning disabilities, psychiatric disorders, and depression may affect scores related to symptom reporting and concentration.6 It is worth noting that a greater proportion of females than males reported a history of diagnosed depression, anxiety, or other psychiatric disorder. Although these conditions may be more prevalent among females, another explanation is that females may be more likely than males to report medical symptoms to their physicians. Because of a limited number of participants with premorbid conditions, the power was insufficient to control for all factors. The decision to include the data of individuals with premorbid conditions in the analysis increases the generalizability to a population of high school athletes in Alberta.
CONCLUSIONS
Reference scores on the SCAT5 for high school rugby players in Calgary, Alberta, have been provided. Notably, the addition of the 10-word list to version 5 of the SCAT eliminated a ceiling effect for memory scores in high school athletes. Considering the sex of a player may be important when interpreting baseline SCAT5 scores. Further studies examining changes on the modified SAC postconcussion are needed to better understand the clinical utility of the tool and postinjury cutoff values.
ACKNOWLEDGMENTS
This work would not have been possible without the help of the teachers, coaches, and athletes who participated in the study and helped organize the testing sessions. We thank all of the research assistants and coordinators at the Sport Injury Prevention Research Centre for their help with data collection. The Sport Injury Prevention Research Centre is one of the International Research Centres for Prevention of Injury and Protection of Athlete Health supported by the International Olympic Committee. Dr Black was supported by a postdoctoral fellowship through the Integrated Concussion Research Program at the University of Calgary.
Two authors, Drs Schneider and Emery, were members of the scientific committee for the 5th International Consensus on Concussion in Sport. Dr Yeates was an expert panel observer at the Berlin meeting. Dr Schneider reviewed the SCAT5 before publication.