Evaluating Knowledge Attainment and Retention of a Multimodal Approach to Concussion Education in Collegiate Athletes

• Effective and efficient education should focus on the audience. Adult learning is best facilitated when the information is relatable to the audience's life experiences, geared toward changes in societal roles, and directly applicable to the audience's life.

• The active learning active reflection session appears to engage student-athletes to process and think critically about the information presented, which enhances perceived knowledge.

• As a required portion of preseason meetings, passive concussion education typically fails to improve knowledge and retention of concussion. Including an active learning active reflection session to concussion education might reinforce learning and enable athletic trainers to gauge athlete understanding of concussion information.

• Future research should investigate population-specific educational techniques that use active learning strategies across a multimodal concussion-education program to ensure engagement across all learning strategies and to enhance knowledge and retention in the collegiate setting.

In the last decade, concussions have been in the spotlight in the media and the sport world causing heightened awareness of the injury. However, the media's interpretation of concussion may be flawed, starting with the definition and incidence of concussion. The National Athletic Trainers' Association defines a concussion as a trauma-induced alteration in mental status that may or may not involve a loss of consciousness.¹  This broad definition increased the number of reported cases, with an estimated 1.6 to 3.8 million sport- and recreation-related concussions reported annually in the United States.²  However, many concussions still go unreported. Research reveals that some reasons athletes often underreport a concussion are that they believe the symptoms are not serious enough or they do not recognize they have a concussion.^2–6 

Previous studies have shown a lack of student-athlete knowledge in the definition of concussion, recognition of signs and symptoms, and possible long-term consequences from head injury.^2–5  Sefton et al⁶  reported 70% of their collegiate student-athlete population did not report concussions sustained in football because they were not aware of the signs and symptoms of concussion. A similar lack of knowledge has been reported in both high school and collegiate athletes, which has emphasized the need for better educational practices for student-athletes at both levels.^3,4  The most recent consensus statements have deemed concussion education as the cornerstone for concussion management, and it has been required by laws and mandates.⁷  As a result, multiple organizations have developed their own educational materials on concussion, even though^8–15  the laws and mandates do not state how the education should be delivered to the target audience.

An increasing number of educational resources are available for sport-related concussion and have been developed to include multiple modes of delivery. These modes of delivery are designed to foster learning across learning preferences such as visual, auditory, reading/writing, and kinesthetic.¹⁶  The most common mode of delivery for educational interventions is the didactic format, through meetings and lectures. According to Kroshus and Baugh,¹⁷  athletic trainers (ATs) delivered concussion education via this modality almost 78% of the time, read a written document almost 75% of the time, and showed a video 30% of the time. Unfortunately, many of these delivery modes are passive in nature, which has been the mainstay for teaching practices in early educational models as well as current educational practice.^17,18 

In contrast, active learning requires students to work on activities and reflect on their learning.¹⁹  Studies have researched active versus passive learning strategies and found participants had more positive attitudes towards the active teaching method, with participants feeling fully engaged in learning the material.^18,20–23  However, in one study,²²  no significant differences in knowledge scores were seen between participants in a passive and active learning environment. Active learning may be the best practice to facilitate learning and retention; however, the feasibility of creating an active learning environment that is time efficient and works across all learning preferences and audiences can be challenging. Organizations have tried to combat these challenges by developing various educational materials across groups of learners of different ages.

In developing concussion-education materials, learning activities must be tailored to the audience who is receiving the information and geared toward student-athletes, parents, coaches, and the community. The Centers for Disease Control and Prevention (CDC), along with other organizations, has produced a wide variety of concussion-education materials.⁸  The Centers for Disease Control and Prevention's Heads Up program (see https://www.cdc.gov/headsup/about/index.html), a program to educate individuals on concussions, is composed of educational videos, fact sheets, podcasts, online training modules, and mobile apps such as Symptom Shock.⁸  These resources are catered to middle and high school student-athletes, coaches, sport officials, and parents.⁹  As new educational resources are developed, research studies have examined the efficacy of programs and materials for different audiences.^4,9  The importance of providing population-specific educational materials has prompted other organizations to follow suit and create sport-specific educational videos like the US Soccer Federation's Recognize to Recover, a Concussion Initiative.¹⁰ 

As technology advances, more educational resources are readily available and provide an online learning medium for student-athletes. The CDC Heads Up program, the ThinkFirst Smart Hockey video (see https://youtu.be/41V8iLCRu6Q), and the Symptom Shock video game have been tested in high school populations,^11–13  but no such research thus far has been conducted in the collegiate setting. The National Collegiate Athletic Association's (NCAA's) concussion fact sheet caters to student-athletes, coaches, administrators, ATs, and team physicians in the collegiate setting.¹⁴  In addition, the NCAA has a concussion safety video for student-athletes.¹⁴  Most modes of delivery for concussion education in collegiate athletes consist of didactic lectures, informational videos, and the NCAA concussion fact sheet. When addressing the need for concussion education, previous research suggests using an educational video with athlete testimony on concussion may benefit student-athlete engagement during concussion education.¹⁵ 

Within collegiate athletes, the audience of interest is the adult learner. There is evidence that adult learning is best facilitated when information is relatable to the audience's life experiences, is geared toward changes in societal roles, and is directly applicable to the audience's life.²⁴  Therefore, as shown by Kroshus et al,¹⁵  including videos with athlete testimonies may cater to adult learners by an association of relatable experiences. Adult learning theories suggest that adults engage in challenging educational sessions, develop more complex thoughts, and have a better integration of knowledge, which could be fostered through the use of active learning strategies.^19,21,25  However, limited research is available to understand the best modes of delivery and strategies to enhance learning. Although some studies show no significant changes between active and passive learning groups immediately after learning the material, the active learning group demonstrated better knowledge retention when tested a week after the intervention.^19,21,25  However, specific examinations of concussion-knowledge retention are limited.

In addition to the consideration of how adults learn, examination of concussion-knowledge retention has shown inconsistent findings. Cook et al²⁶  evaluated the ThinkFirst Smart Hockey video and found significant concussion-knowledge improvements that were sustained after 3 months. Miyashita et al²⁷  conducted a study on the impact of a 20-minute PowerPoint lecture in collegiate men's and women's soccer and men's and women's basketball, finding significant improvements in student-athlete knowledge scores at the conclusion of each team's competitive season (about 6 months postintervention).²⁷  Further, in a follow-up study using a larger population of minor-league hockey players, the players had significant improvements in knowledge immediately after watching the ThinkFirst Smart Hockey video.²⁸  However, their improvements were not maintained 2 months later.²⁸  Each of these studies evaluated educational programs that included only one mode of delivery. Although the research is mixed on knowledge retention, none of these studies have used a multimodal approach to concussion education. Therefore, the purpose of this study was to evaluate knowledge and retention using a multimodal approach to concussion education in collegiate athletes. We expected that (1) a multimodal concussion-education intervention would increase knowledge immediately after the intervention, (2) the use of an active learning strategy would increase knowledge retention at approximately 3 months, (3) student-athletes would be engaged with the video and discussion, and (4) student-athletes would have a perceived increase in knowledge after the intervention.

This study used a sequential explanatory mixed-methods design. This methodology allows for the qualitative data to “explain or build upon initial quantitative results.”^29,30  To that end, quantitative results were analyzed first (phase 1), then interviews were conducted (phase 2) to help better understand the athletes' experiences as participants in the educational intervention. The third phase of quantitative data collection was approximately 3 months after the qualitative interviews to test knowledge retention.

A convenience sample of male and female Division I collegiate athletes from the following sports was used: all-female and coed cheerleading, men's and women's soccer, women's volleyball, men's and women's tennis, women's swim and dive, women's rifle, women's basketball, women's golf, men's baseball, women's track and field, cross-country, and women's softball. Two-hundred twenty-two student-athletes between the ages of 18 and 22 years completed the preintervention survey. Of the 222 who completed preintervention surveys, 142 completed the postintervention survey, and 58 of those individuals completed the retention survey at the end of the semester. One softball player quit the team halfway through the semester and was removed from the data set. Therefore, 57 participants were included in the final analysis (Figure 1).

Respondents for individual semistructured qualitative interviews were randomly sampled via a random number generator from student-athletes who completed both the preintervention and postintervention surveys. The random number generator was set to select 10 initial student-athletes across both groups (5 first-year students, 5 returners), with the plan to continue recruitment until saturation was obtained. If selected student-athletes declined, additional student-athletes were randomly selected to participate. Data saturation was defined as the repetition of key words and statements across numerous participants that supported the stability of identified themes and patterns.²⁹  Theme saturation was obtained after 4 coded transcripts in each group and additional recruitment was halted for a total of 10 respondents (6 freshmen and 4 returners).

Inclusion criteria included Division I student-athletes from an NCAA institution, incoming freshmen, transfer student-athletes, and returning student-athletes. Participants were excluded if they were under 18 years of age or if they were no longer associated with the institution's collegiate athletics (quit the team or transferred). All participants signed an informed consent form approved by the Human Subjects Institutional Review Board (IRB) before participating in the study.

The Rosenbaum Concussion Knowledge Index (RoCKI)^31,32  was used as a tool to evaluate the effectiveness of the concussion-education intervention. The instrument was used in preintervention, postintervention, and retention surveys. The RoCKI has previously been established to examine basic concussion knowledge such as the definition of a concussion, sign and symptom recognition, and short- and long-term effects of concussion.³¹  The RoCKI consists of 2 sections that contained 21 true or false questions. Each correct answer receives 1 point, for a maximum total of 21 points.^31,32  Higher scores on the knowledge index are associated with greater knowledge. Validity and reliability have previously been established in the collegiate population.³³ 

Qualitative semistructured interviews were developed to further explore student-athletes' perceptions of the intervention. Interview guides were developed according to the research questions and current literature and adjusted as necessary after expert and peer review as per standard qualitative methodology.^29,30  After review, 11 open-ended questions were used, addressing previous concussion education, type of concussion education received, perception of the intervention, and perceived knowledge (Figure 2 for a truncated interview guide).

The multimodal concussion-education intervention was developed in collaboration with the head AT, team physician, primary investigator, experts, and previously published research in concussion education.^{14,15,26–28}  The multimodal educational intervention consisted of 3 modes of education delivery: (1) traditional didactic lecture using a PowerPoint presentation, (2) video, and (3) active reflection session. The didactic PowerPoint presentation included the definition of concussion, reporting instructions, and new research followed by a review of the NCAA concussion fact sheet to support both the visual and auditory learner. Viewing the US Soccer Federation video Recognize to Recover, a Concussion Initiative provided an opportunity for the kinesthetic learner to engage in real-life scenarios observed in soccer athletes.¹⁰  A 10-minute active reflection session followed the conclusion of the video. This session provided an active learning opportunity and encouraged reflection. The head AT asked questions that required student-athletes to answer after reflecting on the recent information obtained. For example, one question that was asked by the head AT was, “So how many of you had more concussions than you initially thought? Do you think you had a concussion that was not diagnosed based upon the symptoms that were discussed today?” The total intervention took approximately 45 minutes to complete.

The primary investigator (L.C.S.) attended preseason meetings held during physicals and before the first practice sessions in July and August to explain the research study. Although the educational intervention was required as part of the preparticipation concussion education, it was explained that only student-athletes who consented (by signing the consent form) to participate would be asked to complete the surveys. Researchers and team staff were unaware of athletes' decisions to participate in the study. Questions regarding the study and participation in the study were answered before the concussion-education intervention. Once these questions were answered, student-athletes who consented to participate in the study were asked to complete the preintervention survey to obtain baseline concussion knowledge before their competitive season. Participants were deidentified and coded by using numbers within their cell phone to avoid any sense of coercion and ensure confidentiality. The preintervention survey was administered via a web-based portal, using Qualtrics 2015 software. All meetings were held in classrooms/meeting rooms with Wi-Fi access.

Consistent with the sequential exploratory mixed-methods design, quantitative data for this study were collected first (phase 1), using the RoCKI. The goal of the preintervention survey was to examine baseline concussion knowledge before the intervention. The preintervention survey was initiated after IRB approval during preseason meetings in July and August. After completion of the preintervention survey, the head AT met with each team and delivered the multimodal intervention. Phase 2 of data collection occurred up to 48 hours after the intervention, during which the postintervention survey was administered. The goal of the postintervention survey was to examine the effect of the intervention on concussion knowledge immediately postintervention.

Additionally, in the second phase of the study, qualitative interviews were conducted to examine athletes' perceptions of the intervention, after the completion of the postintervention survey. Ten randomly selected student-athletes who participated in the postintervention survey were interviewed approximately 1 month after the intervention regarding their perceptions of the intervention and concussion knowledge. The randomly selected respondents represented both men and women from different sports across 2 age groups (first year at the institution and returners). Individual one-on-one semistructured interviews were conducted by the primary investigator (L.C.S.) approximately 1 month after the intervention in a quiet location chosen by the participant. Each interview lasted approximately 1 hour. The respondents were made aware of the interview purpose, procedure, and audio recording of the interviews and they gave written consent to participate in the research.

The researchers acknowledge personal interest and direct involvement in concussion education of collegiate athletes. Although the primary researcher has a relationship with the student-athletes, they did not interview their own student-athletes to avoid conflict and potential bias. The involvement with student-athletes by the research team combined with the personal interest in concussion-education methodologies provided ample opportunity for the researchers to relate to participants in an effort to understand and provide the athletes a voice regarding the effectiveness of the intervention. The primary investigator (L.C.S.) transcribed the interviews verbatim and the research team triangulated themes regarding the student-athletes' perceptions toward the intervention and perceived increase in concussion knowledge of the athletes during the intervention.

The goal of the third and final phase (retention survey) was to examine knowledge retention over the course of a semester. During the last 2 weeks of the fall semester, the retention survey link was sent to each participant who completed the postintervention survey, via Qualtrics. This survey software allowed researchers to send reminders to participants who had not completed the survey. A reminder email was sent once per week for the last 2 weeks of classes if participants had not completed the retention survey. Data were collected through the first 2 weeks of December 2018, approximately 3 months after the intervention (Figure 1).

Descriptive statistics (means and standard deviations) were calculated using IBM SPSS Statistics for Windows, version 23.0 (IBM Corp). All data were examined for outliers. Further statistical assumptions of normality and homogeneity of variances were also examined. A repeated-measures analysis of variance was used to compare the participant's survey scores over 3 different time points (preintervention survey, postintervention survey, and retention survey). Included in this analysis was the examination of effect size via partial η²  and post hoc power analysis. Additionally, a 1-way analysis of variance with repeated measures was used to compare survey scores between freshman and returning student-athletes. The Tukey honestly significant difference test was used in post hoc analyses to examine differences over time. Alpha levels were set a priori at 0.05.

Semistructured interviews were audio recorded and transcribed verbatim by the primary researcher to ensure data integrity. Credibility and trustworthiness were established through member checks (completed after the transcription process) and use of a certified AT on staff as an auditor. After member checks, the transcripts were analyzed using open coding to identify emergent themes through inductive reasoning.^29,30,34  The primary investigator (L.C.S.) and one member of the research team (T.N.H.) evaluated the transcriptions and audio recordings for themes associated with (1) perceptions towards the intervention and (2) perceived increase in concussion knowledge of the athletes during the intervention. Responses were coded independently by the researchers based on existing literature. After initial theme development (n = 2), peer debriefing with the remainder of the research team (L.C.S., J.L., D.B., T.N.H.) (n = 2) enabled triangulation of themes and finalized analysis. Once themes were analyzed, representative quotes were chosen to display appropriate justification for themes and results.

All data were examined for normality with no violations observed. Participants ranged from age 18 to 21 years (mean = 19.58 ± 1.09 years) with a relatively even distribution of participants for each year in school. However, the highest percentage (31.6%, n = 18) of participants were sophomores (Table 1). The majority of our sample reported no previous history of concussion (75.4%, n = 43), and 68.4% (n = 39) reported receiving concussion education from an NCAA college or university. However, the majority of our sample reported not receiving concussion education in high school (64.9%, n = 37), with 1.8% (n = 1) not responding to the question.

The analysis revealed no significant changes in concussion knowledge or retention over time, F₂ = 1.95, P = .147, η² = 0.034, with mean scores of 15.88 ± 1.63, 16.19 ± 1.84, and 16.30 ± 1.57 across preintervention survey, postintervention survey, and retention, respectively. In addition, post hoc power analysis indicated sufficient sample size to determine an effect. The mean scores for the preintervention survey were lower than expected with 73% correct. No significant differences were found when comparing freshmen's scores with returners' scores over time, F₂ = 2.09, P = .129, η² = 0.038. Similarly, a post hoc power analysis also indicated that there was sufficient sample size to observe an effect.

Ten randomly selected respondents answered the semistructured interview questions regarding the multimodal concussion-education intervention. The average time between the intervention and interviews was 45.8 days. All 10 respondents represented Division I collegiate student-athletes from an NCAA institution. There was a favorable mix of genders and student-athletes from multiple sports, as illustrated in Table 2. The analysis of the qualitative data led to the emergence of 4 themes relating to the athletes' attitudes and perception of knowledge attainment using a multimodal intervention.

Understanding the respondents' previous experience with concussion education provided a comparison with the attitudes and perceptions of the multimodal intervention. The first theme that emerged was the respondents did not perceive having previous formal concussion education. Overall, all the respondents stated that previous educational opportunities were not formal, and they did not remember specific concussion-related information. One respondent stated, “No, nothing,” and another stated, “No, like I have talked about it but like actual education, no.”

More specifically, returners remembered previous concussion education at their university, though they did not recall much of the information, whereas the freshmen reported not having received formal concussion education in high school. One returner stated, “I want to say it was just someone talking to us. I don't remember it being anything formal. Just one of our many meetings we have at the beginning of the year.” This finding is concerning considering that since 2010 state legislation in all 50 US states has mandated that all student-athletes at the secondary school level receive concussion education yearly while participating in school athletics, and the NCAA mandates student-athlete education annually.³⁵ 

The second theme surrounded recall of information from the intervention. When asked what they were able to recall from the multimodal concussion-education intervention, all the respondents were able to recall signs and symptoms of concussion, as evidenced by one respondent:

Interestingly enough, we noticed that the freshmen recalled physical symptoms and the mechanism of concussion, such as one freshman respondent stating, “It made me more aware of symptoms of a concussion and what to do if you think you have one.” Another freshman respondent stated, “One beneficial thing I learned was the symptoms and how I could hit my head and not know, like, ‘Oh, I just have a headache,' but really I need to get checked out for a concussion.” However, the returners focused more on the possible long-term symptoms and the current concussion research after the intervention, including one respondent stating, “I remember [head AT] was talking about learning more and more about concussions and how they have found new studies and people are donating their brains to keep it going.” It is possible that because of the perceived lack of formal education in high school, the freshmen focused more on physical signs and symptoms and what could constitute a concussion, whereas the returners were able to focus on new information like the current research.

The third theme that emerged was a perceived increase in knowledge. Overwhelmingly, all respondents reported a perceived increase in knowledge after the multimodal concussion-education intervention. One respondent stated, “Yes, it made me more aware of how you get concussions and what to do. How you can get them anywhere at any time.” Another stated,

All respondents regarded the intervention in a positive way, stating they had learned from the multimodal concussion-education intervention.

The fourth and final theme described student-athletes' perception toward the multimodal education intervention. All respondents had a positive perception about using multiple modes of delivery for the intervention. One respondent stated,

Some respondents discussed how they thought the components were effective together and did not feel they would have received the same information if we had used one component alone, as described by another respondent,

Multiple respondents stated that the video was relatable and gave them a real-life example of what a concussion could look like. Though they liked the video component, they believed it could not stand alone, as explained by one respondent: “I think they all kind of piggybacked off of each other. If we had just watched the video, I don't think we would have got as much out of it.”

The primary purpose of this research study was to determine if a multimodal concussion-education intervention would improve concussion knowledge and retention in collegiate student-athletes. Previous studies^3,4,32,36,37  have suggested student-athletes have a basic level of knowledge; however, the modes of educational intervention have varied. Educational practices have included didactic lecture, video education, and recent technological advances such as interactive computer modules and video games.^11,13,27,38  Whereas the goal of any educational intervention has been to improve knowledge, little research has examined the effectiveness of the delivery mode of the intervention. Therefore, this research study sought to evaluate the use of a multimodal educational intervention in NCAA Division I collegiate student-athletes using a sequential exploratory mixed-methods design.

The NCAA has mandated the use of a fact sheet to educate all student-athletes within NCAA member institutions¹⁴ ; however, questions always arise regarding the efficacy of this mode alone to enhance concussion knowledge and retention.³⁹  A multimodal approach was developed to enhance previous educational practices by incorporating multiple modes of delivery, including video, a lecture including facts on the NCAA concussion fact sheet, and an active reflection session. The goal of this approach was to promote student-athlete engagement and ultimately improve concussion knowledge and retention. We expected to find an increase in concussion knowledge and retention over time after the educational intervention; however, no significant changes in knowledge or retention were found.

This finding is consistent with previous studies that found no significant changes in concussion knowledge or retention after an educational intervention.^13,15  Kroshus et al¹⁵  examined concussion-knowledge scores across 6 ice hockey teams with each team receiving different modes of delivery, such as the NCAA concussion fact sheet sent via email, hard copies of the concussion fact sheet, lecture, and/or a video form of information. Overall, no significant differences existed in knowledge scores across groups.¹⁵  The current study supports Kroshus's¹⁵  previous work finding that a multimodal approach using lecture and video forms together does not increase knowledge in collegiate athletes. Therefore, other educational resources may need to be examined to improve knowledge for collegiate student-athletes.

Educational resources such as videos, seminars, and websites are designed to communicate knowledge about concussions. However, there is no evidence to support that these modes are the most effective.⁴⁰  There are various web-based educational tools, such as video games, that have been developed specifically for youth and adolescent athletes at the secondary school setting. Therefore, the majority of the education research has been carried out with youth sport and high school populations. Goodman et al¹²  found a positive impact of the Symptom Shock app on concussion symptom recognition after youth hockey players played the game.¹²  The use of widely accessible web-based educational tools and video games may be effective in providing student-athletes with basic concussion knowledge, but it has yet to be researched in the collegiate population.¹² 

Whereas some studies found a lack of concussion knowledge among youth and collegiate student-athletes,^3,5,32,36,37  other studies have reported student-athletes may have a basic understanding of the mechanism of concussion, signs and symptoms, and who to report a concussion to.⁴¹  Sye et al⁴¹  surveyed high school rugby players on their understanding of concussion and the return-to-play guidelines and found that the athletes demonstrated “reasonable” knowledge of classic concussion signs and symptoms. Other studies have shown baseline scores for concussion knowledge as high as 80%.¹³  Echlin et al¹³  reported a 77% mean score at baseline for concussion knowledge, whereas Register-Mihalik et al⁴²  reported a baseline score of 79%. In the current study the mean preintervention survey score was 73%, which is consistent with Cournoyer and Tripp.³⁶  These findings suggest that student-athletes are coming into college with basic concussion knowledge.

When determining if concussion education is effective, it is important to address what kind of knowledge student-athletes should possess within the framework of adult learning theories. Palis and Quiros⁴³  state that an adult's readiness to learn comes from the need-to-know information to cope with situations they face in their lives. In collegiate athletics, concussion is a common injury that can happen in sport and in daily life. Basic concussion knowledge may allow the student-athlete to understand basic signs and symptoms; however, it takes a higher understanding to cope with the potential long-term effects and cognitive deficits.⁴³  The current study, along with others that have examined basic knowledge of concussion in collegiate athletes, demonstrates that student-athletes do not understand the potential for long-term effects from sustaining a concussion.^4,6 

Prior concussion knowledge obtained in high school is crucial to facilitate understanding of the potential long-term consequences of a concussion in collegiate student-athletes. Previous concussion knowledge that is accurate and correct facilitates a foundation for building new knowledge.⁴⁴  Though concussion education in secondary schools is mandated by law,³⁵  student-athletes who participated in this study did not recall receiving formal concussion education in high school. This finding was supported by our qualitative data, in which our respondents did not perceive that they had obtained formal concussion education in high school. Furthermore, returning student-athletes who had had concussion education previously at the university only remembered having their AT lecture them about concussion education, with many stating that they had just reviewed the NCAA concussion fact sheet with their AT during preseason meetings.

Our quantitative data analyses revealed no significant differences between the freshmen's and returners' scores on the RoCKI, suggesting similar knowledge scores regardless of previous education exposure. Similar scores regardless of age of participants may be related to the mode (didactic lecture/meeting) in which the concussion education is delivered. Though the freshmen did not recall receiving formal concussion education, it appears that the previous concussion education that the returners received did not impact their concussion-knowledge score compared with freshmen. Although knowledge scores were similar, qualitative analysis provided more detail that student-athletes were able to recall basic concussion knowledge and current concussion research from the lecture. Although knowledge scores did not significantly change over time in either group, knowledge plateaued from the postintervention survey (immediately after the intervention) to the retention survey (approximately 3 months later). This finding suggests that student-athletes in our sample were able to retain and recall concussion knowledge over the 3-month time frame.

Knowledge-retention research is controversial, especially in concussion-education literature. Both Miyashita et al²⁷  and Cusimano and colleagues²⁸  found significant increases in knowledge directly after intervention, but only Miyashita et al²⁷  observed retention of those scores. Yet Echlin et al¹³  and Cusimano et al²⁸  both had concussion-knowledge means that demonstrated a plateau of concussion-knowledge scores over time, consistent with the current study's findings. Although knowledge scores did not significantly improve, student-athletes were able to freely recall the information presented to them during the qualitative interviews (approximately 1½ months after the intervention), which provides additional evidence of retention after the intervention.

Interviewing respondents after the multimodal concussion intervention enabled the researchers to obtain the student-athletes' perspective of the intervention and knowledge retention. Themes extracted from qualitative interviews revealed that the majority of our sample could recall the education they received, as well as information including signs and symptoms of concussion, long-term effects, and current concussion research. Specifically, we noticed the freshmen recalled basic signs and symptoms of concussion, whereas the returners focused on the long-term effects and current research. As the interviews occurred more than 1 month after the intervention, the ability for the respondents to freely recall the information explains their ability to retain the information months later even if the quantitative scores may not reflect their knowledge.

We found student-athletes reported a perceived increase in concussion knowledge after the intervention. Although we implemented a different educational approach, our findings mimic those found in similar research regarding concussion education.^13,15,36,42  A primary theme that emerged was a perceived increase in concussion knowledge in all respondents after the multimodal concussion-education intervention. Overall, the respondents said they had known some of the basic material before the intervention, but felt they had learned additional information during the multimodal intervention, specifically the new, emerging concussion information.

Most importantly, all respondents had a positive attitude toward using multiple modes of delivery for the education. Kroshus et al¹⁵  found the hockey team who watched a video along with the lecture had the highest recall rate (92%) across the different modes of delivery. Additionally, they asked the other teams what changes would make the education more effective, and the response was to provide the information in video form with testimonials from other athletes about their history of concussion. It seems that although a video alone has not been proven to be the most effective for increasing knowledge,^13,28  it may have a positive impact on student-athletes' knowledge because it provides real-life scenarios to which the athletes can relate. Further, it supports the kinesthetic and adult learning theories that explain how to keep the student-athlete actively engaged and reflecting on the information presented.

Our study had several limitations. The primary limitation of the study was the educational intervention itself. The initial design of the study originally included the lecture, the video, and an active educational portion that consisted of a panel discussion including other athletic training staff and student-athletes to ensure student-athletes reflected on the materials presented. Inclusion of an active learning strategy in an educational intervention has proven to heighten knowledge in adult learners.²¹  Unfortunately, because of time constraints, the panel discussion was eliminated. The head AT presented all components that were used in the intervention. Athletic trainers' concussion education will always involve concerns about time constraints and the ability to provide an efficient session that leads to knowledge increases both immediately and throughout the season. This study provided preliminary evidence that incorporating active learning strategies, such as an active reflection session, may be completed effectively in collegiate athletes with perceived engagement and learning from the participants.

The second limitation of this study was the inconsistent timing of the survey after the intervention. The postintervention survey was originally intended to be administered immediately after the intervention. However, because of time constraints and meeting schedules, some teams did not complete the postintervention survey until 48 hours later. Although we did not get consistent data collection postintervention, knowledge scores were consistent across teams that completed the postintervention survey immediately after the intervention versus the teams that completed it 48 hours later. It appears that the timing of the immediate postintervention survey did not influence knowledge scores.

The third limitation was the sample size. This study consisted of a sample of convenience and was limited to student-athletes at one Division I institution. Because of delayed IRB approval and scheduling conflicts, football, men's basketball, and men's golf were not included in this study. However, without the previously listed sports, our sample of 222 surveys during preintervention survey provided appropriate representation of the student-athlete population. Future research should examine all sports to ensure that the findings are consistent across all Division I collegiate sports.

The final limitation may be the choice to use the RoCKI as the assessment tool. Using a questionnaire that was previously validated and designed provided a valid measure to gauge knowledge and make appropriate comparisons with previous research. However, given that the intervention was provided by the head AT, and the attempt to align with previous educational sessions for the returners, the PowerPoint was based upon the current literature and NCAA fact sheet. As such, some questions specific to the RoCKI may not have been addressed during the intervention. Therefore, the scores may not be completely reflective of increased knowledge and may further explain the perception that the student-athletes increased knowledge and enjoyed the sessions. The current study focused on developing and implementing a novel multimodal intervention for concussion education within a real-world collegiate setting. This first step found that the student-athletes enjoyed the educational intervention and perceived an increase in knowledge. Future studies should develop assessment tools to specifically examine the effectiveness of the multimodal educational intervention to increase and retain concussion knowledge over multiple competitive seasons.

Concussion education has been deemed a cornerstone for concussion recognition and management. State legislation and the NCAA have laws and mandates in place that require coaches, parents, and student-athletes to be educated on signs and symptoms of concussion and the importance of reporting symptoms to an AT or coach.^14,35  However, these laws and mandates do not specify the mode of education intervention. Effective and efficient concussion education should focus on the audience. Adult learning is best facilitated when the information is relatable to the audience's life experiences, is geared toward changes in societal roles, and is directly applicable to the audience's life.²⁴  Although the goal of concussion education is to improve identification of concussions to initiate timely and proper management, retaining the knowledge is equally important.

Concussion education will be effective in increasing knowledge and retention only if it can be administered efficiently while enhancing student-athlete engagement. This study provides evidence that a multimodal approach using a lecture, a video, and an active reflection session may not be the most effective for increasing student-athlete knowledge. However, our study did demonstrate that collegiate student-athletes perceived an increase in concussion knowledge and were able to recall basic knowledge from the intervention, with specific emphasis on enjoying the video component. Future research should investigate educational interventions that target specific populations and include active learning strategies within a multimodal program to increase concussion knowledge and retention in the collegiate setting.