Concussion legislation mandates that health care providers have experience in concussion management. Unfortunately, standards for current continuing and clinician education are ill defined.
(1) Determine if a didactic-based educational intervention would increase knowledge and perceived expertise and (2) examine the correlations between the variables of knowledge, experience, and perceived expertise.
Prospective cohort study, level II.
Novice health care providers were divided into 2 groups: college sophomore athletic training students (n = 16) and college sophomore medical dietetics students (n = 19).
Both groups were administered a knowledge questionnaire before the intervention (Time 1) and again 30 weeks later (Time 3). The athletic training student group completed a didactic intervention and completed the questionnaire at the end of the quarter ∼15 weeks later (Time 2).
The main outcome measure was a 34-item questionnaire designed to examine knowledge, experience, and perceived expertise using true-false items, scenarios, Likert-scaled items, and open-ended questions derived from existing evidence and current literature. The scores from the knowledge, perceived expertise, and experience items served as dependent variables.
No statistically significant interaction between groups existed on knowledge scores after the didactic intervention (P = .10). Statistically significant interactions existed between group and time for both perceived expertise (F1,33 = 86.38, P ≤ .001) and experience (F1,33 = 14.2, P ≤ .001) with the athletic training student group demonstrating significant increases in scores over time. There was a statistically significant correlation between the number of concussions evaluated and perceived expertise (r2 = 0.630, P ≤ .001).
Educators need to implement the best educational techniques to maximize knowledge attainment and perceived expertise. While it appears that clinical experience may supersede didactic education, a combination of both will encourage higher-level thinking and implementation.
Concussion awareness and educational efforts have increased exponentially in recent years. This increased awareness can be attributed to catastrophic injuries in high school athletes and concussions suffered by high-profile athletes. Ultimately, this led the Centers for Disease Control and Prevention (CDC) to classify concussion as an epidemic.1 There are an estimated 1.6 million to 3.8 million sport-related concussions occurring in the United States annually in contact sports at both professional and amateur levels.2,3 Direct medical costs and indirect costs, such as lost productivity from mild traumatic brain injury to severe traumatic brain injury, totaled an estimated $3 billion to $12 billion in the United States, with each mild traumatic brain injury costing anywhere between $33 284 and $35 951.4,5
The heightened awareness of concussion in both the media and scientific community has led to several agencies, such as the National Athletic Trainers' Association, Concussion in Sport Group, and the American Association of Pediatrics, producing position statements addressing assessment and management of concussions.6–8 Many states are following suit by enacting legislation mandating that health care providers be trained in concussion evaluation and management in order to make return-to-participation decisions after injury. The cornerstone to each position statement and legislative bill is education. Researchers, administrators, and politicians believe that providing education to clinicians working with concussed patients will decrease the risk of secondary injuries and potential catastrophic events.6–8
The implementation of legislation that requires medical clearance before returning to participation increases the likelihood that physicians will be called upon to manage a concussion at least once in their careers.9 However, current medical education does not include assessment and management of head injuries unless the physician completes additional training in specialties such as physical medicine and rehabilitation, neurology, neuropsychology, or neurosurgery.9 Unfortunately, these specialties rarely see patients with the isolated diagnosis of concussion. These specialties are typically consulted with retracted recovery or when more severe injuries such as hematomas, second impact syndrome, or postconcussion syndrome are observed.9–10
Concussion, on the other hand, is typically managed by sports medicine certified physicians, pediatricians, or athletic trainers.10–13 Pediatric residency programs were studied in 2003 and concluded that 24% of the programs did not include formal teaching about concussions in their curriculum.14 When family physicians and pediatricians are not available, many families will go to the emergency room for concussion visits. Emergency room visits attributed to concussion in 14- to 19-year-old patients have tripled from 1997 to 2007.15 When seen in the emergency room, Sarsfield et al15 reported 95.1% of pediatric patients received instructions to follow up with a primary care physician, 82.8% received anticipatory guidelines regarding expected symptoms, and 15.2% received specific restriction time from sports.15 Another study of concussion assessments in the emergency room found discharge instructions were found to be inappropriate 30% of the time.16 When examining physicians on the front lines of concussion assessment and management, there is an obvious gap in clinical practice knowledge.
It is estimated that over 25% of practicing athletic trainers work exclusively and directly with adolescent patients, and these athletic trainers appear to be the first line of defense, evaluating an average of 8 concussions per year.17 Ferrara et al18 found 17.1% of athletic trainers evaluate more than 10 concussions per year. Athletic trainers should be prepared through both didactic and clinical education to recognize the potentially negative physical effects of concussion, as well as the appropriate assessment and management of the injury.
Steps have been taken to begin the educational process among health care professions. The CDC, along with partner organizations, created Heads Up: Concussions in your Practice.19,20 This toolkit includes educational materials such as a video, wallet card listing signs/symptoms, posters, fact sheets, and other concussion-related resources. To date, the CDC has disseminated more than 1.3 million educational materials on concussion in sports for health care professionals, coaches, parents, and athletes. In an effort to continue to expand the initiative, the CDC created a national concussion educational initiative aimed at school and health care personnel who work with students aged 5 to 18.20 Although no quantitative research has demonstrated evidence to support the efficacy of this type of educational program, its availability is a step in the right direction.
Recent legislation requires medical care for all athletes sustaining concussion, and the heightened awareness of concussion necessitates both novice and experienced clinicians to have a thorough understanding of the injury, assessment techniques, and management.21 Therefore, the objective of this study is to determine if a didactic-based educational intervention will increase knowledge and perceived expertise in novice health care professionals. A secondary objective is to examine if any correlations exist between the variables of knowledge, experience, and perceived expertise. The results of this study will lead to a greater understanding of the effectiveness of a concussion education intervention, which we will use in future educational endeavors across multiple allied health professionals, coaches, parents, and others associated with concussion assessment and management.
A convenience sample of college students in the athletic training and medical dietetics programs in a school of health and rehabilitation sciences was asked to participate. Participants were divided into 2 groups: (1) novice health care providers and (2) controls. The novice health care providers were represented by sophomore level (n = 22) athletic training students who were concurrently enrolled in an athletic training course and a clinical practicum. The course focused on the orthopaedic evaluation of the head, neck, and spine, and the clinical practicum involved a 10-week rotation with a Division I varsity athletic team where concussions were anticipated to occur. Sophomore athletic training students provided the researchers with a unique population of novice health care providers as they obtain didactic instruction concerning evaluation of an injury or illness and proceed to clinical rotations where this knowledge is immediately reinforced via active hands-on learning.
The control group was represented by sophomore level medical dietetic (MD) students with no previous experience or training in concussion assessment, management, or care (n = 25). When compared to the athletic training students, the MD students maintained the same educational schedule, were similar in size, and held the same academic rank. Further, both groups completed similar general education prerequisite course work, such as English, math, biology, and anatomy, as freshmen.
Inclusion criteria included (1) students enrolled in a health care-related major with no previous formal education in concussion injury, (2) students enrolled in the first professional year of a health care professional education program, and (3) completion of a questionnaire during designated time points. Exclusion criteria included (1) being diagnosed with a concussion injury within the previous 3 months and (2) known history of diagnosed vision or hearing disorder that would impair ability to complete the questionnaire. All participants signed an informed consent form approved by the Human Subjects Review Committee before participating in the study.
Data Collection Procedures
During the first week of the spring quarter, the principle investigator attended 1 of the study participants' classes and explained the research study. Questions regarding the study and participation in the study were answered, and the students were given 24 hours to decide whether they would participate in the study. Once participants provided consent, all students (athletic training students and MD students) were administered the pre-intervention questionnaire (Figure 1). This was considered baseline testing (Time Point 1) and was used to assess knowledge before the educational intervention.
Athletic training students completed 1 week of didactic education and 1 lab session on concussion assessment and management techniques for health care professionals. At the conclusion of the class, approximately 9 weeks after the educational intervention, the questionnaire was re-administered to the athletic training student group to analyze knowledge attainment after the intervention (Time Point 2). Long-term follow up for both groups occurred after the conclusion of summer, which included additional clinical experiences for the athletic training student group; this was approximately 30 weeks after the baseline questionnaire and 21 weeks after the end of intervention (Time Point 3; Appendix).
Intervention: Didactic Concussion Lecture
After the completion of the pre-intervention questionnaire, the athletic training students participated in the educational intervention, which was delivered in a lecture format guided by a PowerPoint (Microsoft, Redmond, WA) presentation. The educational intervention focused on the assessment, evaluation, and management of concussion and spanned three 50-minute didactic classes. Throughout the didactic classes, videos and scenarios were used to provide explanations of symptoms, such as posturing, and mechanisms that may indicate a concussive injury. Additionally, group learning scenarios, simulated patient evaluations, and interactive jeopardy games were employed to reinforce knowledge attainment and use of assessment tools. Each class session started with the opportunity to share experiences related to concussion and head injury, either from their clinical rotations or personal history. This encouraged reflection, group discussion, and evaluation of clinical care management. If no experiences were shared, a case study was provided to start the session and concluded with a question-and-answer period. Within the curriculum, a 2-hour clinical skills lab provided time to develop clinical skills using concussion evaluation tools, simulate patient communication, and practice a clinical progression for concussion evaluation and management. All content was developed and theoretically based upon current literature and evidence-based clinical paradigms. The content was delivered by the same researcher/educator with an expertise in concussion research and management.
Concussion Education Questionnaire
The instrumentation used in this study consisted of a questionnaire that was developed from existing and new questions to assess knowledge, experience, and perceived expertise (ie, self-efficacy) of concussion evaluation, assessment, and management in novice health care professionals. Several questionnaires are available to assess basic knowledge of concussion assessment for coaches,22,23 student athletes,24,25 and/or clinicians.10,26,27 However, currently available questionnaires only include basic knowledge and management decision strategies. The goal of the current study necessitated the addition of material to assess experience and perceived expertise (ie, self-efficacy).28 The final 34-item questionnaire consisted of demographic information, true-false items, scenarios, Likert-scaled items, and open-ended questions derived from existing evidence and current literature.
The questionnaire was designed to examine 3 aspects of concussion assessment and management (Figure 1). Demographic information included previous concussion education, sources of concussion information, and expected future educational resources were obtained through qualitative open-ended questions. (1) Knowledge was developed to determine the participant's basic knowledge of concussion recognition, assessment, and management. This section consisted of 20 true-false and scenario-based questions. The participant was given 5 points for every correct answer with a possible maximum score of 100. (2) Experience intended to quantify the experience the participant had in assessing concussion. To obtain this information, the demographic section of the questionnaire asked, “How many concussions have you evaluated in the past 3 months?” (3) Perceived expertise assessed perceived expertise in evaluating and treating a concussed patient. This section consisted of 7 Likert-scale questions. The Likert scale was anchored with 1 indicating no expertise and 5 indicating a great deal of expertise.
Content and construct validity were established for the questionnaire. Content validity was established by providing the questionnaire to 8 certified athletic trainers and physicians who were instructed to evaluate the questionnaire and determine if questions were assessing concussion knowledge, assessment, and management. All the certified athletic trainers and physicians had 100% congruence that the questionnaire did evaluate basic knowledge of concussion, assessment, and management. There were no suggested revisions.
Construct validity was determined by providing the questionnaire to 5 college seniors with formal training in concussion management and 5 sophomore-level students with no formal training in concussion management. The mean scores for knowledge were statistically higher in the senior-level students than the sophomore-level students (F1,8 = 4.89, P = .021), and the experience level and perceived expertise scores were lower in the sophomore-level students compared to the senior-level students. Test-retest reliability of the questionnaire was assessed using a Pearson correlation coefficient. A group of 9 college students completed the questionnaire twice, 1 week apart (r = 0.81).
This was a simple experimental design where the intervention group of athletic training students received traditional classroom didactic sessions and a clinical laboratory session, supplemented with clinical work in the field. The control group of MD students received neither didactic, laboratory, nor clinical exposure to concussion education. Both groups were measured at Time Point 1, at the beginning of the academic quarter, to establish baseline scores for knowledge, experience, and perceived expertise. The athletic training student group was tested at Time Point 2 to determine the effectiveness of the educational intervention on knowledge and perceived expertise. Finally, both groups were retested at Time Point 3, 21 weeks after the intervention, to determine long-term retention of knowledge and the impact of clinical experience on perceived expertise.
Open-ended questions were analyzed thematically. Credibility was established by researcher triangulation. Quantitative data were analyzed using SPSS (version 21.0; SPSS Inc, Chicago, IL) statistical software. Group by time repeated measures analyses of variance were used to compare groups over time on 3 dependent measures: knowledge about concussion; perceived expertise about concussion identification, management, and education; and experience with concussion. Correlations between knowledge scores, experience, and perceived expertise were calculated using 2-tailed Pearson correlations.
Sixteen of 22 (72%) athletic training students completed the questionnaire at all 3 time points, while 19 of 25 (76%) MD students completed the questionnaire at both time points and were included in the analysis.
Agreement was attained between the 2 researchers analyzing the qualitative section of the questionnaire. The researchers agreed the main themes for the qualitative section of the questionnaire included that (1) all participants were aware of concussion and had previous knowledge of the injury from the media; (2) the athletic training students seemed to believe they should learn about concussion from class, instructors, and clinical rotations, while the MD students indicated they would learn about concussion from a doctor if they were injured or coaches if they were involved in athletics; and (3) all participants thought they should know the signs and symptoms of concussion; however, athletic training students expressed they should know how to assess and manage concussions.
This study did not find a statistically significant interaction between groups on knowledge scores across time (F1,33 = 2.80, P = .103; Table 1). However, we observed a group main effect, which we interpreted as the 2 group combined means across the 2 time periods were significantly different with the athletic training student scoring higher than the MD students (F1,33 = 5.40, P = .026; effect size; Cohen d = 0.664; see Figure 2).
The observed downward trend of athletic training student group knowledge scores over time (see Figure 3) prompted a post hoc analysis of the athletic training student group knowledge scores over time. A 1-way repeated measures analysis of variance revealed the athletic training students showed a significant decline in concussion knowledge from the pretest to the 30-week follow up (F1,15 = 5.983, P = .027; effect size; Cohen d = 0.698).
Perceived Expertise and Experience
The 2 × 2 analyses of variance with 1 repeated measure indicated statistically significant interactions existed between group and time for both perceived expertise (F1,33 = 86.38, P ≤ .001) and experience (F1,33 = 14.2, P ≤ .001) scores. For both measures, the athletic training student group showed significant increases in scores across times 1 and 3 (see Figures 4 and 5).
Two-tailed Pearson correlations indicated a statistically significant correlation between the number of concussions evaluated and perceived expertise (r2 = 0.630, P ≤ .001; Table 2). As athletic training students evaluated more concussions, they gained experience and reported an improved perceived expertise.
The primary goal of this study was to determine if an educational intervention for novice health care professionals would increase knowledge and perceived expertise. Based upon existing research, which indicates an increase in knowledge scores for athletes,24,25,29 coaches,22,23 and other health care professionals,27 it was hypothesized that knowledge scores would improve for the athletic training student group after the educational intervention. However, in the current study, the knowledge scores in the athletic training students decreased after the intervention and even more at the 30-week follow up from the educational intervention. Previous pre-educational/posteducational intervention studies have repeated the questionnaire immediately after the educational intervention.22,24,25 This study did not conduct the questionnaire immediately after the educational intervention, as to not interfere with the educational model in coursework. The decreasing knowledge scores at time points 2 and 3 may be an indication of poor retention. Kroshus et al29 found similar findings when they examined various educational methods for collegiate ice hockey players and found no significant improvements in knowledge. This may suggest the need for additional education through continuing education modules to maintain and reinforce knowledge attainment.
Nevertheless, while poor retention is one possible explanation, the researchers do not feel this is the root of the declining knowledge scores for the athletic training students. The researchers feel additional rationales may be (1) athletic training students gained an increased appreciation for the complexity of assessing and treating concussion as a result of the educational intervention. It may be possible that, as the athletic training students' experience and practical knowledge increased, the questionnaire became too simplistic or dichotomous to capture the contextual nature of concussion and the critical thought processes used in clinical experiences. (2) Athletic training students may have experienced a general lack of effort to complete the questionnaire during Time Point 2 (before the start of spring break) and Time Point 3 (return from summer break). These additional time points arrived when academic motivation may be diminished. Therefore, the students may not have provided an earnest effort as opposed to the initial questionnaire at the beginning of the course.
When comparing the 2 groups, we did not find a statistically significant difference between groups in knowledge scores. It appears both groups scored relatively high, and the simplicity of the questionnaire was unable to differentiate between groups. This questionnaire was administered to the athletic training students before a course on head, neck, and spine injury evaluation, but this study did not take into account previous clinical experiences or learned knowledge. We believe high scores at baseline for both the athletic training students and MD students can be attributed to the mass media attempts to educate the public and health care professionals about the seriousness of concussion. Since the inception of the Lysdedt law, major educational implementations are occurring at the youth sports level as well.30 This study included sophomore-level students who may have participated in sports in high school when education was initially being implemented, resulting in the high baseline knowledge scores for both groups.
As a means to gauge each participant's experience, participants reported the number of concussions that were evaluated across the 3 time points in order to determine if experience enhances knowledge and perceived expertise. Previous studies10,17,18 have examined the practice patterns of clinicians by evaluating years of experience and number of evaluations conducted as means to classify practitioners into experts and novice groups. The rationale is that more experience will result in the clinician being more competent in evaluating and treating injury.
In our study, athletic training students showed an increase in concussion experience measured by the number of concussions evaluated during their clinical experiences. The MD student group maintained 0 to 3 evaluations (the lowest selection possible), while the athletic training student group had a statistically significant increase in the number of concussion evaluated. Even though the knowledge scores decreased, there was a statistically significant increase in the number of concussions evaluated. This finding is supported by Boggild and Tator,11 who found previous experience with a concussion patient was not a significant predictor of knowledge scores.
This finding is not surprising, considering most health care professions' educational preparation includes clinical experiences designed to reinforce knowledge and strengthen clinical evaluation skills and competence. In particular, athletic training curriculums stress learning over time where students are introduced to the material in didactic settings and progress through clinical placements, working toward competence in skills.
The current study appears to lend support to the theory that clinical experience may not improve basic knowledge about concussion but may prove to be a key educational component to obtaining expertise and competence in concussion evaluation and management skills.
Perceived Expertise (ie, Self-Efficacy)
A secondary objective was to examine the correlation between experience and knowledge, and perceived expertise. We found athletic training students showed a significant positive change in their perceived level of expertise over time, while controls remained constant. It appears increased experience evaluating concussions leads to increased self-perceived expertise, or self-efficacy as defined by Bandura.28 The increase in perceived expertise in the current study is potentially important, as legislation requires concussed patients be evaluated and managed by those with expertise in concussion. Unfortunately, there is no definition of a concussion expert or clinician who is experienced in concussion. However, the results of the current study suggest perceived expertise (ie, self-efficacy) may be an alternative to determining a clinician's experience, expertise, and ability to evaluate and manage a concussed individual.
This study demonstrated an increase in perceived expertise, but a decrease in knowledge attainment. While there is no data that supports this theory, we believe, as students gained more experience and perceived expertise, the simplistic, dichotomous questions used to assess knowledge may not have been advanced or complex enough to capture the contextual nature of concussion symptom presentations. After all, few concussions present alike, and few concussions follow the same recovery timeline. Thus, it is possible a black-and-white paper test of knowledge fails to capture clinical knowledge accurately. Further, the decrease in scores may have been a factor of effort rather than decreased knowledge attainment.
It has been well established that there is a need for additional education which provides up-to-date assessment and management strategies for concussed athletes. Moreover, the documented increase in diagnosed concussions along with current concussion legislation dictate the need to offer education among health care providers who are authorized to treat and make return-to-participation decisions after concussion.1–3 Since athletic trainers are typically the front line of evaluation and management of concussions, they must have the educational foundation to be considered experts. Physicians, clinicians, and educators need to determine the best mode of education that will enhance knowledge retention. The results of this study suggest that self-efficacy ratings may offer one measure of concussion expertise to legitimize health care providers working with concussed patients.
There are several limitations of this study that warrant discussion. First, caution was taken to control for age and educational level, but information regarding gender, previous sports participation, and prior knowledge regarding concussion was not gathered. Previous education resulting from sport participation, which was not assessed, may have influenced baseline knowledge. As the control group believed they would obtain information about concussion from sport participation and coaches, examining the relationship between previous sports participation and knowledge may be necessary. However, Boggild and Tator11 examined medical students and residents and found previous history of concussion, recreational sport participation, and gender were not significant factors associated with knowledge scores. Regardless, future studies should examine the role of the previous sports and concussion knowledge (both correct and incorrect) in the education provided.
Second, the knowledge test embedded in the questionnaire may have been too easy, which resulted in those with limited knowledge scoring relatively high. The knowledge section of the questionnaire contained questions commonly asked in previous concussion education questionnaires.23–26 Due to the design of this study and use of control participants, the information had to be basic enough to distinguish those with limited to no knowledge of the concussive injury. While the questionnaire was able to differentiate between the control group and those with more knowledge in the pilot test, the instrument appeared to be unable to determine the effectiveness of the educational intervention in the novice health care professional group. Future studies should develop a questionnaire specifically, complete with more sophisticated problem scenarios and evidence-based questions, as suggested by Provvidenza et al,31 so multiple levels of educational competence can be assessed.
Finally, this study was not designed to assess instructional method or knowledge transfer. The concussion educational module was designed by a researcher with expertise in concussion. The researcher has an educational philosophy that incorporates techniques to engage all types of learners using the best available research evidence. Therefore, the educational sessions consisted of lectures and assignments focused on (1) individual and group learning; (2) visual, auditory, and kinesthetic learning styles; and (3) video and gaming techniques to solicit engagement of the students in the educational sessions. It appears knowledge was attained, but the quality of the knowledge as defined by retention is questionable. Metrics to assess retention should be a focus in future research. Development of future studies should include assessment of knowledge transfer and retention to ensure the most effective method is used and disseminated to other health care educators. Additional studies should assess other factors such as preceptor knowledge and clinical situations that enhance perceived expertise of the preceptor to support the student learner and reinforce knowledge retention.
Medical educators need to implement the best educational techniques to maximize knowledge attainment, competence, and clinical expertise; however, the most effective method of education is debatable. Most education regarding concussion is accomplished by experience and/or education. This study reinforces the need for didactic lectures to provide basic knowledge and foundational information. However, it appears clinical experience may supersede didactic education; thus, a combination of both may encourage higher-level thinking and implementation necessary of health care professionals. This study found experience may be the most effective tool to promote knowledge attainment and retention for novice health care practitioners. The authors of this study believe educators can use active learning environments to incorporate aspects of experiential learning, such as case studies, learn-pair-share, small group discussions, and patient simulations. Further, educators can use clinical simulations or standardized patients to assess knowledge, critical thinking, and competence in a controlled environment in lieu of a simple knowledge test.32
Legislation requires health care providers with experience in concussion be the people evaluating and determining return-to-participation criteria. As a result of this legislation, there is a need to better define health care provider with experience in concussion assessment.21 This study offers support to use the concept of perceived expertise (ie, self-efficacy) as a potential metric to assess educational outcomes and experience in concussion assessment. Clearly, this concept requires further study and vetting as the need for expertise in concussion assessment and management will continue to grow as we seek better understanding of the injury and its management.
Researchers and clinicians continue to rely heavily on education as the cornerstone to prevention; however, there are limited studies providing evidence in support of effective educational interventions for health care practitioners.33–36 While knowledge retention in this study was examined using didactic knowledge sessions, it confirmed that clinical experience may be the guiding key for not only knowledge attainment and retention, but also self-efficacy. Health care professions, such as athletic training, physical therapy, occupational therapy, physician assistants, nursing, and psychology programs, have already implemented didactic lessons with clinical fieldwork to emphasis knowledge retention34,35 ; clinicians should look toward self-efficacy as a measure of expertise to determine if educational interventions are truly effective.
Appendix. Study design.
Dr Hunt is currently Assistant Professor and Graduate Program Coordinator in the School of Health and Kinesiology at Georgia Southern University.