Case Characterization and Perceptions of Athletic Trainers Regarding Medical Disqualification Following Concussion

We used a convergent parallel mixed methods design. Convergent parallel designs include data being collected and analyzed separately and then combined to achieve triangulation.^13,14  Triangulation is a qualitative research concept in which multiple methods are used to triangulate results about a topic, which is necessary to offset the biases associated with each method and counteract sources of error.^14,15  Aims 1 and 2 were cross-sectional, whereas aim 3 was conducted using qualitative research methods guided by a phenomenological philosophical framework. Phenomenology was the philosophical tradition guiding this research for aim 3, as it examines the lived experience.¹⁶  In this case, the lived experience in aim 3 is that of an MDQ following a concussion from career collegiate sport participation (ie, MDQ decided but not necessarily filed through the National Collegiate Athletic Association).

For aims 1 and 2, we recruited participants through a convenience sample of 6500 ATs contacted through the National Athletic Trainers’ Association (NATA) Research Survey Service for survey completion in spring 2021. Participants were drawn from a diverse, nationwide sample and included if they were certified members of NATA providing clinical care in a collegiate setting. Exclusion criteria included individuals who were not ATs credentialed in their respective state or members of NATA.

For aim 3, we contacted ATs from aims 1 and 2 who indicated that they had been involved in at least 1 MDQ case and demonstrated willingness to complete a follow-up interview (criterion-based sampling). In a phenomenological approach, one would anticipate interviewing no more than 25 participants to achieve data saturation.¹⁶ 

Following an extensive literature search, we determined that no validated surveys existed regarding the decision-making processes of MDQ following concussion; therefore, we created a new survey to answer aims 1 and 2. Face and content validity were assessed by content experts. Following face and content validation, we calculated a content validity index (CVI) with 6 content experts (3 concussion experts, 1 biostatistician, and 2 clinicians) rating the relevance of each survey item, excluding demographics.¹⁷  Reviewers were instructed to rate each item, excluding demographics, based on how each item was relevant to the underlying construct (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant).¹⁷  The CVI-scale score was 0.88 for the entire measure, which is above the acceptable value of 0.80 for 6 raters.¹⁸  However, 3 individual items were deemed not relevant by raters with a CVI-item score below 0.78.¹⁹  Thus, the 3 items were removed, and the new CVI-scale score was 1.00, indicating perfect scale and rater agreement. Because the survey was used to describe ATs’ beliefs regarding what should be included when considering MDQ following concussion, it was determined from a statistical consultation that a reliability analysis was not warranted.^20,21 

The final survey included 3 sections of MDQ experience and specific case information (22 items maximum; sample survey item: “How many medical disqualification cases following concussion have you been involved in?”), MDQ beliefs (18 items maximum; sample survey item: “What do you believe should be considered when making the medical disqualification decision following concussion?”), and demographic items (13 items maximum). Item total varied based on branching logic (see Supplemental Table, available online at https://dx.doi.org/10.4085/1062-6050-0567.23). Additionally, a variety of survey responses were used, including binary, open-ended, multiselect, number bar, rank order, and Likert-scale. The MDQ experience section included asking participants if they had encountered an MDQ case. If participants selected yes, they continued to answer specific items such as how many cases they have been involved in and specific items for each case including who was involved and so on. The MDQ beliefs section included item examples such as “How many concussions do you consider to be too many before MDQ should be considered?” and “Do you believe sex should be considered when contemplating MDQ following concussion?” Examples of demographic items included age, years of experience, etc.

We developed semistructured interview protocols that included a specific series of 11 open-response questions to address the guiding questions for aim 3. Once the interview protocols were developed, a qualitative expert, as well as a content expert, reviewed it to ensure face validity. We asked participants to focus on MDQ cases as related to concussion only. The interview protocol was pilot tested with 2 individuals from the participant group who met the inclusion criteria but who did not participate during data collection. Pilot participants provided the feedback that we should instruct participants to review cases before the interview. No other modifications to the interview protocol were made.

Participants who met the inclusion criteria were contacted via email through the NATA Research Survey Service for aims 1 and 2. Participants answered the MDQ case and specific case information if they had been involved in at least 1 MDQ case following concussion and if they would like to participate in follow-up interviews. If participants had not participated in an MDQ case following concussion, they were surveyed on items related to what they believed should be considered in potential sport disqualification following concussion. Participants were able to access the survey for 4 weeks, with 2 reminders being sent 1 week apart. The survey required about 10 to 15 minutes to complete.

Following survey distribution and completion, we contacted those participants who expressed willingness to participate in follow-up interviews for aim 3. Interviews of ATs used criterion-based sampling of those who had been involved in at least 1 MDQ case following concussion. We sent emails regarding interest in interview participation. Written and verbal consent was obtained before data collection. We obtained general participant demographics before digitally audio recording the interview via Zoom (Zoom Video Communications, Inc). Data collection occurred until data saturation was achieved. Interviews were anticipated to require approximately 45 minutes to 1 hour, and participants were compensated $75 for their time. Audio recordings were transcribed using Zoom. Transcripts were checked for accuracy by a research assistant, who listened to the recordings and read/modified the transcripts simultaneously.

For demographic findings and aims 1 and 2, we calculated descriptive statistics, including means, SDs, medians, ranges, and frequencies. In general, a content analysis was performed to examine the narratives provided by participants for aim 3. Specifically, a 5-cycle process was completed as described by Anderson and Wertz et al.^22,23  Cycles 1 and 2 were completed through an introduction and literature review to clarify the topic of interest in the course of writing this manuscript. Cycle 3 involved collecting and examining data from interviews for meaningful units and codebook creation. To create a codebook, 4 transcripts were randomly selected and read among the author group, looking for repeating and/or interesting ideas. The research team (M.L.W.R., T.C.V.M., and J.D.S.) met using Zoom to discuss any identified meaningful units and to create an initial codebook with themes and subthemes. The initial codebook was then used to code 4 different transcripts, and the research team met again to discuss coding line by line. Following this process, the final codebook was created. The finalized codebook was then applied to all transcripts. Cycle 4 involved synthesizing the findings with the existing literature, whereas Cycle 5 encompassed the development of final interpretations of the findings. Outcomes from Cycles 4 and 5 are included in the discussion of this manuscript.

Data trustworthiness was evaluated in Cycle 3. First, credibility was assessed through member checks.²⁴  To perform member checks, each participants’ interview transcript was emailed to the participant, who had the opportunity to review the transcript. The participant checked for any areas that they wished to further describe within the instructed bounds of not changing the transcript itself.²⁴  This allowed for data to be assessed by the participant, supplemented by any additional information that the individual provided following transcript review, while preserving the original data. In addition, dependability and confirmability of the data were assessed by an external reviewer. The external reviewer received the interview guide, a small number of uncoded transcripts, the finalized codebook, and the corresponding coded transcripts. The external reviewer read through all materials and determined whether there was agreement between their review and the codebook created and implemented by the research team.

For aims 1 and 2, 502 participants completed at least 1 portion of the survey, with 413 completing the survey in its entirety (completion rate = 82.3%). Participant demographic variables are provided in Table 1. Athletic trainers included in our sample had been providing athletic health care for an average of 12.5 ± 10.5 years and managed an average of 7.5 ± 5.7 concussion cases annually. Nearly half of our sample had been involved in at least 1 MDQ case from career collegiate sport participation following concussion (n = 246, 49.0%). Athletic trainers who had been involved in at least 1 MDQ case following concussion had involvement in an average of 2.3 ± 1.9 cases (n = 241).

To address aim 3, we interviewed 20 participants (men = 13, 65.0%; women = 7, 35.0%; average age = 40.7 ± 11.0 years) until data saturation was achieved. Participants had been practicing clinicians for 17.6 ± 10.1 years with 11.3 ± 10.9 years at their current institution. Interview participants managed an average of 9.7 ± 6.8 to 12.6 ± 8.5 concussion cases annually (some participants provided a range; means and SDs were calculated based on end ranges) and had been involved in 3.6 ± 3.0 to 4.0 ± 3.5 MDQ following concussion cases (some participants provided a range; means and SDs were calculated based on end ranges).

Nearly half of respondents had been involved in an MDQ case following concussion (49.0%, n = 246/502; not involved = 51.0%, n = 256/502), with ATs being involved in an average of 2.3 ± 1.9 cases (n = 241). Athletic trainers reported that MDQ patients had 4.6 ± 1.9 concussions, on average. Respondents from our survey sample were involved in a total of 563 MDQ cases following concussion. Individuals included in the MDQ decision-making process most often included the student-athlete (72.6%, n = 409/563), followed by the team physician (69.3%, n = 390/563), head AT (60.6%, n = 341/563), sport-specific AT (53.3%, n = 299/563), parents/guardians (44.9%, n = 253/563), neurologist (40.1%, n = 226/563), coach (28.4%, n = 160/563), neuropsychologist (14.0%, n = 79/563), primary care physician (12.4%, n = 70/563), sport psychologist (8.2%, n = 46/563), other (3.2%, n = 18/563), and teammates (0.01%, n = 6/563). Respondents who indicated “other” noted involvement from athletic directors (16.7%, n = 3/18), psychologists/counselors (16.7%, n = 3/18), school support staff (11.1%, n = 2/18), concussion specialists (5.6%, n = 1/18), siblings (5.6%, n = 1/18), neuromechanists (5.6%, n = 1/18), and faculty who specialize in concussion research (5.6%, n = 1/18).

Percentages, frequencies, and importance ranking for each item related to MDQ following concussion beliefs are included in Table 2. Respondents believed most frequently that team physicians, student-athletes, and sport-specific ATs should be involved in the MDQ following concussion decision-making process (Table 2). The ATs in this study who answered the item believed that 5 concussions were too many before MDQ should be considered (median = 5, interquartile range = 3.00–6.00, n = 433). Approximately 77% of respondents believed that sport contact level should be considered in the MDQ decision-making process, with contact/collision considered the most influential in the MDQ decision-making process, followed by limited contact and noncontact. Nearly 78% of participants believed that sex should not be considered when contemplating MDQ following concussion. Athletic trainers most frequently perceived that an athlete’s concussion history should be considered in the MDQ decision (87.1%, n = 437), followed by long-term health (86.1%, n = 432), medical professionals’ opinions and recommendations (85.7%, n = 430), and long-term health-related quality of life (85.7%, n = 430). Additionally, respondents believed that the athlete’s concussion history should be most influential, followed by recovery time and athlete preference.

Following analysis, we discovered 6 overall themes. This manuscript focuses on the theme and subthemes for factors ATs use in the decision-making process. Athletic trainers described numerous factors involved in weighing the decision of MDQ for athletes. Factors included the number of concussions (both diagnosed and undiagnosed), long-term health consequences and future well-being, health-related quality of life, symptom severity, disclosure, academics, and length of recovery. Table 3 includes additional evidence for each subtheme associated with the theme.

Participants described that the overall number of concussions sustained played a role in their MDQ following concussion decision-making process. One participant noted that the number one factor that contributed to their decision was “the number of concussions that the student-athlete has sustained, both within a set time frame and over the course of their lifetime.”

Participants did not always have a set number of concussions necessary to have a discussion; however, others did, noting, “I’ve always been told 3. That’s what it was at one of the schools I was at, it was after 3, we need to have a discussion with the physician to determine if this is really safe for that athlete.”

Other factors that participants described were long-term health consequences and future well-being. One participant noted, “It’s overall health and well-being and his well-being in the future that led us to disqualify him.” One other participant described specific medical conditions:

A patient’s overall health-related quality of life also played a role in the MDQ following concussion decision-making process. One participant described, “…One of the things is quality of life for the student-athlete after they finished playing, you know, where they’re going to be 10, 20, 30, 40 years from now.”

Participants also described symptom severity as a factor, saying that MDQ should be considered “if the athlete is still kind of like my athlete where they’re struggling with their symptoms a little bit, they’re still taking the medication, they can’t do exercises because they’re having symptoms.”

Concussion disclosure was a subtheme from our findings in participants recounting how that factored into the MDQ decision-making process:

Participants not only noted concussion disclosure when seeking care for the current concussion but also how disclosure plays a role in the overall number of concussions disclosed, saying,

Several participants discussed the role of academics. One participant noted,

Last, the length of recovery was an indicator that MDQ may be considered. This participant described the length of recovery for each successive injury:

The aims of this study were 3-fold: (1) describe the MDQ following concussion cases ATs have been involved in, (2) describe the beliefs of the AT regarding MDQ following concussion, and (3) explore the factors ATs believed should be involved in the MDQ following concussion process. Deciding MDQ or retirement from sport following concussion is a difficult decision sport medicine professionals face, often with limited evidence to inform decisions. Our results indicate that, regardless of the available evidence, nearly half of our sample was involved in MDQ following concussion, and of those, the average number of MDQ cases encountered was 2. Additionally, participants interviewed generally had more years of experience, managed more concussions annually, and had more involvement in MDQ cases following concussion than the surveyed sample. Participants believed that many factors should be considered in the MDQ decision-making process, including sport contact level, concussion history, future well-being, and how the injury impacted the patient’s current life aspects, such as quality of life and academics.

The decision to stop sports participation following concussion has been historically rooted in a 3-strike rule first described by Thorndike in 1953.²⁵  The 3-strike rule has continued to be an accepted criteria for MDQ following concussion even though its roots are in expert opinion and not empirical data.⁸  The acknowledgement of the 3-strike rule was also apparent in our findings, with participants describing that patients may not be truthful of a third concussion to health care providers and therefore only state a concussion history of 2. However, ATs described an average of 4.6 concussions in the self-reported data of how many concussions had been experienced by each patient. Additionally, respondents noted that they believed that 5 concussions were too many before MDQ should be considered. From the available literature, athletes themselves reported that they would retire from sport participation after 3 or 4 concussions.²⁶  Currently there is no threshold for concussive injuries when considering an athlete for MDQ. Clinicians, like those described in our sample, should consider each patient individually and holistically with a variety of factors to determine if sport participation is no longer safe for an athlete following concussion.

The current literature cites that no evidence exists to recommend an absolute indicator for retirement or MDQ from sport participation following concussion.²⁷  Collectively, we offer 8 factors that may influence cessation of sport participation following concussion as guided by the existing literature and our findings, including (1) concussion history, (2) persistent diminished academic or workplace performance, (3) persisting concussion symptoms, (4) structural abnormalities in neuroimaging, (5) prolonged recovery, (6) perceived reduced threshold of sustaining recurrent concussions, (7) psychosocial factors, and (8) future risk considerations.^4–8,27  Results from our study mirrored several of these factors to consider in the MDQ decision-making process, including concussion history, impact on academic performance, sport-specific factors, and recovery length. Additionally, Makdissi et al recommend an interprofessional approach, including input from the individual, parents/guardians, health care professionals, and school personnel, when necessary.²⁷  This was also corroborated in our results, as the most frequent individual involved in MDQ following concussion was the patient, followed by various health care professionals and parents/guardians. Makdissi et al also states that the final decision for MDQ is up to the patient and/or their parent/guardian, especially because the autonomy of an MDQ decision was shown to be associated with lower levels of anxiety and depression later.^27,28  Interestingly, athlete preference was rated frequently and with high importance in the MDQ decision from our sample, and nearly 50% of our sample believed parents/guardians should be involved. Unfortunately, it is not always a viable option to incorporate athlete preference given institutional risk and policies; however, having collective conversations with the patient and many individuals important to the patient where information is presented and questions are answered may create a seemingly shared decision. Recommendations from Makdissi et al were recently published in 2023 and seem to be contrary to our findings; therefore, future research should continue to examine perceptions of who should be involved in the MDQ decision-making process following concussion, roles and responsibilities, how they should be consulted, and, importantly, the weight of those views.²⁷ 

Several resources are available to help clinicians obtain information and support the patient following their MDQ decision. Patient-reported outcomes can be useful to gain objective information to aid in the decision-making process and, once the decision has been made, to medically disqualify. The information obtained in patient-reported outcomes could help initiate conversations and provide objective data to present to the patient when MDQ is considered. Especially because athletic identity is high in athletes, it may be difficult to stop sport participation. Patient-rated outcomes can also assist clinicians to ensure support even after the decision has been made. Supporting self-acceptance and reframing interests may also assist in restricting views high in athletic identity and ensure that new goals and career planning offer new directions.^29,30  Other resources like Sidelined.org provide other support resources for patients who have been medically disqualified with a support network. Clinicians can also consider if the patient may be involved in other potentially lower-risk sports or have a different role within their current team to enhance social connection and belonging. DeFreese et al found that having no transition plan following sport participation discontinuation was associated with higher depression and anxiety levels in former athletes.²⁸  Given this, it is imperative to support the patient not only during the MDQ decision-making process but also in the transition from sport participation.

There are limitations to our study. Responses from our study are not generalizable to those who did not complete the survey or interview. Additionally, although the current literature suggests that patients drive the MDQ decision-making process following concussion, our results presented views from clinicians only.²⁷  Future research should continue to examine the long-term health consequences and outcomes of concussion, examine MDQ practices among clinicians while incorporating the patient perspectives of those who have been medically disqualified following concussion, and examine why certain individuals should be involved in this process over others. As this study was only completed in health care professionals at the collegiate setting, future research should examine MDQ following concussion at other levels, which likely will change the frequency of involvement of individuals, especially parents/guardians. Last, many of our multiselect survey items did not contain a “none” item response, which may have forced participants to skip the item; however, we believe that our results are still important to present, as limited information on this topic exists. Future studies should include a “none” item response to capture this.

Deciding when sport participation is no longer viable is a difficult decision with limited evidence to guide patient and clinician decisions. However, our results indicate that numerous clinicians are embarking on this process and considering multiple factors to guide their decision in conjunction with multiple partners. Clinicians believed sport contact level, concussion history and recovery length, and the patient’s overall well-being, including health-related quality of life and academic performance, should be considered in the MDQ following concussion decision-making process. These findings along with others presented here can be used to assist clinicians in the MDQ following concussion decision with consideration of patient preference if feasible. Future research should continue to examine MDQ following concussion, especially to incorporate the patient perspective.

Supplemental Table. Medical Disqualification Following Concussion Survey^a

Found at DOI: https://dx.doi.org/10.4085/1062-6050-0567.23