Navigating Athletic Training Position Statements: The Strength of Recommendation Taxonomy System

An evidence-grading taxonomy is a practical system that helps readers readily assess the relevance and validity of evidence that supports a clinical recommendation. Clinicians should understand how a grading taxonomy is applied and what the level of evidence and strength of recommendation mean before using the recommendations to inform clinical practice. The NATA has adopted the Strength of Recommendation Taxonomy (SORT)⁴  grading system to provide context to the evidence supporting recommendations in position statements. The purpose of this commentary is to provide an overview of the SORT, the process of generating a strength of recommendation (SOR), and the interpretation of the SOR as it relates to recommendations made in position statements.

Evidence-based practice is the integration of a clinician's expertise (internal evidence), the findings from published research (external evidence), and the contextual experiences, values, and preferences of patients (patient evidence) to inform clinical decision making for the promotion of positive patient outcomes.⁵  These 3 sources of evidence (internal, external, and patient) result in dynamic interplay during the clinical decision-making process.

One of the most challenging factors in evidence-based practice is gauging the value of published research (external evidence) and its role in informing clinical decisions. The SORT was developed for this reason.⁴  Research designs use many forms to capture important factors that may affect humans from the cellular to societal level. On a scale of control, these research designs range from basic research to epidemiologic investigations.⁶  Along this scale, a subset of applied research involves investigations of patients, known as clinical research. Clinical research typically centers on pathologic conditions (diseases and injuries) that occur within a given population. The SORT was specifically designed to help practicing clinicians gauge the external evidence derived from clinical research when various sources of information are considered together in the literature (eg, systematic review, clinically appraised topics). The SORT is advantageous to clinicians, researchers, and educators because it is simple to use, includes various forms of internal and external evidence, and addresses the outcomes that are important for clinical decisions.

Not all evidence is created equally,⁷  and the SORT provides a universal system for ranking the evidence used to support clinical recommendations.⁴  The SORT has been adopted by several health care professional organizations along with the NATA, including the American College of Sports Medicine and the American Academy of Family Physicians. The SORT is used to (1) evaluate the characteristics of the evidence used to support a recommendation by classifying the level of evidence (LOE) and (2) develop a weighted recommendation based on the collective evidence derived from research or clinical practice or both, known as the SOR. Within position statements, LOEs are applied to individual sources of evidence, whereas each recommendation receives an SOR to provide context regarding the collective evidence used to support it.

The SORT considers 3 main categories of clinical research: diagnosis (factors that help identify the disease or injury); therapy, prevention, or screening (factors that influence the development or resolution [or both] of the disease or injury); and prognosis (the likely sequelae for a patient who develops the disease or injury).⁴  The different categories of clinical research designs uniquely contribute to the collective body of research evidence from observation to intervention.⁶  Within each clinical research category, a hierarchy of evidence exists, stratified into 3 levels (ie, LOE 1–3).⁶  The nature of the outcomes, the quantity and quality of the external evidence, and the consistency of the results affect how evidence is rated on the SORT (Table).

The SORT provides a grading system for the recommendations derived from the external evidence for clinical practice. Individual articles are evaluated using the 3 levels of evidence (LOE 1–3), whereas the body of literature is graded using the SORs (A–C). Grade A is given to recommendations based on level 1 evidence with consistent results across studies; grade B is reserved for recommendations based on limited quantity, quality, or consistency (or some combination of these) of patient-oriented evidence (Figure 1). Grade C classifies recommendations based on level 3 external evidence (Figure 1). Grading is vital for health care providers seeking to determine how to apply recommendations in their clinical practices. In this way, the NATA position statements provide readers with immediately interpretable recommendations from the writing groups.

However, to understand how the NATA establishes an SOR grade, it is crucial to understand how the quality of evidence is evaluated. The first factor for determining the SORT LOE is the nature of the outcomes (dependent variables) used in clinical research. Within the SORT, outcomes of interest are separated into patient-oriented and disease-oriented outcomes.⁴ Patient-oriented outcomes are variables measured in research studies that are typically important to a patient (eg, decrease in pain or symptoms, improved quality of life, the ability to return to play after an injury or illness). Readers should note that patient-oriented outcomes as recognized by SORT can encompass a wide variety of variables that are not solely in the format of patient-reported outcome measures. Patient-reported outcome measures are specific assessments using validated questionnaires to quantify symptoms that represent 1 category of patient-oriented outcomes. Disease-oriented outcomes are markers of biological or physiological changes (or both) within the patient (eg, increases in range of motion, strength improvements, or changes in blood cell count). Disease-oriented outcomes contextualize the potential changes in patient-oriented outcomes. However, by themselves, disease-oriented outcomes do not necessarily indicate changes in health status or patient-oriented outcomes. As a result, disease-oriented evidence rates lower on the SORT LOE scale. However, both patient- and disease-oriented outcomes can be important to the clinician. Clinicians use a combination of disease- and patient-oriented progress evaluations during recovery or rehabilitation to capture the dynamic interplay of biopsychosocial factors associated with their clinical decisions.

Within each level of clinical research evidence, the SORT favors a higher quantity of external evidence in the form of systematic reviews or meta-analyses.⁶ Systematic reviews and meta-analyses are the synthesis of multiple studies to answer a particular question within the 3 categories of clinical research evidence. The highest LOE on the SORT is, therefore, reserved for patient-oriented evidence synthesized from multiple studies.⁴  Higher-quality evidence is also favored in the SORT. Results of studies with more control and better generalizability across the 3 categories of clinical research are ranked higher in the SORT. The results of studies with less control and generalizability are ranked lower and should be interpreted with more caution. Factors such as blinding, random allocation, and adequate sample size and spectrum of patients increase the quality (control and generalizability) of the evidence.⁸ 

The last factor is the consistency of the results across multiple investigations.⁹  As stated earlier, the SORT favors systematic reviews and meta-analyses over individual studies because they evaluate the consistency of evidence across multiple studies. One of the major hallmarks of science is that results are independently and consistently reproducible. In reality, results are often inconsistent, even when similar designs are used. Outcomes that display consistent results across studies are graded higher on the SORT; inconsistent results across studies are graded lower.⁴ 

Position and consensus statements bring together much of the research on a given topic (eg, prevention of anterior cruciate ligament injuries, abuse of anabolic steroids, psychological concerns at the secondary school level).^10–12  The evidence reviewed by position statement writing groups is organized into concise, stand-alone recommendations at the beginning of the document that health care providers can (ideally) directly implement in their clinical practice. The authors⁴  of the article that introduced the SORT provided a detailed explanation of how to use and implement the scoring and grading system (Table). Level 1 evidence (good-quality patient-oriented evidence) includes systematic reviews and meta-analyses of well-designed studies incorporating patient-oriented outcomes that demonstrated consistent results across studies. Level 2 (poor-quality patient-oriented evidence) is reserved for patient-oriented evidence of limited quantity and quality with findings that may be inconsistent across studies. Level 3 (other evidence) captures evidence derived from disease-oriented outcomes, published expert opinions (eg, expert consensus, typical practice), or published clinical observations (eg, case series). Ultimately, the position statement authors choose the LOE based on the study's design, outcomes, and quality. As of 2016, readers could find LOEs for all references cited in NATA position statements.

For each recommendation in a position statement, readers will see the graded SOR. This can be visualized as a continuum (Figure 2) from do not recommend to strongly recommend. Weaker recommendations are in the middle of that continuum.

Strength of recommendation A is defined as a “recommendation based on consistent and good-quality patient-oriented evidence.”⁴  For recommendations that are beneficial, or assessed using SOR A, the clinician should feel secure that the recommendation is firmly supported by patient-oriented research and should strongly consider its implementation in clinical practice. The supporting evidence is strongly in favor of that intervention (offer the intervention and similar results can be expected in similar circumstances). An example of an SOR grade of A, with a strong recommendation to offer intervention is shown in Figure 2. It is important to note that grade A recommendations can be generated for interventions that are detrimental; the supporting evidence can be strongly against the use of a given intervention (do not offer the intervention as the same poor results are expected). For example, in the position statement on superior labral anterior-posterior injury,¹³  the recommendation to not use a history of clicking, popping, or catching as a diagnostic criterion was given an SOR of A. Thus, athletic trainers were strongly advised to not use this evaluation method when making a clinical decision. One additional note: even with a grade A recommendation, clinicians should recognize that similar results should be expected when they follow the recommendation but only if the circumstances are also similar.

Strength of recommendation B is defined as a “recommendation based on inconsistent or limited-quality patient-oriented evidence.”⁴  The recommendation is based on evidence that is inconsistent, but the outcomes used to generate that recommendation were patient oriented. The SOR is weak, either for or against a given intervention. Each clinician should make an individual decision regarding the relative importance of incorporating that recommendation into clinical practice and in educating the patient. An example of an SOR grade of B, with a weaker recommendation to offer an intervention, is provided in Figure 3.

Strength of recommendation C is defined as a “recommendation based on consensus, usual practice, opinion, disease-oriented evidence, or case series.”⁴  This recommendation is based on studies that are typically at the lowest level of the clinical research hierarchy. Yet in some cases, disease-oriented evidence describes critical outcomes that are important to clinicians. For example, range of motion (ROM) is generally considered a disease-oriented outcome that can be very meaningful in clinical practice. A recommendation for an intervention that increases ROM can be based on consistent evidence but graded C in the SORT system. The clinician should be able to recognize the advantages (eg, the intervention is good for increasing ROM) and limitations (eg, increasing ROM may not affect patient-oriented outcomes, such as morbidity, pain, or the return-to-play time frame) of the recommendation. In contrast, a C recommendation could also be applied to an expert clinician's observation of patients with atypical presentations or conditions in which a higher LOE may not be appropriate. For example, many recommendations regarding lightning have a grade C because they were largely based on expert opinions rather than randomized controlled trials.

A caveat for some SOR C recommendations is that it is not always possible to conduct an LOE 1 study type of patients with certain injuries or conditions. Smith and Pell¹⁴  pointed out that designing a randomized controlled trial to determine if parachutes can prevent death after free fall would be ludicrous, so challenging the results of observational studies looking into such matters would be the same. For example, a C recommendation for applying an automatic external defibrillator within 3 minutes of a sudden cardiac event would not mean a clinician does not need to follow that recommendation. Designing a randomized controlled trial in which withholding an automatic external defibrillator for different times would be unethical. Therefore, a C grade for this recommendation simply means that although research is limited, based on the available evidence, clinicians should still follow the recommendation. It is then important for readers to consider the evidence in the context of their unique clinical environments and the potential outcomes associated with not implementing the recommendation. In contrast, a C grade for a recommendation about the use of core exercises to prevent a low back injury may mean that more research is needed in this area. In this case, clinicians should incorporate their clinical experiences and the available research and then weigh the patient's and the clinical site's unique factors to decide if implementation of core exercises is warranted. An example of an SOR grade of C, with a weaker recommendation (as compared with grades A and B) based on the nature of the outcomes (disease oriented), is given in Figure 4.

Position statements need to be continually updated as new external evidence becomes available. Further, position statements are only one aspect related to the standard of care, which encompasses many other aspects: state practice acts, standing orders (if applicable), situational factors, educational competencies, and institutional policies. The ranks (LOEs) of individual studies and grades of recommendations (SORs), both separately and in combination, provide the foundation for position statements supporting an athletic trainer's standard of care.

The NATA Pronouncements Committee strives to guide author groups in developing thorough and clinically applicable position statements. We understand that clinicians may not have the time to read all the literature on a topic and, therefore, may depend on position statements to help summarize key concepts in the area. Clinicians who use position statements to determine best practices should feel confident in understanding the SORT system. Levels of evidence are based on study design and quality, whereas SORs are based on the number and consistency of studies that support a recommendation. Readers should look for both LOEs and SORs in position statements to fully comprehend each recommendation and its application to clinical practice.