Context

Incorporating patient-reported outcomes (PROs) into daily routine is essential for patient-centered clinical practice. Secondary school athletic trainers (ATs) may encounter unique barriers that limit their willingness to use PROs.

Objective

To explore how secondary school ATs who were using PROs perceived their application, benefits, and problems compared with those who did not.

Design

Cross-sectional study.

Setting

Web-based survey.

Patients or Other Participants

A total of 2984 secondary school ATs received an e-mail invitation, and 322 completed the survey (response rate = 10.8%). Respondents were 43 ± 10 years old, with most indicating at least 11 years as a Board of Certification–certified AT (n = 276, 85.7%).

Main Outcome Measure(s)

The ATs were invited to complete a Web-based survey regarding the uses and benefits of and problems with PROs. Those using PROs in clinical practice were asked their criteria for selecting the measures, whereas those not using PROs were asked their reasons for not using them. Dependent variables were endorsements of uses and benefits of and problems with PROs.

Results

The most commonly cited uses of PROs were determining treatment effectiveness (193/264, 73%) and demonstrating effectiveness to administration (174/264, 66%). Improving communication with the patient (267/296, 90%) and helping to direct the plan of care (256/297, 86%) were the most frequently endorsed benefits of PROs. Time to score and analyze (152/284, 53%) and time for patients to complete (134/284, 47%) PROs were the problems encountered most often. For ATs not using PROs (223/262, 85%), the most frequent reason was the lack of a support structure (102/219, 46%). For ATs using PROs (39/262, 15%), quick completion times (32/39, 82%) was the most common criterion used to select individual measures.

Conclusions

A majority of secondary school ATs recognized the benefits of PROs and yet did not use them in clinical practice due to setting-specific barriers.

Key Points
  • Collecting patient-reported outcomes (PROs) is an important component of patient-centered care and evidence-based practice.

  • However, the use of PROs in athletic training practice was limited. Previous authors determined that ATs in the secondary school setting used them the least.

  • Despite the perceived benefits of collecting PROs in clinical practice, strategies to address use of the data and time demands are necessary to increase their application in the secondary school setting.

Evidence-based practice and patient-centered care are core competencies suggested for all health professionals by the Institute of Medicine1  and are becoming more prevalent in the professional education for the athletic trainer (AT).2  Including evidence-based practice content in educational competencies and certification requirements is one of the more recent efforts to bring ATs in line with other health professionals and improve patient care.13  Evidence-based practice uses the best available research, clinician expertise, and patient values to make clinical decisions and achieve the best outcome relative to the patient.4,5 

Patient-centered care recognizes the uniqueness of each patient and respects the different priorities he or she has concerning a health condition.6  Given the same health condition, what is important to one patient may not be important to another.6,7  Previous experiences, personal beliefs, expectations, and the amount of health care information that patients have access to all shape their reactions and preferences regarding their health condition.8  Patient-centered care leads to an educated patient and shared decision making between patient and clinician.1  The benefits of patient-centered care include better overall patient outcomes, increased patient compliance, and a stronger relationship between provider and patient.6  Identifying and understanding how a patient interprets the injury and responds emotionally and behaviorally is important to providing patient-centered care.911  Regular assessments of clinical and patient-reported outcomes help clinicians capture how the patient perceives an injury and the care provided.1012 

Using clinically oriented outcomes to inform and guide clinical practice in athletic training is common and involves disease-oriented evidence.11,1315  These criteria provide good measures of clinical improvement but give little information on how the patients themselves perceive their progress. Patient-reported outcomes (PROs) emphasize the patient's perception of his or her overall disablement as a result of an injury or illness.9,13,16  Multiple health care professions have used PROs to involve patients in their own care, improve interaction between the patient and clinician, guide and enhance clinical decision making, and demonstrate the effectiveness of the profession.7,12,1719  Patient-reported outcomes are a foundation of patient-centered care and will be vital to the athletic training profession in its continual evolution in the allied health field.10,12,16,17 

Despite the importance of PROs, evidence of their use in the athletic training profession is limited.10,11  Lack of knowledge, little administrative support, and the perceived time to administer these outcome instruments have all been suggested as reasons that ATs do not routinely collect this information.10,20  One of the few groups10  that looked into the use of PROs in the athletic training profession noted that, among their respondents, those employed in the secondary school setting most often reported not using PROs as a part of their clinical practice. According to the most recent National Athletic Trainers' Association (NATA) member statistics, the secondary school setting is the second largest practice setting (22.2%), behind the college and university setting (22.8%).14  Secondary school access to athletic training professionals has expanded greatly over the last 10 years, with as many as 70% of schools now having access to some form of athletic training services.15  Athletic trainers in the secondary school setting can have a unique effect on the health care provided to youths and the decisions their parents make. These clinicians have a responsibility to understand the evolution of the profession and to strive to improve the quality of care provided. As educational competencies and certification requirements continue to evolve, it is clear that the principles of evidence-based practice and patient-centered care will be a large part of the future athletic training profession.2,3  Understanding how PRO measures are used in this setting and which factors affect ATs in implementing such measures is an important first step. The purpose of this investigation was to compare how secondary school ATs who used PROs and those who did not use PROs perceived the applications and benefits of and problems with these measures.

METHODS

Study Design

We used a cross-sectional design to assess the knowledge and practices of secondary school ATs, with 3 primary goals: (1) to validate previous findings10  regarding the use of PROs by secondary school ATs; (2) to identify their familiarity with information related to PROs and their use of these measures in clinical practice; and (3) to determine how those who used PROs perceived their application, benefits, and problems compared with those who did not. We distributed an e-mail with a survey link to NATA members who were Board of Certification (BOC) certified and listed the secondary school as their employment setting.

Participants

We used 2016 NATA membership statistics to identify 3011 BOC-certified ATs who listed the secondary school as their employment setting. The e-mail addresses of these professionals were obtained through an application process offered by the NATA. The study was reviewed by the institutional review board at Indiana State University and approved as exempt. Participants implied consent by accessing and completing the survey instrument.

The e-mail invitation to participate in this study was delivered to 2984 of the addresses provided by the NATA and yielded 322 completed surveys, for a response rate of 10.8%. Respondents were 43 ± 10 years old, with most indicating BOC certification for more than 11 years (276/322, 85.7%). Of those indicating their route to certification, internships (150/322, 46.5%) and professional bachelor's degree programs (151/322, 46.8%) were split almost evenly; the remainder specified professional master's degree programs (18/322, 5.6%). Complete demographic data are shown in Table 1.

Table 1

Demographic Characteristics of Respondents

Demographic Characteristics of Respondents
Demographic Characteristics of Respondents

Instrument

Previous authors have conducted surveys to investigate the benefits of and barriers to using PROs in physical therapy and athletic training. Jette et al21  developed and validated a survey instrument designed to investigate the uses and benefits of and barriers associated with outcome measures in physical therapy. The instrument was modified by Snyder Valier et al10  for a 2014 study that examined the benefits of and barriers to using PROs in athletic training. To construct our questionnaire, we modified the instruments used in these previous studies to include familiarity items. For content analysis, we solicited feedback on the entire instrument from an expert panel of 4 clinicians, all of whom had research experience in the field of PROs; they had been BOC certified for an average of 6 years. Three members of the panel served as research experts, based on their work with PROs (h index = 10 ± 5). One member of the panel was a research specialist for a large injury-research and -prevention center. To conduct the content analysis, we shared the items and responses with the panel. They were asked if each item was sufficient as written or needed attention. For items that needed attention, the experts provided suggestions for improvement, and we revised the instrument accordingly.

The survey presented all participants with 46 questions; 2 additional items were available to those who indicated they were not currently using PROs in clinical practice, and 1 additional item was provided to those who were currently using PROs. The instrument was organized into sections: demographics (n = 12), familiarity with patient outcome measures (n = 6), uses of PROs in clinical practice (n = 7), benefits of PROs (n = 8), and problems with using PROs (n = 13). The familiarity items addressed the degree to which the participants were familiar with clinician-rated outcomes, PROs, the disablement model, and patients' perceptions of their own injuries. We also asked participants if they had been exposed to clinician-rated outcomes and PROs and to which modes of education (ie, professional education, postprofessional education, continuing education, evidence-based continuing education, literature, or colleagues) or if they had not been exposed. Participants indicated whether they understood the differences between the outcome measures and the role of outcomes in evidence-based clinical practice. They were then asked if they currently used PROs in their clinical practice. Those who answered yes were asked the criteria for selecting PROs and the specific tools used. Those who answered no were asked the reason for not using PROs and whether they planned to implement PROs in their practice. The core content of the questionnaire included a statement about PROs and asked participants to rate their level of agreement or familiarity with the statement. For the familiarity questions, a 5-point Likert-style scale measured how familiar the respondent was with the statement presented (5 = extremely familiar, 4 = very familiar, 3 = moderately familiar, 2 = slightly familiar, 1 = not at all familiar). For the statements on the uses and benefits of and problems with PROs, the participant indicated his or her level or agreement (5 = strongly agree, 4 = agree, 3 = neither agree nor disagree, 2 = disagree, 1 = strongly disagree).

Procedures

Using e-mail addresses provided by the NATA, we supplied secondary school ATs with information about our study and a hyperlink to the survey instrument, which was housed in a secure Web-based survey system (Qualtrics LLC, Provo, UT). Those who selected the hyperlink were presented with information on informed consent; consent was implied by selecting the next button to start the survey. A reminder e-mail was sent to all potential participants 2 weeks after the initial invitation. Data collection concluded after 4 weeks.

Data Analysis

The dependent variables were the endorsements of statements regarding familiarity with, uses and benefits of, and problems with PROs by the ATs who completed the survey. Familiarity questions were considered endorsed when participants selected a rating of extremely familiar, very familiar, or moderately familiar. Responses to Likert-style questions were considered endorsed when participants selected a rating of agree or strongly agree for the uses and benefits of and problems with PROs.

Descriptive statistics were used to record demographic data, whether or not the participant used PROs in clinical practice, and the frequency with which participants agreed with the benefits and barrier statements. We calculated composite scores for the uses and benefits of and problems with PROs subsections and compared those who used and those who did not use PROs in clinical practice by calculating a Mann-Whitney U inferential test statistic. We also compared respondents based on educational preparation using a Kruskal-Wallis inferential statistic and Mann-Whitney U pairwise comparisons. We were unable to compare samples based on years of experience due to a sample size of fewer than 5 in one category. All data analysis was completed using SPSS (version 22; IBM Corp, Armonk, NY) with the level of significance set at P < .05, except for the pairwise comparisons, where significance was set at P < .01 to avoid a type 1 error. Because we ascribe to the principles of voluntariness outlined in The Belmont Report (www.hhs.gov/ohrp/regulations-and-policy/belmont-report/index.html), which provides the ethical principles and guidelines for the protection of human subjects research, we allowed participants to answer any questions they liked. This resulted in various numbers of respondents per item, which are indicated throughout the “Results” section. Because missing data did not occur randomly, it was difficult to eliminate bias using missing-data–management techniques (eg, listwise or pairwise deletion, multiple imputation, maximum likelihood methods, Bayesian methods).22  Therefore, we used partial data analyses, which were consistent with previous research of this type.

Due to the low response rate, we compared early and late respondents on the composite scores for the uses and benefits of and problems with PROs subsections using a Mann-Whitney U inferential test statistic. The first 50 respondents were not statistically different from the last 50 respondents in their perceptions of uses (P = .577) or benefits (P = .646) of or problems (P = .579) associated with PROs. Late and nonrespondents were thought to answer similarly, and our analysis indicated that the findings likely did not exhibit nonresponse bias.

RESULTS

Participants who completed the applications and benefits of using PROs sections of the survey had a high rate of endorsement, with answers of either agree or strongly agree for all of the statements presented (Tables 2 and 3). The most frequently endorsed benefits were improving communication between patient and clinician (267/296, 90%), helping to direct the plan of care (256/297, 86%), leading to better patient outcomes (246/294, 83%), and motivating and encouraging patients (251/309, 81%). Detailed data on the benefits endorsements are included in Table 2. The most commonly endorsed uses of PROs were examining a change in patient status to determine the effectiveness of a treatment (193/264, 73%), demonstrating effectiveness to administration through patient outcomes (174/264, 66%), documenting patient outcomes by individual clinicians (176/265, 66%), and determining the effectiveness of individual clinicians (165/265, 62%). Table 3 contains further detail on the endorsed uses of PROs.

Table 2

Perceived Uses of Patient-Reported Outcomes Measures in Clinical Practice (Likert Score)

Perceived Uses of Patient-Reported Outcomes Measures in Clinical Practice (Likert Score)
Perceived Uses of Patient-Reported Outcomes Measures in Clinical Practice (Likert Score)
Table 3

Perceived Benefits of Patient-Reported Outcomes Measures (Likert Score)

Perceived Benefits of Patient-Reported Outcomes Measures (Likert Score)
Perceived Benefits of Patient-Reported Outcomes Measures (Likert Score)

Participants also had the opportunity to respond to statements concerning the perceived problems of using PROs. The most frequent problems were taking too much time to score or analyze the results (152/284, 53%), taking too much time for patients to complete (134/284, 47%), and confusing to patients (82/283, 29%). Complete data on the endorsed problems are listed in Table 4.

Table 4

Perceived Problems With Patient-Reported Outcomes Measures (Likert Score)

Perceived Problems With Patient-Reported Outcomes Measures (Likert Score)
Perceived Problems With Patient-Reported Outcomes Measures (Likert Score)

When asked whether they used PROs in clinical practice, only 262 participants responded. Few (n = 39, 15%) selected yes; a majority (n = 223, 85%) indicated that they were not using PROs. Participants who were not using PROs were presented with a list of possible reasons and asked to choose all that applied. The most frequent reasons for not using PROs were requiring a support structure that was not available (102/219, 46%); taking too much time to analyze, calculate, and score (90/219, 41%); and providing information that was too subjective to be useful (89/219, 40%; Table 5). In an open-ended portion of this question, participants were given the opportunity to state other reasons for not using PROs. A total of 52 comments were categorized into common themes: those cited most often were no time or resources (n = 16, 30.8%), just haven't done it, hadn't thought about it, probably a good idea (n = 10, 19.2%), and unfamiliar with or unsure of how to use—wasn't taught in my educational program (n = 8, 15.4%; Table 6).

Table 5

Reasons for Not Using Patient-Reported Outcomes Measures by Those Who Indicated They Did Not Currently Use Them (n = 219)

Reasons for Not Using Patient-Reported Outcomes Measures by Those Who Indicated They Did Not Currently Use Them (n = 219)
Reasons for Not Using Patient-Reported Outcomes Measures by Those Who Indicated They Did Not Currently Use Them (n = 219)
Table 6

“Other” Reasons for Not Using Patient-Reported Outcomes Measures (Open-Ended Responses)

“Other” Reasons for Not Using Patient-Reported Outcomes Measures (Open-Ended Responses)
“Other” Reasons for Not Using Patient-Reported Outcomes Measures (Open-Ended Responses)

Generally, respondents indicated they were moderately familiar with clinician-rated outcomes (mode = 3) and PROs (mode = 3) and agreed that they knew the difference between the measures (mode = 4). However, respondents were not familiar with the disablement model (mode = 1) or how the disablement model and the patient's perception of his or her injury were related (mode = 1). They understood that PROs are a component of evidence-based medicine (mode = 4). One-third (n = 118, 36.9%) of respondents had not been exposed to clinician-rated outcomes in any way. Similarly, about one-third of respondents had not been exposed to PROs (n = 103, 32.2%). Other participants indicated they had been exposed to both clinician- and patient-rated outcomes through various sources but predominantly through the literature (clinician-rated outcomes = 128, 40.0%; PROs = 146, 45.6%; Table 7).

Table 7

Sources Where Participants Have Gained Exposure to Clinician- and Patient-Rated Outcomes Measures

Sources Where Participants Have Gained Exposure to Clinician- and Patient-Rated Outcomes Measures
Sources Where Participants Have Gained Exposure to Clinician- and Patient-Rated Outcomes Measures

The 39 respondents who were currently using PROs in clinical practice were presented with a list of criteria for selecting measures and asked to choose all that applied. The criteria chosen most frequently were the ability to be completed quickly (32/39, 82.1%), easy for patients to understand (31/39, 79.5%), and most appropriate for types of conditions in my setting (24/52, 61.5%). Selection criteria are listed in Table 8.

Table 8

Criteria Used to Select Patient-Reported Outcomes Measures From Those Who Indicated They Currently Used Them (n = 39)

Criteria Used to Select Patient-Reported Outcomes Measures From Those Who Indicated They Currently Used Them (n = 39)
Criteria Used to Select Patient-Reported Outcomes Measures From Those Who Indicated They Currently Used Them (n = 39)

When comparing respondents who used (n = 39) with those who did not use PROs (n = 223), we identified statistical differences in the composite scores for perceived problems (U = 3002.5, Z = −3.093, P = .002), whereby those not using PROs perceived greater problems (2.9 ± 0.5) than those currently using PROs (2.7 ± 0.6). Those currently using PROs demonstrated more agreement (U = 3261.0, Z = −2.516, P = .012) with the statement that there were benefits to using PROs (4.3 ± 0.5) than did those not using PROs (4.0 ± 0.6). We did not find statistical differences (U = 3576.0, Z = −1.746, P = .081) between users and nonusers for potential applications, but overall, respondents neither agreed nor disagreed with the potential uses of PROs (3.6 ± 0.5) and the concept that they were problematic (2.9 ± 0.5). However, they did agree that PROs had benefits (4.0 ± 0.6) for clinical practice.

A main effect indicated that the route to certification influenced the respondents' perceptions of perceived benefits (χ2 = 6.787, df = 2, P = .034) and problems (χ2 = 7.390, df = 2, P = .025), but no pairwise differences were evident. Specifically, those who completed an internship (3.0 ± 0.5), accredited bachelor's degree program (2.9 ± 0.5), or accredited master's degree program (3.1 ± 0.3) neither agreed nor disagreed that potential problems were associated with PROs. Those who completed an internship (4.0 ± 0.7), accredited bachelor's degree program (4.1 ± 0.6), or accredited master's degree program (3.8 ± 0.5) agreed on the potential benefits of PROs.

DISCUSSION

The primary objective of our study was to explore how secondary school ATs using PROs perceived the applications and benefits of and problems with these measures in contrast to ATs who did not use them. Expanding the application of PROs in athletic training is an important component of patient-centered care and has been described as crucial to evidence-based practice.16  Previous authors have investigated the use of PROs in athletic training, but our study is the first we know of to focus solely on the secondary school setting. Another difference from previous studies is that our survey presented questions on the uses and benefits of and problems with these measures to all respondents instead of deselecting those who did not currently employ PROs. Using this approach provided a clearer understanding of how all ATs in the secondary school setting perceived PROs and what strategies might be appropriate to increase their application.

Use of PROs in the Secondary School Setting

Most (85%) of the ATs who responded to our survey did not use PROs in clinical practice. Thus, only 15% of secondary school ATs were using patient-oriented evidence to help guide their clinical decision making. This result is lower than the 26% of ATs in the 2014 investigation10  who consistently collected these types of outcomes in their clinical practice in various settings. However, our findings are consistent with theirs in the low number of ATs in the secondary school setting (20%) who used PROs in their clinical practice.10  In a more recent study23  of PROs commonly used in all athletic training settings, the researchers concluded that a mere 20.9% of respondents used PROs in their practice. This information, plus data from our study, clearly indicates that the athletic training profession is struggling to consistently use patient-oriented evidence and that secondary school ATs may use them the least.

Athletic training is not the only health care profession to struggle with the widespread use of PROs. The field of psychology is based on patient interaction and seems to be a logical setting for the use of PROs.18  Yet in 2 investigations24,25  of the prevalence of PROs, fewer than half (37%) of practicing psychologists reported using these measures in their practices. Authors of the more recent study25  linked the use of patient outcomes to the requirements of their institutional setting and the source of income being from a managed care organization. This result is similar to the result of Snyder Valier et al,10  who found that 46% of ATs using PROs did so as the result of an employer mandate. The physical therapy profession has long recognized the need to collect patient-reported data and has included this as a goal in a number of policy statements and guidelines for clinical practice.19,26,27  In a 2011 investigation,19  72% of private-practice physical therapists and 97% of those in the inpatient setting used PROs. Despite the high percentage of use, further examination of the results showed that patient-reported data were not always influential in clinical decisions, which may reflect difficulty interpreting the results of those measures.19 

The low level of PRO use in our study may be attributed to the demographic makeup of our respondents. A large number of the secondary school ATs completing our survey received their BOC certification without completing an accredited educational curriculum. The internship method of becoming eligible to sit for the BOC certification examination, which was eliminated in 2004, had no formal requirements for educational competencies. The other characteristic of our sample that may explain their limited use of PROs was their years of experience. A majority of our participants (85.7%) had more than 11 years of experience working as an AT. The “Athletic Training Educational Competencies”28  had no requirements regarding evidence-based practice or patient-centered care until the 5th edition mandated this content in 2012. Our respondents described a low level of familiarity with disablement models and how they relate to patient perceptions of injury, which may reflect limited education on these topics. In addition, one-third of those completing the familiarity questions reported no formal exposure to clinician- or patient-rated outcomes. If our sample is representative of the larger population, a significant number of BOC-certified ATs in the secondary school setting have likely not been formally exposed to PROs, how they relate to evidence-based practice, or how to implement these measures in their professional education programs. With practicing clinicians having received no formal exposure to PROs in educational preparation, it would seem logical to mandate continuing education to train them in this essential behavior of all health care professionals. Our findings suggest that a majority of secondary school ATs relied primarily on literature sources to become familiar with different types of patient outcomes and that this mechanism was largely failing.

Perceived Uses of PROs

In an effort to understand how secondary school ATs perceived the use of PROs in patient care, our survey presented statements illustrating how these measures could be used in clinical practice. Most respondents recognized the role that PROs can play in determining the effectiveness of a treatment, documenting patient progress or outcomes by individual clinicians, demonstrating effectiveness to administration, and communicating with other health care providers. These results are similar to those of Jette et al,21  whose 2009 study of physical therapists showed that the top uses of PROs were quality assurance, communication with other health care providers, documentation of patient progress and outcomes, and examination of practice effectiveness. The PROs provide objective outcome data that are a crucial component of measuring the quality of care provided and determining improvements that can be made. A 2010 clinical guide17  to implementation of PROs in all health care settings suggested the same commonly accepted uses for PROs that were endorsed in our study. Assessing the effects of health care interventions on patients, assisting with resource allocation, evaluating the implications of changes to service, and providing feedback to assist with clinical governance have all been suggested uses of PROs17  and received similar endorsements from our respondents. The similarities between the findings of previous investigations in athletic training and physical therapy and the recommendations put forth in clinical guides for PRO implementation lead us to believe that secondary school ATs generally understand how PROs can be used in clinical practice. The one concerning result of our study is that in each of our top 5 reported uses, the response neither agree nor disagree received at least a 25% response rate. Therefore, as many as one-quarter of secondary school ATs may not understand the uses of PROs, so further education efforts are necessary.

Perceived Benefits of PROs

All of the potential benefits of using PROs in clinical practice presented in our survey were endorsed by at least 71% of those responding to the question. Athletic trainers who reported using PROs in clinical practice gave significantly stronger endorsements to the benefits statements. Improving communication between the patient and clinician and helping to direct the plan of care were the most frequently endorsed benefits of using PROs by our sample of secondary school ATs. This mirrors the results of the similar investigations by Snyder Valier et al10  in athletic training and Jette et al21  in physical therapy. Increasing the quality of communication in the patient-clinician relationship has numerous beneficial effects. Clinicians who engage patients through PROs gain a more accurate understanding of how their injury or illness affects their daily lives.19,21,24,25  This provides valuable information that directs clinical decision making, involves the patient in that process, and helps identify the need for alternative interventions.19  As patients become active participants in their own care decisions, their motivation increases and overall outcomes improve.19,25  Once clinicians fully understand the effect of an injury on a patient's daily life, they begin to appreciate the disablement the patient is experiencing. Clinicians can begin to move beyond accustomed clinical outcomes and create treatment goals that directly affect the patient's activities of daily living and quality of life. Similarly, as patients begin to understand their condition and how the clinical outcomes are important to achieving their personal goals, motivation and compliance with treatment interventions increase.

Perceived Barriers to Using PROs

Our secondary school ATs clearly recognized how PROs can be used in clinical practice and the benefits they provide to patient outcomes. Despite these perceptions, only a very small percentage of our respondents actually used these measures in their clinical practice. Multiple investigations7,19,25,29,30  into the potential barriers to using PROs in clinical practice across the health care professions have yielded similar results. Resistance to change is a common reason reported by clinicians for not using PROs.19,25,29,31  Embedded in this concern are the factors of being uneducated on the use of PROs, apprehension about one's work being open to criticism, and fear of added work.7,19,31  Many clinicians reported that the information gained from PROs could identify shortcomings in their own ability to practice7,30 ; however, self-reflective practice that incorporates the principles of quality improvement is an essential behavior of all health care professionals. Clinicians who are unfamiliar with the concept of collecting and using patient data were most likely to be skeptical of PROs' value in clinical practice.19,29  Fear of adding work to an already busy day is a barrier frequently reported by clinicians who have concerns regarding the administration and scoring of PROs.7,25  The perception that PROs burden clinicians' time is also evident in concerns that the information from the patient may not be usable in decision making or that the benefits are theoretical and unsubstantiated.2931  Concern over burdening patients is another barrier reported in the literature.19,29,31  If patients fail to see the value in completing the measurements or no change occurs as a result of the practice, then the patient will perceive a very real burden.31 

Athletic trainers responding to our survey had similar concerns as other health care professionals with regard to the time burden PROs may pose. The most commonly endorsed problems with PROs were that they require too much time to score or analyze and too much time for the patient to complete. These results directly mirror the results of 2 previous investigations10,23  across all athletic training practice settings. These barriers were not corroborated by respondents who were actually using PROs but were perceived by those not using them in clinical practice. Concerns about the patient burden, not helping to direct the plan of care, and not providing usable information were not evident in our study. One interesting outcome of our investigation was the frequency with which respondents indicated they did not have an opinion about the potential problems of PROs; they indicated neither agreement nor disagreement with regard to requiring more effort than they are worth (51.6%), being confusing to patients (50.5%), not being culturally sensitive (48.8%), making patients anxious (48.4%), and providing information that is too subjective to use (48.2%), suggesting they did not know whether these items were potential problems in implementation. This may result from a lack of understanding about PROs or the large number of ATs not using them in secondary school clinical practice.

More clarity on how secondary school ATs perceive PROs may be found in the question we asked after respondents indicated whether or not they used these measures in clinical practice. Those who were not using PROs were asked to indicate the reasons. Concerns over an adequate support structure, the time needed to analyze and score, and the subjective nature of the information were the most frequent reasons chosen by our respondents. These reasons for not using PROs were consistent with the most recent investigation23  of PRO use across all athletic training practice settings. Respondents who currently used PROs in clinical practice were asked to indicate the criteria they applied to select those measures. Quick completion and being easy for patients to understand were the most commonly cited reasons for selecting a PRO measure. Again, our results are similar to those in previous athletic training PRO research.23  It is clear that secondary school ATs had concerns on how the use of PROs would affect the time they had available for patient care. A high ratio of potential patients to clinicians (Table 1) corroborates the perceived time demands that secondary school ATs face and the importance of not adding to those demands. The ability of their patient population to understand the individual measures was also a major concern of secondary school ATs and was consistent with the concerns of ATs in other practice settings. Simple educational efforts may ease these concerns, given that many PROs are geared to the pediatric population and most others are written at an eighth-grade level.

Application of Results

Strategies to address the perceptions and knowledge level of and barriers to using PROs are needed for ATs. Because most of our respondents were forced to rely on self-directed learning and continuing education opportunities, identifying the gap between perceived and actual knowledge may be an important first step. Previous authors32  who studied the continuing education of ATs revealed that the gap between perceived knowledge and actual knowledge may predict the likelihood of pursuing continuing education. In addition, feedback received from external sources altered the participants' information-seeking behavior.32  Evidence indicates that such a knowledge gap exists with regard to using PROs in clinical practice. Research33  into PRO use among physiotherapists in New Zealand demonstrated concerns similar to our findings. However, the concerns expressed about the uses of PROs were the result of the clinician's level of knowledge and understanding rather than what the measures actually showed.33  Successful strategies to increase understanding of and address barriers to using PROs are available. In Australia, a 6-month intervention program focusing on education, improving perceptions of patient outcomes, and providing professional support increased the use of PROs from 30% to 66%.34 

Athletic trainers in the secondary school setting experience a unique set of demands that must be addressed by any intervention program. In addition to closing the perceived to actual knowledge gap regarding PROs, interventions to enhance the ability to quickly score and analyze these measures may address the time concerns that were evident in our population. Infrastructure that allows for quick administration and scoring may ease the burden of using these measures and the apprehension that exists. We hope that our results can help to shape continuing education interventions focused on increasing PRO use by secondary school ATs.

Requiring a support structure that is not available has been reported in our survey and previously published investigations in athletic training as a problem with PROs and a reason for not using them. Although what type of support structure ATs perceive they need to implement PROs is unclear, the consistency of this response across multiple studies indicates an area that should be examined. Many secondary school ATs may not feel they have the necessary support from administration to begin using PROs in clinical practice. Capital investment in infrastructure through advanced medical documentation and health care informatics software to administer PROs, specific policies on where the data will be housed and who has access to it, and allowing increased time needed for patient care have all been suggested administrative strategies for implementing PROs.35  Further investigations into specific support needs in the secondary school setting may illuminate strategies to help ATs implement these measures.

Limitations and Future Research

Our response rate (10.8%) was far below that of other PRO investigations in athletic training. When reviewing the recent literature35,36  in the secondary school setting, we identified variable response rates and variable characteristics of the populations sampled. This makes it difficult to determine the generalizability of our sample. However, we do know from membership statistics among ATs that those in the secondary school setting tend to be somewhat older and have more clinical experience, similar to the characteristics of our sample.37  Although not ideal, this may be a result of our focus on the secondary school setting. With the previously stated concerns about the amount of time available, completing a lengthy survey may not be a luxury that these ATs can afford. Additional research focused on specific athletic training settings may help to identify whether these perceptions are exclusive to secondary school ATs.

Further instrument analysis may identify redundant or unrelated items that could be removed and lead to a shorter, more refined questionnaire. This would address the time burden on ATs participating in future investigations and potentially capture a larger sample from which to draw conclusions.

CONCLUSIONS

A majority of secondary school ATs surveyed recognized the uses of PROs and the beneficial role they may play in helping to direct the plan of care and improve communication between the patient and clinician. Respondents also recognized that PROs can help to assess how their interventions benefit patients, assist with resource allocation, evaluate the effect of changes to service, and provide feedback to assist with clinical governance. These types of quality improvement efforts are impossible to accomplish without objective data such as that provided by PROs. Despite these perceptions, a very small number of secondary school ATs actually used PROs in clinical practice. Concerns over time demands and providing useful data were the most commonly reported problems with and reasons for not using PROs in clinical practice. Large patient-to-practitioner ratios, lack of staff, and hiring structures within the secondary school athletic training setting may contribute to the lack of PRO use in this setting. As the athletic training profession moves toward being a more evidence-based and patient-centered profession, consistent use of PROs to drive patient care decisions will be an important practice. Secondary school ATs may experience barriers to PRO use that are different from those in other settings. Understanding the differences that exist in the secondary school setting will be important for creating strategies to increase PRO use.

REFERENCES

1
Institute of Medicine
.
Health Professions Education: A Bridge to Quality
.
Washington, DC
:
Institute of Medicine
;
2003
.
2
The professional degree
.
Commission on Accreditation of Athletic Training Education Web site
.
http://caate.net/the-professional-degree/. Published 2015. Accessed March 20, 2018
.
3
Maintenance requirements
.
Board of Certification Web site
. .
4
Hankemeier
DA,
Walter
JM,
McCarty
CW,
et al.
Use of evidence-based practice among athletic training educators, clinicians, and students, part 1: perceived importance, knowledge, and confidence
.
J Athl Train
.
2013
;
48
(
3
):
394
404
.
5
Steves
R,
Hootman
JM.
Evidence-based medicine: what is it and how does it apply to athletic training?
J Athl Train
.
2004
;
39
(
1
):
83
87
.
6
Van Lunen
BL,
Hankemeier
DA,
Welch
CE.
Evidence-Guided Practice: A Framework for Clinical Decision Making in Athletic Training
.
Thorofare, NJ
:
Slack
;
2015
.
7
Antunes
B,
Harding
R,
Higginson
IJ;
EUROIMPACT. Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers
.
Palliat Med
.
2014
;
28
(
2
):
158
175
.
8
Arnold
BL,
Wright
CJ,
Ross
SE.
Functional ankle instability and health-related quality of life
.
J Athl Train
.
2011
;
46
(
6
):
634
641
.
9
Vela
LI,
Denegar
C.
Transient disablement in the physically active with musculoskeletal injuries, part I: a descriptive model
.
J Athl Train
.
2010
;
45
(
6
):
615
629
.
10
Snyder Valier
AR,
Jennings
AL,
Parsons
JT,
Vela
LI.
Benefits of and barriers to using patient-rated outcome measures in athletic training
.
J Athl Train
.
2014
;
49
(
5
):
674
683
.
11
Albohm
MJ,
Wilkerson
GB.
An outcomes assessment of care provided by certified athletic trainers
.
J Rehabil Outcomes Meas
.
1999
;
3
(
3
):
51
56
.
12
Michener
LA.
Patient- and clinician-rated outcome measures for clinical decision making in rehabilitation
.
J Sport Rehabil
.
2011
;
20
(
1
):
37
45
.
13
Valovich McLeod TC, Snyder AR, Parsons JT, Bay RC, Michener LA, Sauers EL
.
Using disablement models and clinical outcomes assessment to enable evidence-based athletic training practice, part II: clinical outcomes assessment
.
J Athl Train
.
2008
;
43
(
4
):
437
445
.
14
Members statistics
.
National Athletic Trainers' Association (NATA) Web site
. .
15
Pryor
RR,
Casa
DJ,
Vandermark
LW,
et al.
Athletic training services in public secondary schools: a benchmark study
.
J Athl Train
.
2015
;
50
(
2
):
156
162
.
16
Snyder
AR,
Parsons
JT,
McLeod
TCV,
Bay
RC,
Michener
LA,
Sauers
EL.
Using disablement models and clinical outcomes assessment to enable evidence-based athletic training practice, part I: disablement models
.
J Athl Train
.
2008
;
43
(
4
):
428
436
.
17
Dawson
J,
Doll
H,
Fitzpatrick
R,
Jenkinson
C,
Carr
AJ.
The routine use of patient reported outcome measures in healthcare settings
.
BMJ
.
2010
;
340
:
c186
.
18
Hatfield
DR,
Ogles
BM.
The influence of outcome measures in assessing client change and treatment decisions
.
J Clin Psychol
.
2006
;
62
(
3
):
325
337
.
19
Swinkels
RAHM,
van Peppen
RPS,
Wittink
H,
Custers
JWH,
Beurskens
AJHM.
Current use and barriers and facilitators for implementation of standardised measures in physical therapy in the Netherlands
.
BMC Musculoskelet Disord
.
2011
;
12
:
106
.
20
Hoch
JM,
Druvenga
B,
Ferguson
BA,
Houston
MN,
Hoch
MC.
Patient-reported outcomes in male and female collegiate soccer players during an athletic season
.
J Athl Train
.
2015
;
50
(
9
):
930
936
.
21
Jette
DU,
Halbert
J,
Iverson
C,
Miceli
E,
Shah
P.
Use of standardized outcome measures in physical therapist practice: perceptions and applications
.
Phys Ther
.
2009
;
89
(
2
):
125
135
.
22
Buhi
ER,
Goodson
P,
Neilands
TB.
Out of sight, not out of mind: strategies for handling missing data
.
Am J Health Behav
.
2008
;
32
(
1
):
83
92
.
23
Harrington
KM,
Valier
AR,
Cameron
KL,
Lam
KC.
Commonly used patient-reported outcome measures in athletic training
.
J Athl Train
.
2016
;
51
(
suppl 6
):
S-228
.
24
Hatfield
DR,
Ogles
BM.
The use of outcome measures by psychologists in clinical practice
.
Prof Psychol Res Pract
.
2004
;
35
(
5
):
485
491
.
25
Hatfield
DR,
Ogles
BM.
Why some clinicians use outcome measures and others do not
.
Adm Policy Ment Health
.
2007
;
34
(
3
):
283
291
.
26
Guide to physical therapist practice
.
American Physical Therapy Association Web site
.
http://guidetopractice.apta.org/. Published 2011. Accessed October 15, 2015
.
27
Policy statement: standards of physical therapist practice
.
World Confederation for Physical Therapy Web site
.
https://www.wcpt.org/policy/ps-standards. Updated April 25, 2017. Accessed March 20, 2018
.
28
Athletic training education competencies, 5th ed
.
National Athletic Trainers' Association Web site
. .
29
Boyce
MB,
Browne
JP,
Greenhalgh
J.
The experiences of professionals with using information from patient-reported outcome measures to improve the quality of healthcare: a systematic review of qualitative research
.
BMJ Qual Saf
.
2014
;
23
(
6
):
508
518
.
30
Duncan
EA,
Murray
J.
The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review
.
BMC Health Serv Res
.
2012
;
12
(
1
):
96
.
31
Locklear
T,
Miriovsky
BJ,
Willig
JH,
et al.
Strategies for overcoming barriers to the implementation of patient-reported outcome measures
.
National Institutes of Health Collaboratory Web site
. .
32
Eberman
LE,
Tripp
BL.
Effect of performance feedback on perceived knowledge and likelihood to pursue continuing education
.
Athl Train Educ J
.
2011
;
6
(
2
):
69
75
.
33
Copeland
JM,
Taylor
WJ,
Dean
SG.
Factors influencing the use of outcome measures for patients with low back pain: a survey of New Zealand physical therapists
.
Phys Ther
.
2008
;
88
(
12
):
1492
1505
.
34
Abrams
D,
Davidson
M,
Harrick
J,
Harcourt
P,
Zylinski
M,
Clancy
J.
Monitoring the change: current trends in outcome measure usage in physiotherapy
.
Man Ther
.
2006
;
11
(
1
):
46
53
.
35
Winkelmann
ZK,
Eberman
LE.
Characteristics of secondary school athletic trainers: salary, job satisfaction, and perceived percentage of daily practice
.
Athl Train Sports Health Care
.
2017
;
9
(
3
):
124
132
.
36
Valovich McLeod
TC,
Huxel Bliven
KC,
Lam
KC,
Curtis Bay
R,
Snyder Valier
AR,
Parsons
JT.
The National Sports Safety in Secondary Schools Benchmark (N4SB) Study: defining athletic training practice characteristics
.
J Athl Train
.
2013
;
48
(
4
):
483
492
.
37
Kahanov
L,
Eberman
LE.
Age, sex, and setting factors and labor force in athletic training
.
J Athl Train
.
2011
;
46
(
4
):
424
430
.