A National Study on the Relationship Between Programmatic Factors and Athletic Training Education Board of Certification Pass Rates

• Programmatic factors which the student experiences during their education have a relationship with programmatic outcomes

• Programs seeking to increase their BOC pass rates should evaluate their clinical immersive experiences, faculty composition and faculty development for areas for improvement.

• The positive relationship between admissions selectivity and BOC pass rates makes the evaluation of programmatic factors increasingly important, especially for programs with a low number of applicants.

Athletic training programs are professional degree programs aimed at preparing students to practice as athletic trainers (ATs) postgraduation. Athletic trainers are allied health professionals recognized by the American Medical Association that help prevent, diagnose, treat, and rehabilitate chronic or acute injuries.¹  Athletic trainers have an important role in promoting safety, preventing injury and illness, assisting in recovery from injury, and supporting good health in patients across their lifespans.² 

The athletic training professional education and credentialing are regulated by the Commission on Accreditation of Athletic Training Education (CAATE), and the Board of Certification (BOC). Credentialing for an AT consists of graduation from a CAATE-accredited program and successful completion of the BOC exam. The CAATE is the regulatory agency for athletic training education whose mission is to define, assess, and improve athletic training education.³  The BOC is responsible for providing credentialing programs that support ATs.

Currently, the athletic training profession is undergoing a significant change to the education structure. The current minimum level of education required to qualify to take the BOC exam moves from the bachelor to the professional master's (PM) level as of 2022. In the 2018–2019 program year, there were 98 PM programs. Additionally, 298 bachelor's programs will need to move to PM or withdraw their accreditation.³ 

In 2013, the CAATE enacted an accreditation standard requiring all accredited programs to maintain a 3-year aggregate first-time BOC pass rate of 70%. The first-time pass rate is the percentage of test takers at an institution for a testing year that successfully passed the BOC exam on the first attempt. Programs that are not compliant are placed on probation; if a program fails to comply with the 70% pass rate for a second year but has a pass rate above 50%, they remain on probation. If the pass rate is below 50% in the second year or they do not achieve above a 70% pass rate in the second year of probation, the program has its accreditation withdrawn.⁴  Additionally, athletic training programs are required (Standard 25) to collect student outcome measures annually and post the information for 3 prior years on their programmatic Webpage.⁴  The increase in required transparency and accountability by the CAATE has placed a greater emphasis on measurable student outcomes, such as a program's first-time BOC examination pass rate. In the 2017–2019 reporting years, 31% (n = 66) of bachelor's degree programs were noncompliant with Standard 6 compared with 16% (n = 22) of PM programs.⁵ 

The BOC exam is a comprehensive computer-based test aimed to determine entry-level competence in the athletic training field. Currently, 49 states require completing the BOC exam to obtain mandatory state licensure to practice as an AT in those states. The BOC first-time pass rate is a significant marker of a program's success, and determining predictors of BOC pass rates has been the focus of multiple studies within athletic training.^6–13  However, prior researchers have focused mainly on undergraduate education and student factors. Additionally, with the recent changes to the educational structure and the 7th edition of Practice Analysis¹⁴ , prior research on this topic is outdated. The 7th edition of Practice Analysis, which determines the content areas and weights for the BOC exam was released in April 2017.¹⁴  Therefore, authors of research before this date did not use the current format of the BOC exam that is being administered today.

During the 2016–2019 reporting years, the average first-time BOC examination pass rate for PM programs has also steadily declined from 97% to 93% in the 2019 graduation year, and the average graduation rate for all PM programs has dropped from 83% to 74% for the same period.³  With the recent decline in BOC pass rates, decline in graduation rates, and the enacting of a minimum 70% first-time, 3-year aggregate pass rate, it is crucial to identify factors that may increase student success in PM athletic training programs.

Board of Certification pass rates are typically a marker of programmatic success, and students often use the marker in their decision to attend. While attendance rates in the health care field are increasing, the overall number of applications in PM athletic training programs are decreasing.¹⁵  The maintaining of good accreditation status and high BOC pass rates are 2 areas of importance to perspective graduate students.¹⁵  Therefore, increasing the BOC pass rate may improve a program's sustainability due to the increased competition for students and by maintaining good accreditation status. Additionally, the decrease in number of applications received will force institutions to be less selective in their admissions. The less selective admissions process places much greater importance on programs to modify the students' environment to increase their potential for a successful outcome.

Researchers in academic institutions have used organizational theory and its application to identify students' behaviors and their impacts on student outcomes. Berger¹⁶  suggests that different dimensions of an organization's behavior make up a university's environment. For example, the governance structure, mission, and allocation of funding to resources within the entity all influence its environment. The different levels of different dimensions of organizational behaviors create varying institutional environments.

A simplified model used in the literature to conceptualize the interactions between student demographic variables, organizational behavior, and student outcomes is Astin's¹⁷  input-environment-output (I-E-O) model.

In this model, Astin¹⁷  proposes the 3 categories to measure student learning from attending an institution. Inputs are the student's demographic factors such as race and academic ability, which may impact how an individual performs within different environments. The environment is the institutional control type: public or private, and specific programmatic factors: faculty-to-student ratios, faculty degrees, faculty development, and clinical immersive experience that the student experiences while attending the program. Lastly, the outputs are the knowledge and skills that the student has gained from attendance, which are measured by the percentage of students who successfully pass the BOC exam on the first attempt. Use of Astin's¹⁷ I-E-O model allows for the conceptualization and identification of the different aspects of organizational behavior that may impact student success (Figure 1).

Researchers in athletic training education have primarily focused on student factors and undergraduate programs.^8,10,12,13  Prior researchers in athletic training, allied health, and medicine suggest that grade point average (GPA) is typically a predictor of increased odds of successful completion of the BOC exam.^{6,7,10,18–20}  Researchers also suggest that student factors such as race and ethnicity may be predictive.^19,21  Authors of literature in athletic training, allied health, and medical fields suggest a relationship between programmatic factors and exam pass rates in professional programs.^12,19,22  Clinical rotations are a large part of athletic training education; while the type of rotation has been found nonpredictive, the number of rotations has been found predictive in previous research.^11,13  The clinical immersive experience is a new addition to the CAATE 2020 standards and therefore is important to consider.

Additionally, factors relating to number of faculty, spending on faculty, and highest degrees of faculty members have been found predictive in athletic training, allied health, and medical education.^12,19  Therefore, a relationship may exist between programmatic factors and PM program BOC pass rates. The limited research in PM programmatic factors and their relationship to BOC exam pass rates leaves programs with little information on which to base decisions to increase their BOC pass rates.

In this study, we aimed to identify whether relationships exist between programmatic factors and the first-time BOC pass rate for PM athletic training students while controlling for student and institutional factors. Programmatic factors include faculty composition, clinical rotations, program length, and resource allocation.

This study was a nonexperimental retrospective analysis of data retrieved from the CAATE. The CAATE collects data annually from accredited programs as part of an accreditation requirement.

This study included programmatic level data from all 77 PM CAATE-accredited programs that graduated a cohort in 2019. Figure 2 further illustrates the study enrollment.

Before submitting the data request, a Not for Human Subjects Research application was submitted to an Institutional Review Board and was deemed “Not Human Subjects Research” and was therefore beyond the Institutional Review Board's purview. An external data use request was approved by the CAATE Research Review Subcommittee for the use of the 2018–2019 reporting year program data.

The dependent variable was a continuous variable of 1-year BOC first-time pass rate percentage (1YR BOC). The independent variables were program-level data representing student, programmatic, and institutional factors. Student factors were the cohort race or ethnicity and academic ability. The programmatic factors were faculty number, professional development expenses, faculty highest degree, and clinical immersive hours. The institutional factor was institutional control type.

The student factors consisted of the race or ethnicity of students within each cohort and were reported as the program cohort's diversity. The variable of cohort race or ethnicity was a dichotomous variable of low and high diversity based on the population mean reported in the 2018–2019 CAATE Analytic Report of 0.212 percentage of minority students.³  High diversity represented programs with a diversity ratio equal to or above the population mean ratio of program diversity.

In this study, the admissions selectivity was a continuous variable and represented the academic ability of the programs' cohorts. Typically, more selective programs had lower admittance rates, with admission preference given to higher GPA students.^23,24  We calculated program selectivity by dividing the number of students who applied for admission by the number of admitted students. Two institutions were missing admissions data; thus, we used the population mean for admissions selectivity of 1.45 to replace the missing value for the 2 cases.³ 

Programmatic factors included 2 student-faculty ratio measures that indicated the student per 1 core faculty number and student per 1 laboratory faculty member and were reported as continuous data. In this study, the number of students per laboratory faculty ratio was included to measure the student-to-faculty ratio across different course formats. Laboratory faculty instruct the laboratory skills classes and consisted of core, alternate full-time faculty within the university and adjunct faculty. Laboratory classes tend to have lower student-to-faculty ratios than lecture courses, which in large cohorts had very high student-to-faculty numbers.

Professional development expense is another program factor reported as a continuous variable (natural logged), adjusting for the variable skewness of the data. The use of expenses on professional development variables allowed for conceptualizing the programs' resource allocation related to investment in their PM athletic training programs. The next program variable we explored was core faculty highest degrees, which required the transformation into a percentage of faculty with an academic or clinical doctorate. The CAATE does not stipulate a difference between required doctoral degrees; therefore, academic (Doctor of Philosophy [PhD] and Doctor of Education [EdD]) and clinical doctorates (Doctor of Athletic Training [DAT] and Doctor of Science [DSc])⁴  were appropriate to include. Lastly, clinical immersive experience hours are the number of hours each program required for their students per week. In terms of the institutional factors, we included institutional type measured by whether an institution is public or private.

After data cleaning and recoding, we conducted descriptive statistics and frequency distribution analyses (Tables 1 and 2). The second step was a Pearson correlation analysis on all continuous programmatic variables (Table 3). For the correlation analysis, a coefficient of 0.00 to 0.25 represented little or no relationship, 0.25 to 0.50 indicated a fair relationship, 0.50 to 0.75 indicated a moderate to good relationship, and 0.76 to 0.99 indicated a very strong relationship.²¹ 

We conducted a variance inflation factor (VIF) test for a multicollinearity analysis to assess for any variables that were highly correlated and may affect the analysis (Table 4). Lastly, we ran a multiple regression analysis to identify what student, programmatic, and institutional factors are significantly related to 1YR BOC from the deidentified CAATE accreditation data (Table 5). All significance levels were set at P ≤ .05 a priori.

All 77 PM programs with a 1YR BOC for the 2018–2019 academic year were included for the multiple regression analysis. A power analysis run before the final selection of the number of variables with the G*Power 3.1.9.4 software determined the maximum number of independent variables to include in the sample size model.²⁵  The G*Power results indicated that, with 8 predictors included in the model, a sample size of 74 would be the minimum required sample size to detect a power of 0.95 and a large effect size of 0.3. Therefore, all 8 predictors were included in the model.

Of the 77 programs included, 51% (n = 39) were private institutions, and 49% (n = 38) were public institutions. The mean 1YR BOC for all institutions was 79.48 (SD = 21.9). More than half of the programs, 57% (n = 44), were high diversity, with a percentage of minority students equal to or over 21.2%. Admissions selectivity ranged from 0 to 3.5, with a mean ratio of 1 student admitted per 1.44 (SD = 0.53) applications completed.

Programmatic factors of clinical immersion hours per week ranged from 0 to 60 hours per week with a mean of 30.9 (SD = 14.7). Institutions reported a mean of 3.6 (SD = 1.9) core faculty per program. On average, 83% (SD = 27) of the core faculty highest degree earned was PhD, EdD, and clinical doctorate. The number of students per core faculty member averaged 7.4 (SD = 4.1). The number of students per lab faculty member averaged 8.6 (SD = 3.9) students per faculty member. The mean program expense on professional development ranged from 0 to $25 000 per year with a mean yearly expense of $4861 (SD = $4047). All descriptive statistics for categorical variables are detailed in Table 1 and continuous variables in Table 2.

The correlation analysis demonstrated no concerns for the model, with no independent variables having a strong correlation. Students per core faculty member had a statistically significant moderate positive correlation with students per lab faculty member (r² = 0.546, P < .01). Total professional development costs had a statistically significant very small positive correlation with the students per lab faculty member (r² = 0.233, P < .05; Table 3). Additionally, the VIF test for the multiple regression analysis resulted in VIF values between 1.08 and 1.58, under the threshold of 10.²⁶  Therefore, this model's predictors are not highly correlated, and no multicollinearity issues exist in this model (Table 4).

The multiple regression model (Table 5) was significant in predicting 1YR BOC, and 32% of the variance in pass rates was explained by this model (adjusted r² = 0.316, f = 5.399, P < .001). This model identified 4 significant predictors of 1YR BOC pass rates. Admission selectivity resulted in a 0.23% increase 1YR BOC pass rate for every 1 point increase in selectivity (β = .227, P = .025). Clinical immersive experience hours per week resulted in a 0.30% increase in 1YR BOC pass rates for every 1 clinical immersive hour increase (β = .298, P = .003). Spending on professional development resulted in a 0.263% increase in the 1YR BOC pass rate for every 1 unit increase in the natural log of professional development spending (β = .263, P = .015). Lastly, for every 1% increase in the percentage of doctorate faculty, a 0.24% increase in 1YR BOC pass rate (β = .238, P = .025) occurred. Further results are detailed in Table 5.

The significant predictive result of percentage of doctoral faculty is like results in a study by Williams and Hatfield,¹²  with a sample of 54 athletic training programs, which found that programs with higher numbers of PhD and EdD faculty had higher first-time BOC pass rates. Additionally, Mohr et al,²⁷  who investigated programmatic factors and National Physical Therapy Examination (NPTE) program pass rates in 132 accredited PT programs in 1999, found a significant positive relationship between the number of PhD or EdD faculty and higher pass rates.

However, none of the authors of these previous studies looked at clinical doctorates beyond an EdD. Since the athletic training profession is moving to the PM level, clinical DATs have gained popularity for practitioners looking to advance their skills. The inclusion of clinical doctorates as a terminal degree in this study was to capture faculty members with advanced practice skills and its relationship with programmatic outcomes. Cook et al¹⁹  found that the number of Doctor of Physical Therapy (DPT) faculty with an American Physical Therapy Association credential did have a relationship with increased NPTE pass rates, and scholarly activity did not have a significant relationship with NPTE pass rates. American Physical Therapy Association-credentialed faculty have done coursework and obtained credentials beyond the DPT licensure's entry-level degree requirements.²⁸ 

In both studies by Dickson et al²⁴  and Cook et al,¹⁹  the number of PhD and EdD faculty were not significant predictors of NPTE exam pass rates. However, the minimal degree requirement to sit for the NPTE exam is at the clinical doctorate level. Therefore, the faculty of DPT programs would likely already possess a clinical doctorate, whereas in athletic training, the clinical doctorate is a relatively new degree and indicates advanced training in athletic training. The degree-level difference between athletic training and physical therapy (PT) programs may explain the difference between ATs' and PTs' results. Therefore, the number of academic and clinical doctorate faculty may have a relationship with higher pass rates due to the faculty members with advanced skills instructing the students.

With the addition of new education standards focusing on practicing in interprofessional collaborative teams (Standard 61), use of evidence to inform practice (Standards 62), the use of quality improvement to enhance patient care (Standard 63) and apply the practices of health informatics to the administration and delivery of patient care (Standard 64),⁴  faculty may need increased support or education in these areas to properly disseminate knowledge and skills to students.²⁹  The findings from this study suggest that the development and degree level of the faculty member has a positive relationship with BOC pass rates, whereas the number of faculty per students did not. Faculty with a DAT, which is a clinical practice degree, may have increased knowledge and skills in these areas and may help programs meet the educational demands of the new required curricular content. However, future research is needed to assess the relationship between clinical practice degrees and their ability to address and meet the demands of new curricular content to substantiate these claims.

With the transition to PM level education, programs will need to offer more research components in their curriculums to fulfill the requirement of the master's level degree and will need additional support in developing the research component, supporting faculty and student scholarly activity.³⁰  Therefore, faculty with higher level research degrees and more advanced research skills will be able to assist students in the fulfillment of master's level degree requirements. Therefore, programs that are transitioning to the PM level from a bachelor's degree may benefit from both academic doctoral (PhD or EdD) and clinical doctoral (DAT) faculty to meet the demands of both the research component and the new curricular content.

Another programmatic area related to faculty characteristics investigated in this study was the amount of spending on faculty development. Our findings suggest a relationship between the two exists, with program pass rates increasing as professional development spending increased. Other disciplines have documented the relationship between faculty and advanced clinical training¹⁹  and increased spending on faculty.³¹ 

Our findings of a significant positive relationship between expenditures on professional development support the efforts by the CAATE to require the maintenance and development of contemporary expertise for faculty. Standard 37 mandates that programs document professional development activities for faculty. Programs are required to document faculty development activity category, date, and rationale for how the activity relates to their area of expertise.⁴  Therefore, professional development expenses focused on increasing teaching faculty members in their course area of expertise may be most beneficial.

Additionally, many standards now need to be addressed both clinically and didactically in which faculty and preceptors may have no formal education. Standards relating to working in interdisciplinary teams, using evidence to inform practice, quality improvement, and the use of health care informatics have all been added to the 2020 core competencies effective July 1, 2020.⁴  These are all areas in which ATs certified before enacting the new standards may not have received formal training in the new content area. Therefore, targeting professional development spending in areas in which faculty teach assists in the required maintenance and development of contemporary expertise, but it may also bridge the knowledge gap between faculty and the new core competencies.

While previous researchers have found the type of rotation and hours may not be significant,^11,13  the number of rotations has resulted in a significant relationship with passing the BOC.¹¹  However, the CAATE requires students to complete 4 clinical rotations over 2 years. Therefore, most of a student's clinical rotations are standardized and lack variation between programs. At the time of data collection for this study, clinical immersive experiences were not required. However, immersive clinical experiences have become required in addition to the traditional clinical experience in the 2020 accreditation standards effective July 1, 2020.⁴ 

Multiple disciplines have adopted immersive experiences to provide students with an in-depth understanding of their profession's full scope.³²  Athletic training program directors have reported that immersive clinical experiences increase the exposure that gives the student more time with administrative tasks and day-to-day patient care and fosters stronger relationships with those they interact with while at their rotation.³² 

The clinical immersive experience is designed to give students an intensive practice experience without disrupting their didactic coursework. Therefore, increasing their knowledge and experience through increased clinical immersion time increased the 1YR BOC exam pass rate. However, a large variation in this variable may exist due to the CAATE not requiring a clinical immersive experience until July 1, 2020. Therefore, the number of hours may be significant due to some programs reporting no immersive clinical hours per week.

Additionally, the interpretation of these results should be made with caution. The clinical immersion hours ranged from 0 to 60 hours per week with a mean of 30 hours per week (SD = 14.66). The concern is that, if hours increased too high, students would experience a negative impact. In a study by Turocy,¹³  undergraduate athletic training students who gained precisely 400 hours of experience beyond their required clinical hours had higher BOC pass rates. However, students who had more than 400 hours over the required amount saw no improvement in scores. Therefore, it may be wise for programs to limit maximum clinical immersion hours to avoid burnout and provide for a positive work-life balance.

The focus of this study was on modifiable programmatic factors and their relationship with first-time BOC pass rates. While student and institutional factors were included, only admissions selectivity had a significant relationship with first-time BOC pass rates. Therefore, the results of this study support Astin's¹⁷  theory that the environment for which the student is exposed has a relationship with the program's first-time BOC pass rates. The identification of modifiable programmatic factors aids programs in the decision-making processes for resource allocation or program delivery to improve their programmatic outcomes.

This study's limitations include the small sample size; several institutions with accredited PM programs were not included due to undergraduate data included in their reporting and inability to determine if they had graduated a PM cohort. The small sample size limits the model's ability to detect small effects. Therefore, variables with a smaller effect on first-time BOC pass rates may not be captured in this study. Additional research is needed in the future when a large population of PM programs is available to validate these findings further.

The missing data, for which the population mean was used for imputing the missing data, may have increased bias within the sample. Also, the use of race as a dichotomous variable may have limited the sensitivity of this variable and its ability to differentiate between different categories of race and ethnicity. Therefore, the results should be interpreted with caution.

Additionally, in this study, we evaluated data from 1 year of reporting, and we did not capture changes in the program over the time the cohort was in the program. However, typical PM programs are 2 years in length, and the variables may have changed very little in that time, which may have minimal impact on the results. This study is limited to programmatic factors that are reported to and collected by the CAATE. Therefore, additional programmatic factors, such as program GPA, may have a significant relationship with first-time BOC pass rates not used in this study.

With the enactment of the 2020 accreditation standard, changes will occur within the educational content and structure of PM AT programs. The requirement of clinical immersive experiences and the development and maintenance of faculty contemporary expertise are 2 additions to the standards that warrant further investigation based on the findings of this study. Since in this study we used data before the enactment of the standards, future research is needed to assess if these changes result in a change in their relationship with BOC pass rates.

Additionally, the documentation of contemporary expertise will allow for a more in-depth analysis of the relationship between faculty development and student outcomes. The CAATE is requiring programs to document the type, frequency, and content area of professional development. These factors should be considered for future research to further investigate the relationship between professional development and BOC pass rates.

This study was necessary to fill the gap in the literature on the relationship between programmatic factors and student outcomes. Currently, the entry-level degree requirement is transitioning from the bachelor's to the master's level, and most programs have yet to complete the transition.⁴  Therefore, little literature pertains to first-time BOC pass rates of PM programs. To our knowledge, in this study, we are the first to use the CAATE programmatic data to examine BOC first-time pass rates, therefore including the largest sample of PM athletic training programs to date.

The athletic training program first-time BOC pass rates have become more important in recent years, with CAATE requiring programs to have a minimal pass rate of 70% to maintain good accreditation standing.⁴  Additionally, accreditation status and first-time BOC pass rates often factor into students' decisions to attend a particular program. Therefore, with the declining rates of program enrollments within athletic training programs,²²  it is imperative programs maintain high first-time BOC pass rates. Maintaining high programmatic first-time BOC pass rates benefits the program, students, and the athletic training profession. Programs seeking to increase their first-time BOC pass rates should evaluate their clinical immersive experiences, faculty composition, and investments into professional development of faculty as areas for improvement.