In 2012, the Council on Chiropractic Education (CCE) enacted new admission standards with related provisions under a new Policy 7, the Alternative Admissions Track Plan (AATP). The current study examined the relationships between typically admitted students and their AATP counterparts on three student success outcome measures: Graduation at the 150th percentile time frame, National Board of Chiropractic Examiners (NBCE) Part I pass rates, and completion of all four NBCE examinations within 6-months after graduation.
The authors used three random samples (n = 1050) drawn from a relational database, containing program outcome variables and student characteristics. Assessment of the outcome measures occurred using Pearson χ2 test of independence and the Φ coefficient effect size measure.
Significant relationships with small effect sizes and weak associations were found between AATP status and graduation at the 150th percentile (p < .01, Φ = .118) and NBCE Part I pass rates (p < .01, Φ = .114). No significant association between AATP status and NBCE Completion rates 6-months after graduation (p = .144, Φ = .045) was found.
The weak associations between variables indicate that AATP status did not meaningfully relate to the outcome variables. There likely are other subtle characteristics and attributes that influence successful completion of key programmatic outcomes. The weak associations found in the current study suggest that when governed under the same academic policies with equal access to support resources, there does not appear to be a meaningful association between the programmatic success of AATP and non-AATP students on key outcomes.
In Nathaniel Hawthorne's classic novel, The Scarlet Letter, the Puritans required Hester Prynne to wear a scarlet letter “A” in public to shame and stigmatize her for her involvement in an adulterous affair. As time and circumstances passed in the novel, the scarlet letter took on a different meaning to Hester; one of repentance, dignity, and triumph. While this is an extreme case of societal judgment, it nevertheless exemplifies how perceptions can change when new information is presented.
In 2012, the Council for Chiropractic Education (CCE) enacted new admission standards with related provisions under a new Policy 7, the Alternative Admissions Track Plan (AATP). The aim of the policy was to allow chiropractic colleges greater flexibility in their enrollment criteria, but also greater accountability for ensuring student success within chiropractic programs.
Chiropractic colleges adhere to the minimum admissions requirements as set forth by the CCE, the national accrediting body for doctor of chiropractic programs (DCP). The 2018 CCE Standards require:
“The DCP admits students who possess academic and personal attributes consistent with the DCP's mission, and who have completed the equivalent of 3 academic years of undergraduate study (90 semester hours) at an institution(s) accredited by an agency recognized by the U.S. Department of Education or an equivalent foreign agency. The 90 semester hours will include a minimum of 24 semester hours in life and physical science courses appropriate as undergraduate preparation for chiropractic education as determined by the DCP. At least half of these courses will have a substantive laboratory component.”1
In addition, incoming students must have a grade point average (GPA) of at least 3.0 on a 4.0 scale for the 90 semester hours. Chiropractic colleges may accept students under an AATP designation who have not met the minimum admissions requirements. Policy 7 regarding AATP students mandates, “No student is to be admitted who has completed fewer than 90 semester hours and/or has a GPA for these 90 hours of less than 2.75/4.0.”2
According to Policy 7, each institution's AATP must address alternative criteria for accepting students who do not meet the minimum requirements, including a record of each student's academic load and performance (GPA, academic progress, and results of external licensing examinations); evidence that regular reviews of the student's academic record and performance are used to inform appropriate academic support services; and an academic plan for each student who fails to make satisfactory academic progress.
Inherent within Policy 7 is an untested assumption about what promotes students' successful completion of chiropractic programs and programmatic outcomes. There are three facts and logical conclusions to consider related to Policy 7: (1) Students admitted under CCE Standard 2.G admission requirements do not require additional institutional resources and attention to be successful. (2) Students admitted under AATP will require additional institutional resources and attention to be as successful as those students admitted under CCE Standard 2.G admission requirements. (3) Therefore, students admitted under AATP will be less successful in DCP programs unless they receive additional resources and institutional attention.
Following this logic, the policy appears to assume that typically admitted students (ie, non-AATP status) will have a greater likelihood of success compared to AATP-designated students. Thus, we examined this overall assumption by evaluating the relationships between typically admitted students and their AATP counterparts and three student success outcome measures: graduation rates at the 150th percentile (1.5 times the standard time frame to graduation; ie, CCE Policy 56), National Board of Chiropractic Examiners (NBCE) Part I pass rates, and students' ability to complete all four NBCE examinations within 6-months after graduation (ie, CCE Policy 56).
Although the creation of CCE Policy 7 was a starting point for the profession to broaden admission standards by using the best available evidence to guide its development, a deeper understanding of the true impact of AATP criteria may help with policy revision to assist chiropractic educational institutions with more focused remediation efforts.
To this end, three research questions guided the current study: (1) Is there a meaningful relationship for graduation rates at the 150th percentile under CCE Policy 56 between those identified as AATP and non-AATP students? (2) Is there a meaningful relationship for NBCE Part I pass rates (first sitting) between those identified as AATP and non-AATP students? (3) Is there a meaningful relationship for NBCE completion rates under CCE Policy 56 between those identified as AATP and non-AATP students?
The institutional review board of Palmer College deemed this retrospective, quasi-experimental study (ie, participants were not randomly assigned to experimental groups) as nonhuman subjects research as it was a secondary data analysis of deidentified, pre-existing institutional data.
Palmer College of Chiropractic is one college with three campuses: a main campus in Iowa and branch campuses in Florida and California. Although the curricular schedules for each campus differs, and the programs differ slightly, academic policies, procedures, and assessment of programmatic outcomes across the campuses are the same. The general admission requirements of Palmer College conform to criteria outlined in the CCE Standards. The AATP admission criteria of the college, as posted on the institution's website, are, “Applicants with less than 24 semester credits in life and physical sciences and/or a cumulative GPA less than 2.75 for degreed students and 3.0 for non-degreed students may be considered under an alternative admissions track plan (AATP). This point of entry requires a more extensive review of the applicant's enrollment factors by both academic and admissions personnel. Decisions for admission are made on a case-by-case basis.”3
In line with CCE Policy 7, Palmer provides student services and academic support and other student services to any student who could benefit from them, including AATP students. Although the mechanism by which AATP and non-AATP students enter the pipeline for certain academic and support services may differ, ultimately, such services are available to all students at each campus. To this end, although any student can elect to go on special schedule, the portal of entry for special schedules typically occurs for any student experiencing academic difficulty (ie, academic warning, probation, etc) or early alert referrals by faculty.
The data used for this study consisted of 1796 students (AATP = 342, non-AATP = 1454) admitted to three college campuses between January 2012 and October 2014, with “on-time” graduation dates between March 2015 and December 2017, and 150th percentile graduation dates between December 2016 and September 2019. Within this population sampling frame, 60% were men, 76% were White, 1.5% were academically dismissed, and 13% withdrew from college. Of the withdrawals, 28% stopped out (ie, left the college and re-enrolled at a later date). Table 1 reports the demographic information for the population sampling frame.
Inclusion and Exclusion Criteria
The inclusion criteria for the first question concerning graduation at the 150th percentile was admission to the program between January 2012 and October 2014 with corresponding 150th percentile graduation dates between December 2016 and September 2019. Also included in this random sample were students who either dropped out (ie, withdrew and did not return) or were academically dismissed. Stop outs (ie, re-enrolling at a later date after withdrawing) also were included in the analysis only if their return back to college remained congruent with their original 150% graduation time frame. Thus, students who stopped out 1 or more times and whose readmission pushed their 150% graduation time frame later than the confines of this study were excluded from the analysis. Also excluded were students who remained enrolled within the program, but who had not yet exhausted their 150% time frame.
Inclusion criteria for the second question regarding NBCE Part I performance were admission to the program between January 2012 and October 2014 and who sat (first-time takers) for the NBCE Part I examination during the study's time frame. Finally, inclusion criteria for the third question regarding the completion of all NBCE examinations 6-months after graduation were admission to the program between January 2012 and October 2014 with corresponding 150th percentile graduation dates between December 2016 and September 2019. Included students must have sat for at least one NBCE examination and must have graduated. Those students who graduated and had not yet completed all NBCE examinations, but who also had not exhausted their 150% time frame, were excluded from the study.
The study team dichotomized all variables of interest in this study. The explanatory variable was AATP status, (AATP designation and non-AATP designation; ie, 0 = AATP, 1 = non-AATP). The outcome variables for the study were graduation within a 150th percentile of the standard time to graduation (0 = not graduated, 1 = graduated); NBCE Part I (first sitting) exam results (0 = fail, 1 = pass); and completion all four NBCE examinations within 6 months after graduation (0 = not successful, 1 = successful).
Data and Power Analysis
The analytic method used in this study was the Pearson χ2 test for independence, with a 2 × 2 design. Two basic assumptions must be satisfied to use this technique: (1) each subject must contribute data to only 1 cell and (2) the total number of subjects should be at least 20. The study team reported the χ2 statistics, along with the phi coefficient (Φ) coefficient as a measure of effect size. Association thresholds for Φ are .1 (small), .3 (medium), and .5 (large).4 More precisely; negligible (.00 and under .10), weak (.10 and under .20), moderate (.20 and under .40), relatively strong (.40 and under .60), strong (.60 and under .80), and very strong (.80 and under 1.00) association.5 For this study, negligible and weak associations were considered not meaningful (possessing little to no practical utility), whereas moderate to very strong associations were considered meaningful (possessing practical utility). The authors also provided 95% confidence intervals (CI) for all effect size measures [CI95 = r ± (Tc) (Sr)], where r is Φ and Tc is the critical t-value (ie, 1.960); and Sr is the standard error where r is Φ and n is the number of cases in the study:
Using CIs allow the study team to understand that 95% of the time, the true population effect size parameter would fall between the lower and upper limits of the CIs, which conversely means that 5% of the time, the true population parameter would fall outside the CI bandwidth.
The α level for this study was .05 and data were analyzed using SPSS 25 (IBM Corporation, Armonk, NY). The study team selected three random samples of 1050 cases each from the database to achieve a power of .9, to detect an effect size of .1, meaning that the study team had a 90% chance of detecting a small effect size.4
From the overall dataset (n = 1796), application of the inclusion / exclusion criteria outlined above resulted in three final sampling frames to address the three guiding research questions (RQ1, n = 1768; RQ2, n = 1623; and RQ3, n = 1462), from which we selected three random samples (n = 1050). The samples were drawn in this way to control for each of the unique inclusion and exclusion criteria that were not necessarily mutually exclusive across the three research questions (Fig. 1). Table 2 reports the demographic information for the three random samples, which mirrored the sampling frame population in Table 1.
Assumptions for the χ2 test were met successfully for all analyses, as the samples were robustly over the minimum 20 cases and each case contributed to only one cell (ie, cases were not duplicated across cells).
Is There a Meaningful Relationship for Graduation Rates at the 150th Percentile and AATP Status?
Results of the χ2 for graduation rates at the 150th percentile, under CCE Policy 56, and AATP status indicated a significant relationship, χ2(1), 14.744, p < .001. However, the effect size indicated a weak association, Φ = .118 (95% CI, .058 – .178).
Is There a Meaningful Relationship for NBCE Part I Pass Rates and AATP Status?
Examination of the potential relationship between AATP status and NBCE Part I pass rates did not indicate a meaningful relationship. Despite a significant χ2 result, χ2(1), 13.615, p < .001, the effect size indicated a weak association, Φ = .114 (95% CI, .054 – .174).
Is There a Meaningful Relationship for NBCE Completion Rates and AATP Status?
The study team found a nonsignificant relationship for AATP status and NBCE completion rates under CCE Policy 56, which did not indicate a meaningful relationship, χ2(1), 2.133, p = .144 (see Table 3 for all χ2 statistics). Additionally, the effect size indicated a negligible association, Φ = .045 (95% CI, −.015 – .105).
Higher education has documented decades of research on undergraduate student success in college. Most notable of these impacting factors is students' engagement in their college experience. Students' class attendance, positive interactions with their faculty members, involvement in extracurricular activities, and their participation in cocurricular activities increase their odds of successful program completion.6
Few studies address factors impacting successful completion of graduate healthcare educational programs. Researchers in one pharmacology study found that female sex, ACT scores, and average organic chemistry grades inversely related to academic probation7 (ie, being female, having lower ACT scores, and lower organic chemistry grades equated to higher rates of academic probation).
Although researchers in two medical education studies found that previous academic performance predicted achievement in medical programs,8,9 it may not be a perfect predictor.8 Other predictors of successful medical education performance were female sex,8 strategic learning styles,8 white ethnicity,8 entrance exam scores,9 and parental education.9
Noteworthy is that while female sex seems influential in some studies, researchers in other studies did not find relationships with sex and successful completion of medical programs.9 Additionally, researchers in pharmacologic and medical education studies found cohort effects related to successful program completion (ie, admission year predicted greater success).7,9
Research studies concerning factors impacting student success within chiropractic colleges also are sparse. Similar to findings from undergraduate studies, researchers in one study found that student engagement (ie, class attendance) predicted students' successful achievement in unit and practical assessments within a chiropractic course.10
Chiropractic research seems to focus more on characteristics and attributes that relate to students' successful achievement on national board examinations and less on completion of other key chiropractic educational outcomes. In the context of the current study, the combination of variables that comprise AATP status for this institution seems to have little to no meaningful impact on students' ability to successfully complete key chiropractic educational outcomes.
Although the study team found relationships between AATP status and key outcomes, such as graduating the program within 150% of the standard time to graduation and successfully passing NBCE Part I, those relationships were not meaningful within the context of the current study. The 95% CI data in this study suggest that AATP status has a weak to negligible relationship to the key outcome measures within the population. The study team also found that no relationship existed for successfully completing all four NBCE examinations within 6-months after graduation and AATP status. These findings signal that there are likely other personal attributes, characteristics, academic, and/or noncognitive variables that are more significant contributors to students' progress and successful completion of key programmatic outcomes within the DCPs than AATP status.
Given the relatively recent implementation of CCE's Policy 7, few research studies, if any, have examined the relationship between AATP status and student learning or programmatic outcomes. Despite this, recent research studies address myriad factors that associate and/or predict NBCE I examination scores. For instance, incoming GPA11,12 and in-program GPA11,13 appear to be highly associated with NBCE testing, and in some cases, predictive of successful completion of NBCE Part I. Some study teams have found strong correlations with students' grades in courses that parallel NBCE Part I domain areas,12,13 with scores in anatomy and chemistry being predictive of successful completion.14 To this end, tutors significantly outperformed tutees on NBCE Part I success.14 This makes sense, as one might expect, given the heightened knowledge that tutors must possess within key content areas, commensurate with their tutoring responsibilities.
Researchers have reported mixed findings concerning the associations and predictive quality of NBCE practice examinations on students' successful completion of NBCE Part I. For instance, some have found practice exams beneficial,12 while others have found this method of preparation not as predictive as other variables.11
Several researchers have investigated how study strategies and other noncognitive attributes have impacted NBCE Part I scores. For example, some have found that students' goal orientation, as measured by the Learning and Study Strategies Inventory (LASSI), significantly predicted overall mean NBCE I performance,15 whereas others found that LASSI domain scores significantly correlated with NBCE I performance.15,16 Other noncognitive variables, like demographics and employment status, have not been as predictive as other variables.11
Relevance to CCE Policy 7
When our secondary education schools required us to read The Scarlet Letter in our younger years, we were caught up in the injustice of the brand itself, which we suspect, is what Hawthorne wanted because it drove a phenomenal story. However, at its core, Hawthorne's masterpiece is really a story about redemption, a story about the inherent good within people, and a story about individual triumph. This is the story that we see time and again with our AATP students. When we examine institutional and programmatic outcomes comparing AATP students and their counterparts, the lines that determine which students and groups do well and which do not often blur without meaningful insights.
We would argue that CCE likely fashioned Policy 7 in line with higher education's long standing in loco parentis (ie, in place of parents) history. Chiropractic institutions and educators care as much about their students' successful learning and development as their newly minted chiropractic doctors care about their patients' health and wellbeing. While CCE Policy 7 was developed using the best available evidence about which factors might contribute to students' success within healthcare educational programs, more recent evidence suggests that perhaps it is time to revisit Policy 7, to seek answers to critical questions about its effectiveness, and to address ways that chiropractic institutions can identify and work with underperforming students to afford them the greatest opportunity for successful completion of key student learning and professional outcomes.
With the recent changes to CCE Policy 56 that require chiropractic programs to demonstrate 70% graduation within the 150th percentile and 80% completion of all four NBCE examinations within 6 months after graduation, is Policy 7 still needed? The program outcome requirements of Policy 56, which are congruent with other accrediting bodies, may be sufficient alone to impact regulation of admissions criteria and academic support services for chiropractic programs.
Future Research and Practice
As institutional research, assessment, and academic support personnel wrestle with the subtler characteristics that promote academic success of AATP and non-AATP students in our DCPs, the inclusion of key noncognitive variables related to stress management, persistence, personal resilience, and so forth, may be useful examinations, particularly within Alexander Astin's seminal, Input-Environment-Output framework of higher education.6,17
The greatest limitation of the current study is its potential lack of generalizability due to possible differences in AATP admission criteria across chiropractic programs within the confines of CCE Policy 7. DCPs have a great deal of latitude in determining the criteria to admit AATP students. Although the level of flexibility is positive for each college, the downside is that it makes comparisons across colleges challenging. Additionally, this study includes only students from one college with three campuses, which also potentially limits generalizability.
Another limitation of the current study is that the study team did not study support service usage rates between AATP and non-AATP students. Although it could be true that AATP students accessed support services at higher rates than their non-AATP counterparts, we did not study this phenomenon. In reviewing various data related to AATP and non-AATP students, the study team observed relatively equivalent utility rates of support services between AATP and non-AATP students. This observation is anecdotal, however, and should be examined within a separate study.
Significant relationships with small effect sizes and weak associations were found between AATP status and graduation at the 150th percentile and AATP status with NBCE Part I pass rates. No association was found between AATP status and NBCE completion rates 6-months after graduation. The inherent assumption within CCE Policy 7, that AATP students would underperform and require additional resources DC programs when compared to typically admitted students (non-AATP), appears to be unsupported by our findings. Although there certainly are student characteristics and attributes that allow some to succeed and others not to succeed, AATP status alone does not appear to be a valid variable that allows DCPs to meaningfully identify key traits of underperforming students to assist them with successful matriculation and goal attainment. The weak associations found in the current study suggest that when governed under the same academic policies with equal access to support resources, there does not appear to be a meaningful association between the programmatic success of AATP and non-AATP students on key outcomes.
FUNDING AND CONFLICTS OF INTEREST
This work was funded internally. The authors have no conflicts of interest to declare relevant to this work.
Dustin Derby is the Sr. Director of Institutional Research and Effectiveness at Palmer College of Chiropractic (1000 Brady St.; Davenport, IA 52804; firstname.lastname@example.org). Robert Percuoco is the Vice Chancellor of Institutional Effectiveness at Palmer College of Chiropractic (1000 Brady St.; Davenport, IA 52804; Percuoco_r@palmer.edu). Amy Everetts is a Coordinator for Institutional Research and Effectiveness at Palmer College of Chiropractic (1000 Brady St.; Davenport, IA 52804; Amy.email@example.com. This article was received March 22, 2019, revised May 5, 2019, and accepted June 3, 2019.
Concept development: DCD. Design: DCD, AE. Supervision: DCD. Data collection/processing: DCD, AE. Analysis/interpretation: DCD. Literature search: DCD, AE. Writing: DCD, AE, REP. Critical review: DCD, AE, REP.