Engagement as predictors of performance in a single cohort of undergraduate chiropractic students

A student's success at university is attributed to their academic performance in summative assessment, and for many students, their future careers may depend on the grade point average or degree classification obtained. Being able to identify students at risk of poor performance with the possibility of ameliorating such risk by appropriate intervention seems a self-evident goal. In light of this, several studies have attempted to predict students' academic performance, using preentry academic criteria,^1–3  intraprogram performance,⁴  intraprogram nonacademic criteria,⁵  and noncognitive assessments.⁶ 

It is to be expected that a small number of students across higher education fail to perform to the required standard of the assigned learning outcomes for their year of study; and this is no different for students undertaking chiropractic training. In most chiropractic programs, failing students are required to either retake (resit) unit assessments, repeat units in the next academic year before progressing to the next level, or are terminated from the program.

Little research has examined the factors that may influence academic performance in chiropractic education. Therefore, chiropractic educational programs are limited to interpreting conclusions from the few studies that have been published and other healthcare educational programs (eg, medicine, dentistry, nursing) as guides to student performance. However, the results of these studies may not be generalizable to a specific program. Therefore, more awareness and understanding of why students might be unsuccessful within a chiropractic educational program may allow institutions to identify factors particular to this environment, and in so doing target support to students at critical points in their education.

A number of studies related to the admission of students to medical school have focused on an analysis of secondary education performance or clinical aptitude testing^1,7  as a way of predicting student performance in the early years of training. Some debate remains in the literature but general agreement is that student performance in secondary education is a valid method of predicting student performance in the early years of medical¹  and chiropractic⁸  training. While these studies have focused on potential predictors before a student starts their undergraduate training, few have attempted to explore intraprogram academic performance as a method of predicting future academic performance and degree classification.⁴ 

While the majority of studies have looked at academic markers to predict performance, a relatively small number have begun to explore nonacademic factors. Drawing on the work of Lotkowski et al⁹  and Chow,⁵  nonacademic factors can be summarized as factors that influence a student's psychological wellbeing, their sociodemographic background, and overall motivation with their program of study (including educational aspirations, class attendance and hours spent studying). Students engaged in health professions programs are under pressure to learn and apply a amount of information that may directly affect another person's health. This, coupled with the substantive general stresses of university education¹⁰  and the complexity of the transition from learner to clinician,¹¹  would seem to be a crucial factor in student performance.

Of particular interest are the nonacademic factors affecting a student's engagement with their program of study. Exploration of behavioral engagement, when students comply with behavioral norms, such as attendance and involvement, was the starting point for the current study. Chiropractic education in the United Kingdom combines basic science knowledge with vocational training in clinical environments and mastery of clinical skills. Therefore, it is felt that attendance is an important contributing factor to the overall success of the student and their professional development. The link between attendance and academic performance has been well researched in the health professions; positive and negative correlations between these two variables are reported in clinical and nonclinical classes.^12–15  However, a general trend of positive correlation seems to be evident.

Similarly, there have been positive¹⁶  and negative¹²  correlations between the use of online virtual learning environments (VLE) and academic performance. A VLE is an online web-based platform that allows faculty members to share educational materials with their students. While VLEs have become a standard part of higher education in the United Kingdom, their effect on learning and academic performance is not clear.

Little is known about additional, or special, learning needs (ALN) and how they may affect a student's academic performance despite a significant growth in the number of students being identified as having ALN in the United Kingdom.¹⁷  Up to 10% of students in higher education are registered with a disability, of which learning difficulties, such as dyslexia and dyspraxia, are the most reported type.¹⁸  The Quality Assurance Agency Standards¹⁹  prescribe “reasonable” adjustments for students with ALN, such as the allocation of additional time in a written assessment. What is not clear is if reasonable adjustments are effective or if there is a link between poor academic performance and ALN students.

Similarly, the link between previous degree qualification and performance in professional training is unclear. While students with previous degrees tend to achieve higher marks than traditional students²⁰  and are attractive to employers due to their life experience,²¹  not all degrees seem to be influential^22,23  and previous degrees may not be an advantage at all.²³ 

Therefore, we aimed to explore the potential links between academic performance in an undergraduate chiropractic master's program and many nonacademic factors, including attendance, VLE use, ALN, and previous degree qualification.

All year 1 students at the AECC University College enrolled in the integrated undergraduate chiropractic master's degree (MChiro) program in 2015 were eligible to participate. Students were given an information sheet with details of the study and an opportunity to ask questions. Students who volunteered for the study signed an informed consent sheet. Data were gathered from centrally controlled databases from first year (Framework for Higher Education Qualifications Level 4) chiropractic students. Ethical approval was sought from the AECC ethics committee in accordance with the AECC ethics policy. Ethics was approved in September 2015.

Attendance registers were taken by tutors in all six units of the year 1 MChiro program. Attendance at lectures was not tracked as lectures are not compulsory for students; however, all tutorial and clinical skills class attendance was taken.

VLE use was tracked through Moodle (available in the public domain at https://moodle.org/). Moodle records each time a student logs onto the VLE and what each individual student does on the VLE. Moodle collects these data as number of “clicks.” The number of “clicks” for each student in each unit was summed to give a total of Moodle use for the academic year. Data were tracked from the first day of the academic year until the date of the final assessment.

Information regarding students who had completed a previous degree to bachelor's level or higher was collected from the AECC Registry Office.

Information regarding students with confirmed ALN, such as dyslexia or dyspraxia, was collected from the AECC Undergraduate Programmes Office.

First sit marks for all units in year 1 were collected from the AECC Undergraduate Programmes Office. In the United Kingdom, students are given two opportunities to pass an assessment in a unit. The first attempt of an assessment is called the “first sit.” If a student is unsuccessful in this “first sit,” they are given a second opportunity to be assessed called the “resit.” If a student is unsuccessful in their resit, they will enter a repeat unit mode and be required to repeat the entire unit in the next academic year.

Anonymization of summative marks and ALN was done by the AECC Undergraduate Programs Office. The investigators received the data anonymized. Anonymization of all other student data was done by an investigator. All data were kept on a secure, password-protected computer within the administration department at the institution.

Data were analyzed using IBM SPSS Statistics 24 (IBM Corporation, Armonk, NY). Comparison between and resit/nonresit students was performed with the χ² test. Independent t tests were used for comparisons where continuous variables (exam marks) were present. When multiple t tests were used, a Bonferroni method to minimize type I errors was used. Normality of data were determined by a Kolmogorov-Smirnov test and determined the use of nonparametric (Mann Whitney U test) and parametric (t test) analysis, respectively. Associations between variables and end-of-year marks were explored using correlation and the ability of attendance, gender, and semester 1 assessments to predict average end-of-year marks was explored using stepwise linear regression analysis.

A total of 66 students consented to take part in the study. The ratio of male-to-female students was 1:1. Of the 66 student participants, nine (13.6%) were registered as having ALN and six (9.0%) had completed previous degrees.

A comparison of performance was made between male and female students and a separate analysis between those students who did and did not undergo resit examinations at the end of the academic year. Table 1 shows descriptive data for the whole cohort. An analysis of performance, split by gender, is shown in Table 2. Generally, male students achieved higher marks than female students across all units reaching statistical significance in four units and the end-of-year average mark. Not unexpectedly, the average marks for those requiring resits at the end of the year (first sit assessments) across all units were significantly lower than those not requiring resits as shown in Table 3. A subanalysis showed that there were no differences in age, gender distribution, previous degree qualification or ALN status between those who did and did not require a resit.

An exploration of the association of whole year attendance with final year average marks and semester 1 assessment outcomes on final year average mark was conducted. For attendance as a predictor of end-of-year performance, the end-of-year average was not distributed normally; thus, Spearman's rank order correlation was used for analysis. There was a significant correlation between attendance and end-of-year marks for the whole group (r_s = .59, p < .001). This also was seen in the resit (r_s = .47, p < 0.01) and nonresit groups (r_s = .30, p < 0.05), respectively, following removal of a single outlier case that had very low attendance but high performance (Fig. 1).

Linear regression analysis of attendance and end-of-year average marks resulted in a predictive linear regression model with attendance and a semester 1 unit involving human structure (unit 105) remaining as independent predictors (adj R² = 0.86, p < .001, Table 4). In a further analysis, attendance was dichotomized using the AECC policy of 80% attendance in clinical skills classes and tutorials. Students with less than 80% attendance across the year were nearly three times more likely to require a resit at the end of the academic year (relative risk [RR] = 2.6; 95% confidence interval [CI], 1.4–4.9).

The dichotomized continuous score of the human structure unit exam at the cutoff between an upper second class and first class honors (>70%) and the categorical pass/fail of the practical skills assessment, both at the end of semester 1, significantly predicted one or more resits from semester 2 assessments. For the <70% category in the human structure course, RR = 3.5 (95% CI, 2.2–5.7). For the fail practical clinical skills assessment, RR = 3.2 (95% CI, 2.0–4.9). Interestingly, lack of previous degree status conferred no increased risk of requiring a resit at the end of the year.

There were no significant differences across all unit marks, attendance, and VLE use between those students classed as ALN and those not in this category.

Previous research undertaken on nonacademic factors affecting performance is minimal. To the investigator's knowledge, there is no research in this area within chiropractic education and as such, research has been gathered from other healthcare programs, such as medicine, nursing and dentistry, to draw comparisons.

Investigation into the association of nonacademic factors with performance is important to consider as this may allow an institution to identify struggling students before they fail summative assessments. Once identified, the institution can bring to bear educational support to potentially ameliorate such risks.

In this cohort, it was interesting to note male students achieving higher marks across all units than female students, with the exception of practical assessment in the second semester. It is unclear why this cohort of students showed this result as such a finding contradicts some studies that have found female students outperforming male students at university.²⁴  However, this may be something peculiar to this cohort or chiropractic educational environments.

On the issue of attendance, several investigators have commented on the association between attendance and performance. Across different disciplines and subjects there is general agreement that students who attend classes frequently are more likely to be successful in assessments.^12–14  One might assume this is self-evident; however, the increasing use of technology in education has brought with it a generation of students who are able to view and interact with learning materials without the need to attend classes. We noted a strong correlation between attendance and performance whereby students with less than 80% attendance in practical classes and tutorials were three times more likely to resit one or more units.

We might assume that in subjects based on understanding and factual recall, a combination of electronic resources and recommended reading may be sufficient for a student to achieve the required learning outcomes. However, in healthcare programs there also is a need to apply knowledge to patient scenarios and to learn clinical skills; these are unlikely to be learned to a sufficient level based on self-directed learning alone. The nature of the chiropractic program is one of a practical vocational course, which requires practice and supervision to reach a mastery level. If students do not attend and engage with the course, it appears reasonable to state that they are unlikely to achieve a successful performance at first-sit assessments. In an effort to help students identify this lack of engagement and link absence from classes to professional conduct, some have suggested sanctions be placed on students²⁵  to aid their professional development. Assuming such sanctions were effective at dissuading students from being absent from classes, the effect would aid their overall performance and be a positive measure in ensuring student success.

To further support the assumption that students may not learn all materials without attending classes, we found that VLE use was not correlated significantly with summative marks. This is in keeping with the findings of Azab¹²  and Schreiber et al,²⁶  both of whom found that there was no significant correlation between student's use of online resources and grades. In contrast, Bertheussen et al²⁷  found that digital learning activities were correlated with assessment performance indicating there is a place for their use. However, this study used accountancy problem solving as the digital activity and so direct comparisons to health care cannot be made and further investigation is required.

The observation that those students who achieved less than 70% in one of the semester 1 written exams were approximately three times more likely to have one or more resits at the end of the year was surprising. Since a first-class honors in the United Kingdom is equivalent to marks of 70 and above, one might consider the bar in this subject to be high to be less likely to achieve a resit at the end of the year. The reason for this is unclear and was not the focus of this study. Similarly, achieving a fail in the practical clinical skills assessment increased the risk of the student requiring a resit by three times. This practical assessment may highlight students who are struggling to acquire the necessary clinical skills to become chiropractors and may be a more reliable predictor of struggling students than the 101-unit assessment. However, the linear regression analysis did not retain this predictor in the model and, therefore, its influence may be less than expected when considering other variables.

There was no significant correlation between summative marks, VLE use, ALN status, or previous degree achievement. The findings of VLE use are consistent with other studies¹²  and support arguments that accessing resources online is not obviously advantageous. With regards to ALN, we see the lack of significance as a positive finding that indicates the AECC's policy on reasonable adjustments for ALN students is counteracting the disadvantage of having an ALN. As academic, nonvocational programs seem to have a significant difference between students with and without ALN,²⁸  this result may indicate that chiropractic as a vocational and practical program is better suited to ALN students. With regards to previous degree achievement, it is possible that the chiropractic degree is so different from other higher education programs that there was no advantage to having studied a previous degree.

Some limitations to the study may have affected these results. Firstly, students are required to log into the VLE individually for data to be tracked. However, it is possible that students viewed materials as part of a study group where only one student logged onto the VLE but several engaged in learning activities. This is common practice at the AECC and so students may be using the VLE more than the data suggest and as such further research into student's online learning experience would be beneficial. The number of ALN students and those with previous degrees in this study was small, so there is a possibility of a type 2 error within the data where no significance has been recorded where, in fact, with higher numbers there may be. In addition, the investigators did not record what the previous degrees were and knowledge of this may help us understand why previous degrees were not predictive of performance. Lastly, correlation does not imply causation and given the large number of potential confounders in a study such as this, there may be other factors linked to attendance that are affecting final grades.

We hope that early identification of predictors of performance may facilitate milestones where institutions might target support to students at the beginning of their academic career and at times of crisis in the student's education. This study indicates a relationship between attendance, resits, and performance during the first year of training in our chiropractic students. Our results may assist others in healthcare education during investigations of their own programs.

This work was funded internally. The authors have no conflicts of interest to declare relevant to this work.