The influence of online review videos on gross anatomy course performance among doctor of chiropractic students

Basic science courses in healthcare curricula are evolving rapidly,¹  but traditional lecturing remains the primary component of most anatomy courses within healthcare education.^2–4  Incorporating technology, in and out of the classroom, has the potential to enhance the student experience and revolutionize healthcare curricula. Technology-enhanced learning (TEL) is an interactive, learner-centered pedagogic approach to teaching that uses hardware, software, or mobile apps to improve teaching. Online courses are primarily TEL, but face-to-face instruction may also include components of TEL. Previous research has shown that digital lectures produced similar effects on course performance, when compared to traditional methods of instruction, and students frequently report a preference for access to face-to-face instruction and digital resources.^2,5–9  Several investigators have previously outlined how use of TEL has the ability to accommodate students perceptual and cognitive differences.^2,5,7  Additionally, research suggests that combining TEL with traditional methods of teaching can increase student engagement^5,10  and improve outcomes within medical education.^3,11 

Within the anatomic sciences, recent technologies have allowed for the development of exquisite visual aids on a variety of digital formats. Incorporating detailed anatomic imaging in healthcare education may serve to enhance student engagement, due to the highly visual nature of these topics. Examples of TEL commonly used in healthcare education include: virtual or augmented reality, 3D models, digital anatomy apps, eBooks, and virtual dissection.² 

Although literature on the use of technology to enhance the student learning environment has dramatically increased in the last decade, most research has focused on student satisfaction scores.¹¹  Additionally, the influence of TEL on learning within anatomic sciences, such as gross anatomy, remains relatively unexplored. Of the few published studies on the influence of online material in anatomy courses there have been mixed results.^1,12–14  Development of TEL resources is time-consuming and research suggests that use of videos to supplement student learning should be assessed for use, efficiency, and effectiveness.^1,18  Although TEL resources may influence student engagement and be satisfying for students, little is known about the impact TEL may have on student learning.

Within medical education, use of online review videos has been shown to enhance the instruction of an anatomy course when the videos were specifically designed to address course objectives.¹⁵  Similarly, the use of YouTube videos has been shown to increase student engagement and facilitate learning in a variety of healthcare settings,^15,16  but research is limited regarding the use of TEL within chiropractic education.

Gross Anatomy I at the chiropractic institution involved with this study primarily consists of a traditional daily lecture that is combined with instruction in a cadaver lab. Before this research project, no TEL resources were part of the lecture or lab, other than presentation slides. The purpose of this pilot study was to evaluate the relationships between doctor of chiropractic students' use of online review videos and classroom performance in a traditional face-to-face gross anatomy lecture course. Student satisfaction with the review videos also was assessed.

This quasi-experimental study used a convenience sample of students at Palmer College of Chiropractic in the United States. Ethical approval to perform this project was obtained from the Palmer College of Chiropractic institutional review board.

We invited all students enrolled in the Gross Anatomy I course to participate in this project. Data were collected during the Summer 2017 and Fall 2017 trimesters. Students with special testing accommodations, arranged through the college's Department of Student Academic Services, were ineligible for enrollment in this study because their exams were administered at an alternate testing location. This study had limited personnel and we were unable to perform informed consent and distribute the survey at multiple locations. A total of 215 students were eligible to participate in this survey, which included 112 students from the Summer 2017 trimester and 103 from the Fall 2017 trimester. Participating students completed a 13-item questionnaire at the end of their gross anatomy course, following their final examination.

We designed a series of seven individual gross anatomy review videos, which reviewed content that correlated with various course objectives. These videos used screen capture software that recorded the gross anatomy tutor's voice while displaying various anatomy software applications. Video content included anatomic images, such as three-dimensional (3D) anatomic visualizations from the Complete Anatomy application (3D4Medical, Dublin, Ireland), with related text presented using PowerPoint (Microsoft, Redmond, WA) as displayed in Figure 1. In other videos, hand drawn diagrams were displayed using OneNote (Microsoft). These review videos highlight selected concepts that correspond with content presented in the course lectures. However, the videos presented the information in a unique way that was distinctly different from lecture. Table 1 provides an overview of the content included in each video, the format and duration of the video, and number of views each video received.

Individual URL links to the online review videos were made available to all students enrolled in the Gross Anatomy I course, via a password protected website. These URL links were made available to students via the college's Academic Support Services Department website, hosted by the Brightspace learning management system (Desire2Learn, Ontario, Canada).

Informed consent was obtained before distributing the survey questionnaire. All students received a verbal description of this project and information was provided about what data would be gathered, how data would be protected, how the results would be analyzed, and how the results of this project would be communicated. In addition to a verbal informed consent, a written informed consent document was attached to each questionnaire and signed by each participant.

This project involved the development of a 13-item questionnaire to evaluate students' self-reported use and perceptions of the online review videos. Although several anchors were used across the instrument, students' perceptions of the online review videos were evaluated using four-point scales (e.g., extremely satisfied to extremely unsatisfied) and assessed the following: level of enjoyment using the review videos, helpfulness of the review videos, satisfaction regarding their understanding of course content, and satisfaction regarding their course performance. Questions were designed to evaluate the student perceptions related to ease of use, relevance to class material, and whether the videos were an enjoyable method of studying. To ensure appropriate face validity of the questionnaire, we developed item and scales in accordance with survey design best practices and content domains from the literature. We pretested the questionnaire with content experts and students, to ensure content validity. Subtle grammatical revisions were made following the pretesting process, based on input from students, tutors, and faculty members.

This project combined data regarding each student's final course grade with the student's individual questionnaire responses. The college's registrar provided the Institutional Research office with demographic data of participants including mean age, distribution of sex, ethnicity, and prior education information (prior degrees obtained, incoming grade point average [GPA] and incoming science GPA). Final course grades were obtained from the course instructor. A research assistant collected the student's survey responses. The Institutional Research office matched survey responses with each student's course grade, deidentified the data, and provided the deidentified data to us for data analysis. Only Institutional Research office personnel had access to identifiable data.

To assess the effectiveness of the tutorial videos, two research questions guided the current study: (1) Does students' use of the tutorial videos predict their final course grade in a chiropractic gross anatomy I course? (2) Is there a difference in students' overall satisfaction with their academic performance and satisfaction with learning between those who did and did not use the tutorial videos?

Data were entered into SPSS version 25 (IBM Corp, Armonk, NY) and Excel (Microsoft). Descriptive statistics were calculated to summarize the data. Based on the ordinal nature of overall satisfaction variables, we used ordinal regression to understand the predictive qualities of students' video use on their corresponding satisfaction with their academic performance and understanding the course materials.

We used model fit information to determine if the explanatory variables in the model improved our ability to predict the outcome variable by comparing a baseline, intercept-only model to the model containing the explanatory variables. Pearson's χ² goodness-of-fit statistic, that is, the omnibus test, was used to determine if the observed data were consistent with the fitted model; significant values indicating a misfitted model wherein the data and model are dissimilar. Negelkerke pseudo r-square (nr²) was used to explain the amount of variation between the explanatory and outcome variable.

It was not necessary to assess multicollinearity between explanatory variables because this study investigated only 1 potential explanatory variable (video use). Proportional odds were examined using the ordinal model, with one set of coefficients for all thresholds, to an alternate ordinal model with a separate set of coefficients for each threshold. Significance of this test indicates that the model does not fulfill assumption of proportional odds. No outliers were present within the data.

Noteworthy is that some consider the proportional odds test to be, generally, an anti-conservative test,¹⁷  often resulting in smaller p values that tend to increase the probability of type I error (i.e., rejecting the null hypothesis that the odds ratios are equal in favor of the alternate hypothesis that the odds ratios are not equal when the researcher should retain the null hypothesis). In the event of violation of the proportional odds assumption, we used binary logistic regression analysis to manually examine the odds ratios at each threshold to determine if the variable was appropriate for ordinal regression.

In addition, using the same dependent ordinal variables for satisfaction (i.e., academic performance and understanding course materials), we used the Mann-Whitney U test to evaluate group differences in students' use of the online review videos (i.e., comparing those who did and did not use the videos). We inspected histograms to assess distributional assumptions of the Mann-Whitney U test and calculated the effect size of group differences using the formula: All statistical analyses relied on a standard α (p < .05) to determine significant outcomes.

Of the 215 eligible students, 20 were excluded due to special testing accommodations that required an alternative testing location. A total of 143 (53 summer term; 90 fall term) completed the survey questionnaire, which corresponds to a total response rate of 73.3% (143/195). The demographic data for all respondents are reported in Table 2. The two groups surveyed were comparable in age, sex, incoming GPA, and incoming science GPA. The percentage of students who had not yet completed an undergraduate degree was higher in the Fall 2017 term (23.9%; 22/92) when compared to the Summer 2017 term (13.7%; 7/51). When compared across terms, students did not differ significantly with respect to incoming GPA (t (141) = −.691, p = .491), incoming science GPA (t [141] = −.422, p = .674), or their final Gross Anatomy 1 course grade (t [141] = .345, p = .731).

Overall, the majority of students (71%; 102/143) reported using the videos. The highest percentage of student video users obtained a B in the class (37%). Similar numbers of users obtained an A (28%) or C (29%) in the class. The lowest percentage of students did not pass the class (5%). A similar distribution was seen among students who reported they did not engage with the online review videos (nonusers). The distribution of grade achievement between online review video users and nonusers is reported in Table 3.

Students who reported using the videos also reported satisfaction with their course performance (n = 80, 78%), and their understanding of course material (n = 88, 86%). The majority of students who reported viewing the videos, reported the videos as being helpful (n = 80, 82%) and enjoyable (n = 71, 73%). Nonusers also reported satisfaction regarding their course performance (n = 31, 76%) and their understanding of material (n = 36, 88%). Student responses to the attitude questionnaire are summarized in Table 4. Similar satisfaction scores for course performance were observed among students who did (M = 3.04; SD = 0.87) and did not (M = 2.95; SD = 0.92) report using the review videos. A similar pattern also was observed regarding satisfaction with understanding course content; students who reported using the videos reported similar satisfaction scores (M = 3.18; SD = 0.79), compared to those who reported not using the videos (M = 3.12; SD = 0.81). No significant differences were discovered between the groups of students who did and did not watch the videos when we evaluated for differences in students' satisfaction with course performance and differences regarding satisfaction with understanding course content.

Does students' video use predict their satisfaction with their academic performance? Although the proportional odds assumption has been satisfied for the ordinal model related to students' video use and their satisfaction with their academic performance (χ²[10] = 6.697, p = .754), and the goodness-of-fit test indicated an appropriately fitted model (χ²[10] = 5.802, p = .832), the results of the model fit information suggests that the final model does not offer any improvement over the baseline, intercept-only model (χ²[5] = 4.608, p = .466). Students' video use accounted for a small amount of variance within the model (nr² = −035, 3.5%). This low accountancy indicated that the model containing video use is likely a poor predictor of students' satisfaction with their overall academic performance.

Does students' video use predict their satisfaction with understanding course materials? The assumption of proportional odds was not satisfied for students' satisfaction with their understanding course materials (χ²[10] = 41.247, p = .000). Investigation of parameter estimates and odds ratios, detailed in Table 5, resulting in variant odds ratios at each threshold, which further indicates possible unequal parameter estimates. Unequal parameter estimates and odds ratios likely indicate differential item functioning of the dependent variable.

Although the goodness-of-fit test indicated that the model appropriately fit the data (χ²[10] = 17.396, p = .066), the model fit information indicated that the model did not offer any improvement over the baseline, intercept-only model (χ²[5] = 6.229, p = .285). Students' video use accounted for a small amount of variance within the model (nr² = .048, 4.8%), which indicated that the model containing video use is likely a poor predictor of students' satisfaction with their overall understanding of course materials.

Review of the data for video users and nonusers (i.e., use of videos 30 minutes or more or no video use at all) for the satisfaction with their academic performance variable indicated that the two distributions exhibited the same, approximately normal mound-shape. As such, the Mann-Whitney U test compared differences in the medians of both groups to uncover potential differences. No outliers were present within the data. A comparison of median scores between video users and nonusers (Table 6) indicated nonsignificant differences in students' satisfaction with their academic performance (p = .582). The effect size of the between group differences was small (.046).

Another review of the data for video users and nousers for the satisfaction with their understanding of course materials variable indicated that the two distributions exhibited the same, approximately normal mound-shape. As such, the Mann-Whitney U test compared differences in the medians of both groups to uncover potential differences. No outliers were present within the data. A comparison of median scores between video users and nonusers (Table 6) indicated nonsignificant differences for students' satisfaction with their understanding course materials. The effect size of the between groups differences regarding learning satisfaction also was small (0.029).

The aim of this project was to examine the relationship between the use of gross anatomy online review videos and classroom performance as well as course satisfaction. Additionally, we collected information regarding students' perceptions of their course performance and understanding of course material. We found that the majority of the students in the gross anatomy course used the online review videos, and their attitudes toward the videos were largely positive. We did not find any statistically significant differences, or predictive relationships, between video users and nonusers regarding satisfaction with their academic performance or understanding of course content.

Most of the students we surveyed reported engaging with the online review videos. However, the extent of engagement varied. Among the students who reporting using the videos, their use ranged from less than 30 minutes to over 2 hours but most students engaged with the videos for 31 to 60 minutes. It is unknown whether students were watching each video once or whether they repetitively watched a limited set of videos. More research regarding this distribution is needed to determine how students are spending their time when viewing the review videos. Our findings are similar to those of Bacro et al.²²  who found that when students are given access to neuroscience recordings the highest percentage of students used them for a duration of 20 to 40 minutes.

When we evaluated the effects of video use on students' overall performance in the Gross Anatomy I course we discovered that use of the videos was not associated with a significant difference in either academic performance or their perceived understanding of course content. Although small differences were discovered between groups, these differences were not statistically significant and exhibited a trivial effect size. These findings are in concordance with those of Meyer et al.²³  whose study of undergraduate chiropractic students found that neuroanatomy scores were not significantly influenced when students were provided access to TEL resources. This same study also identified potential barriers to improved performance, such as a “technology learning curve.”²³  Additionally, the Meyer et al.²³  study did not account for the amount of time students spent engaged with the TEL resources; our survey attempted to remove these barriers, but our results remain comparable those of Meyer et al.²³  This may be due to a combination of factors. While healthcare students frequently engage with online resources, graduate level students already may have a preference for resources they have used previously and could be hesitant to try new or different study methods. Among the students who did elect to use the online review videos, it is possible that they may be passively watching the videos or watching the videos while performing other tasks; this may reduce educational impact of these online gross anatomy review videos.

Similar studies involving populations of medical students have yielded varied results. Some studies have found that adding TEL resources into medical education failed to have a significant impact on course scores.^{9,15,22,24–26}  Bridge et al.²⁴  concluded that students using TEL resources have equivalent outcomes to students using traditional methods of learning. Their conclusion was consistent with the our observed results. Other studies of medical students have found associations between engagement in online learning and improved performance. Singh and Min³  found that students who engaged in online learning performed “modestly better” than those who received face-to-face instruction. They also discovered that these effects were magnified if the student engaged in a combination of TEL and face-to-face instruction. These findings also are supported by a meta-analysis published by the United States Department of Education.^7,24 

Qualitative findings regarding students' attitudes toward TEL resources are largely consistent. Our results indicated that students felt our online review videos were easy to use, an enjoyable way to study, and a helpful resource for studying the course content. Various other studies report similar levels of satisfaction when TEL resources are incorporated into an existing course.^24,26–29  It has been suggested that the favorable levels of satisfaction with TEL may be due to the flexibility of scheduling, unlimited availability, and the ability for students to engage in self-paced learning.^9,24,26 

Despite the positive perceptions our students reported regarding their use of the online review, use of these videos did not have a significant impact on course performance.

This project has several limitations. The current study used a small convenience sample from a limited target population of chiropractic students. Convenience sampling can limit the ability to generalize to other populations and it also limited our ability to calculate power analysis statistics related to statistical power and sample size. However, the effectiveness of a statistical power calculation is greatest when working with a probability sample (i.e., random assignment). Our study used nonprobability sampling (i.e., convenience sample) of a small target population to gain a better understanding of a course-based video tutorial that might be beneficial on a larger scale. As such, this project constituted a pilot study of a course-based intervention, with the intention of collecting preliminary data before, potentially, moving toward a larger-scale study. Despite this limitation, we did include other power analysis information, i.e., effect size, to contextualize any statistical findings. Given the smaller sample size of the study, we wanted to be careful not to attribute importance to findings that are essentially trivial in nature.

Additionally, the pretesting we performed with the study's questionnaire was not a substitution for more formal validation techniques. The differential item function discovered in this study indicated that at least 1 item may have been interpreted differently by different groups of respondents. However, we believe this data artifact to be isolated to the item in question, that is, student's satisfaction with their understanding of course materials.

Our results may help to inform a larger study of a more diverse population of doctor of chiropractic students across multiple college campuses. A larger study would be more likely to impact course design and implementation in anatomic sciences, which is a critical component of all healthcare sciences, especially within chiropractic education.

This study found that most responding students engaged with the course-specific online review videos and most exhibited positive attitudes toward those resources. No significant differences in course performance or perceived understanding were observed between students who did and did not use the videos.

This work was funded internally. The authors have no conflicts of interest to declare relevant to this work.