The influence of online video learning aids on preparing postgraduate chiropractic students for an objective structured clinical examination

Chiropractic students are required to achieve an extensive list of competencies, across a wide range of domains as part of their professional training.¹  It is the task of educational providers to impart the necessary knowledge, skills, and models of behavior to ensure that a student meets the minimum standards expected for an entry-level chiropractor upon graduation. While the directives for educational providers are standardized across most regions by the relevant accreditation bodies, the pedagogical approach through which these directives are to be achieved is less prescribed.

Another aspect in professional training programs that requires the attention of educators is the question of which medium, or combination of media, is appropriate to facilitate the delivery of educational messages/concepts to students. As internet access has become ubiquitous, online educational resources have become commonplace,^2–4  either replacing more traditional teaching methods or providing options for the delivery/function of educational materials, allowing for blended learning.⁵  The use of online procedural videos in education is an example of one of these resources.

An examination of the educational literature reveals that the incorporation of procedural/instructional videos, online or otherwise, into a range of health-care training programs has had mixed results.^6–17  In chiropractic education, instructional/procedural videos have been used within some programs to facilitate skill acquisition and bolster learning. These videos are valued by students¹⁸  and have been shown to be equal to, or superior to, some of the more traditional teaching methods.^19,20  However, the development and production of procedural videos can be costly and labor intensive. With conflicting results observed in other health-care fields,^{6,9–11,14,15}  it is important to establish whether these resources are of value to chiropractic students. Therefore, the aim of this study was to investigate the influence of providing supplemental instructional video on student performance in an objective, structured clinical examination (OSCE).

The study involved the prospective recruitment of a convenience sample of students from the Master of Chiropractic program at Macquarie University in Sydney, Australia. The sample included all consenting students enrolled in the 2017 offering of Neuromusculoskeletal Diagnosis I, a unit dedicated to teaching the diagnostic assessment of common lumbar spine and lower extremity conditions. A second cohort of students consisting of a convenience sample of all consenting students enrolled in the 2016 offering of the unit served as a retrospective control.

All consenting students aged ≥18 years who had been enrolled in either the 2016 or 2017 offering of the unit were eligible to participate.

Ethics approval was granted by the Macquarie University Human Ethics Committee on March 5, 2017 (reference number: 5201700375).

The researchers created a series of short videos that detailed the step-by-step performance of a selection of the diagnostic procedures/techniques taught in the unit, specifically those relating to the assessment of lower extremity conditions. There were 62 videos (high-definition audiovisual recordings) in total: 17 for the hip, 26 for the knee, and 19 for the foot/ankle. The mean duration of the videos was 55 seconds (range, 23–99 seconds). The videos were produced and edited by members of the research team. The content of the videos was based on the lecture, tutorial, and reading materials associated with the unit. Each video included the name and purpose of the procedure, audio voice-over and visual instructions on how to perform the procedure, and a guide to the clinical interpretation of the findings. Students in the 2017 cohort were given electronic access to the videos via the university's online learning management system for approximately 4 weeks prior to the final OSCE in the unit. Students were free to access these resources at their leisure, and no restrictions were placed on the frequency or duration of access. Furthermore, any video or section of a video could be viewed at the student's discretion. The videos were not downloadable but could be streamed to any electronic device. A list of all the procedural videos is detailed in Appendix A, and links to 3 examples of the videos are provided in Appendix B (appendices are available as online content at www.journalchiroed.com).

Students perform 2 OSCEs in the unit. The structure of each OSCE is identical; the only difference is the region-specific content being examined. For example, the first OSCE (OSCE 1) involves an assessment of a student's capacity to perform and interpret the findings from a series of orthopedic tests for the lumbar spine and pelvis. The second or final OSCE (OSCE 2) comprised an assessment of a student's capacity to perform and interpret the findings of orthopedic tests relating to the hip, knee, ankle, and foot. The first OSCE was conducted in week 6 of the semester, and OSCE 2 is conducted in week 12. Each OSCE is designed to assess the previous 6 weeks of course material that has been presented. Importantly, the same teaching schedule and assessment structure was in place for both the 2016 and 2017 cohorts. However, the procedural videos were introduced in the second half of the 2017 semester; therefore, a description of OSCE 2 has been presented below.

The OSCE consisted of 2 stations. Pairs of students were required to perform 4 procedures at each station, whereby 1 student acted as patient while the other student performed procedures and vice versa. Student pairs were not given the same procedures during the assessment at a given station, and the order of pairs as patient–student was reversed at the subsequent station. Procedures were chosen at random from the list of procedures detailed in Appendix A. There was 1 examiner per station, and students were given 4 minutes per station to complete the tasks. Students were awarded marks based on several criteria: correct performance and understanding of the purpose of the procedure; understanding of the relevant indications and contraindications for the procedure; and safety, confidence, and finesse in the application of the procedure. A score out of 24 was awarded to each student at each station, with each of the 4 procedures at each station being worth 6 points. The total final score was the sum of the scores from each station, giving a numerical score out of 48. Regarding the examiners, 3 out of the 4 examiners who marked the 2017 OSCE were also involved with the marking of the 2016 OSCE.

The primary outcome for this study was the change in OSCE scores between the initial and final OSCE. The null hypothesis was that the introduction of the learning aids for the 2017 cohort (intervention group) would have no significant effect on performance (i.e., OSCE score) during the final OSCE compared to the 2016 cohorts' (control group) performance in the same assessment in the previous year. It is of importance that the structure, content, and marking scheme of the 2016 OSCE were identical to that of the 2017 OSCE. Several additional variables were collected from both cohorts at similar points in their academic candidature: grade point average (GPA) (on a 4-point scale); age; gender; and score on the initial OSCE for the unit. The intervention group was also issued with a questionnaire that sought to elicit details regarding their access to the videos, how useful they found the videos, and their perceptions regarding whether the videos improved their preparedness for the final OSCE. The questionnaire was developed by the research team and given to students directly after their participation in the final OSCE. The questionnaire took approximately 5 minutes to complete and is provided in Appendix C.

Data were collated, deidentified, coded, and cleaned. Descriptive statistics were generated for all variables using mean and standard deviation (SD) and a 95% confidence interval (CI) for numerical variables such as the OSCE total scores and counts with percentages for categorical variables. Change in OSCE score for each cohort was assessed using paired t tests, and the difference between both cohorts' OSCE scores was assessed using Welch 2-sample t tests. Linear regression was used to examine the influence of various factors thought to be linked to score achievement (age, gender, GPA) on the final OSCE score for both the 2016 and the 2017 groups after adjusting for the initial OSCE score. Standard methods were employed in which all variables were entered at once. Simple linear regression was used to examine the association between the number of times resources were accessed (once, twice, three times, more than three times) or timing of access (night before, week before, weeks before, as soon as possible) and change in 2017 OSCE score. Regression assumptions were checked using the usual diagnostic checks of normal quantile plots and histograms of the residuals, scatterplots of the residuals versus the fitted values, and leverage plots to determine the presence of influential points. Partial regression plots were used to determine the linearity of the relationship of each continuous predictor with the OSCE score after adjusting for the other predictors. If influential observations were identified, models were fit with and without the influential values to determine the sensitivity of the results to these observations. A significance level was set at .05 for all analyses. Analyses were conducted using various packages within the R statistical software program (version 3.4.1; R Core Team, Vienna, Austria).

Content analysis was performed to determine the major and minor themes in the free text comments made by students regarding the video resources.

The 2016 sample consisted of 83 students (41% female) with a mean age of 24 years (SD = 5 years). The 2017 sample consisted of 90 students (34% female) with a mean age of 25 years (SD = 6). Descriptive statistics for each of the study variables are detailed in Table 1.

A significant difference in the mean change in initial and final OSCE scores between the 2016 and 2017 cohorts was observed (mean difference: −4.66 points, 95% CI: −6.34 to −2.99; p < .001, 2-sample t test, Table 1). There was no significant change in OSCE (final–initial) scores for the 2016 cohort (mean difference: −1.10 points, 95% CI: −2.39 to 0.20; p = .09, paired t test). A significant increase in scores was observed for the 2017 cohort (mean difference: 3.57 points, 95% CI: 2.49 to 4.65; p < .001, paired t test) in the final OSCE compared to the initial OSCE.

Using linear regression in the control (2016) group (Table 2), undergraduate GPA was associated with significantly higher final OSCE scores (each unit increase in GPA was associated with between 4.9 and 8.6 more points on the OSCE, p < .001), but age and gender were not significant (p > .43). For the intervention (2017) group, neither age (p = .30), nor undergraduate GPA (p = .08), nor gender (.40) were significant predictors of final OSCE score. All regression assumptions were met, and while 2 participants in the 2017 cohort were found to be potentially influential because of older age compared to the rest of the group, removal of their data did not substantially change the parameter estimates or significance (not presented).

Regarding the survey questions, all but 1 student (who forgot about the resource) accessed the videos and used them to prepare for the final 2017 OSCE. The majority (60.7%, [54/89]) of students reported accessing the videos more than 3 times prior to the OSCE; however, simple linear regression indicated that the number of times a video was accessed was not associated with amount of change in OSCE score (p = .48). The median score for perceived usefulness of the videos for learning the procedures was 10/10 (interquartile range [IQR] = 2), with 95.6% (85/89) of students providing scores of ≥7/10. In terms of access, the greatest proportion of students accessed the video resources in the week leading up to the final OSCE (see Fig. 1), though the timing of access was not associated with change in OSCE score (simple linear regression, p = .15). When asked how the inclusion of the video resources affected preparedness for the OSCE, the overwhelming majority (92.1%, 82/89) reported that the videos increased their sense of preparedness.

Sixty-nine (77.5%) students contributed to the additional comments section of the questionnaire. Content analysis was performed on the students' free text comments. Comments were first sorted into 3 categories; positive, negative, and mixed. Comments that contained both positive and negative remarks or neutral statements were categorized as mixed. There were no purely negative comments made by the students. Purely positive comments were made by 88.6% (62/70) of the sample, and 11.4% (8/70) gave mixed responses. Students giving positive comments had significantly higher change in OSCE scores than did students giving mixed comments (mean difference = 1.18, 95% CI: 0.53 to 1.83, p = .002, 2-sample t test). Positive comments most commonly took the form of praise and gratitude for the resources, with comments regarding how useful/helpful the videos were and the desire for similar resources to be made available in other units. Themes that arose in the mixed comments group were that more detail could have been added to some of the videos and that making the resources downloadable would have been beneficial. There were no negative comments about the content accuracy or quality of the videos.

The findings from this study suggest that the introduction of these types of resources had a small but positive effect on student performance despite the control group having a higher GPA. The resources were well accessed and were highly regarded by the students in the sample.

There has been some investigation into the use of video as a learning aid in chiropractic education. In 2010, Hecimovich et al.¹⁹  studied the effect of video self-assessment versus clinician feedback on student communication skills during a history-taking task. The authors reported that student communication skills were enhanced through the use of video self-assessment and that this method was comparable to receiving feedback regarding performance from a supervising clinician. Zhang and Chawla²⁰  examined the impact of adding procedural training videos to student performance in an ophthalmic examination. The researchers provided both procedural videos and mistake-referenced videos and compared these against a more traditional method of content delivery. The findings of Zhang and Chawla's study revealed that student physical examination performance was enhanced by having access to standard procedural and mistake-referenced procedural videos of the ophthalmic exam.

Procedural videos have also been used in the training programs of other health-care disciplines. There are, however, some issues with comparing the results from our study with these studies, namely the vast differences in the nature of the procedures taught in this study versus those being taught in these related fields. The diagnostic procedures being taught in this study are relatively innocuous. Students can practice these procedures without fear of injuring themselves or their practice partners. Unfortunately, much of the literature on procedural videos in health-care settings is focused around surgical or other invasive medical/dental procedures, where repeated practice and rehearsal can be challenging due to the difficulty, risks, or cost associated with the procedure or as a result of limited access to cadavers, models, or real patients.¹² 

In this study, students were given unrestricted access to the video resources. Although the researchers did not examine this issue specifically, it is likely that the students accessed the videos during their practice sessions, which may have helped to facilitate correct psychomotor skill development, accuracy, timing, confidence, and preparedness. The students in this study reported that having access to procedural videos did in fact increase their sense of preparedness for the upcoming OSCE. This is a common theme across the majority of studies that have implemented this type of educational resource in health-care settings.^6–9  It is interesting that increased comfort or heightened preparedness for an upcoming performance through access to procedural videos generally doesn't correlate with, or have a significant impact on, actual objective performance/outcomes compared with more traditional teaching methods.^6,9–11,14  One exception to this trend is a study by Mehrpour et al.¹⁵  who studied the effect of adding procedural videos to a group of 474 medical students learning orthopedic splinting techniques. The treatment group received traditional teaching methods plus additional access to a procedural video of the splinting technique. The control group was provided with identical teaching resources minus the procedural video. The 2 groups' splinting skills were then tested 6 months later in an OSCE. The authors found that the intervention group performed significantly better than the control group, with an estimated difference of approximately 7% between the groups. It is not clear why some studies have demonstrated improvement in student performance and others have not. It may be that there is a disconnect between the learning of the procedure and the way in which the performance of the procedure is being examined in an OSCE setting. Using this logic, a video specifically designed to help students with the performance of the procedure during an OSCE may yield more consistent results. However, such a practice would be deviating from the objectives of the educational program.

Whether or not a particular medium/media is appropriate is partly dependent upon an individual's learning style. Learning style represents a learner's distinctive and habitual manner of gathering, processing, interpreting, organizing, and analyzing information.^21,22  The inherent educational value of a lecture, tutorial, or prescribed reading item is influenced by the student's learning style. Using the Visual, Aural, Read/Write, Kinesthetic (VARK) questionnaire, a measure of learning style based on sensory modalities,²³  Whillier et al.²⁴  conducted a cross-sectional study investigating the learning styles of 407 chiropractic students sampled from each level of a 5-year university program. The authors found that the majority of their sample were classified as multimodal learners but preferred taking in information using all the 5 senses (kinesthetic type).²³  These results would suggest that from an educational design perspective, offering learning materials across a variety of media should maximize the congruence between the material being presented and an individual's learning style. In educational design, such an approach is termed a “balanced” design.²²  The approach used in this study aligns with both a blended and balanced approach to educational design as the procedural videos represented an addition to the normal unit materials. Although not used to its full capacity in this study, the learning management platform also has the capacity to allow students to share, comment, and take part in discussions about the videos with other students. This could be used in the future to add a social constructivist element to the design of the unit.

YouTube and similar platforms are frequently accessed by students looking to find supplementary resources to aid with their learning. As these types of platforms are openly accessible to both content creators and consumers, the quality of the materials can sometimes be questionable.^3,25  Students in this study were provided with resources that had been developed with, and reviewed by, the lecturer associated with the unit. This provided some assurance as to the quality of the content and regulation over version control, attributes that may not be present in open-access forums.

There are a few limitations with this study. Constructing the intervention and control groups from 2 consecutive-year groups could imply that there is an examiner effect or that the OSCE tests in the former year were somehow more difficult. The list of assessable procedures was identical for the 2 cohorts, which negates the content argument. However, there was 1 new examiner introduced into the 2017 teaching team, which may have had a small effect on OSCE scores. The pairing of students could possibly also impact the scores or produce dependence into the data. Future work should look for carryover effects in the analysis if a similar design is used. Furthermore, it is not known to what extent factors such as workload in other units, socioeconomic status, outside work, travel time, psychosocial issues, or individual attitudes toward learning may have differed between groups. The questionnaire used in this study was purpose-built by the research team. There was no formal examination of the internal and external validity of this instrument, nor was it piloted prior to being administered. The information obtained from the questionnaire should therefore be interpreted with caution.

It is important to note that the positive effect that has been observed in this study is based on a single comparison of test scores between 2 small cohorts. While it seems intuitive that the addition of extra learning resources to 1 cohort would be advantageous to student performance, the result obtained in this study may be due to chance or other factors that have not been accounted for in the research design.

While the addition of procedural videos to the curricula of health-care programs is nothing new, the way in which students are accessing and interacting with such resources has changed dramatically. The advent of wireless internet and the creation of Web 2.0-enabled mobile devices means that the student experience now extends beyond the traditional classroom setting. Learning is now more independent⁴  and interactive as a result of these developments. The use of procedural videos allows students to view techniques in familiar settings with realistic timing and motion²⁶  and allow for a richer communication of information that is not present in some of the more traditional teaching resources.²⁷ 

Creating videos is more time consuming than the production of traditional educational materials such as lectures.²⁸  Furthermore, a fair degree of technical expertise is required on the part of the designer.⁴  The creation of the procedural videos in this study, including both pre- and postproduction work, took approximately 100 hours. While most students found the resources helpful, educators will need to consider whether the amount of work required to produce these types of resources is justifiable given that only marginal improvement in performance was observed. Despite producing only small improvements, students did perceive that the resources were both helpful and contributed to their learning, which from an educational design perspective might be considered sufficient justification for the effort.

Chiropractic clinical education involves far fewer invasive procedures when compared to other health-care fields such as medicine or dentistry. Replacing/supplementing the real-life performance of a procedure in these fields is intuitive from an ethical, fiscal, and practical point of view. However, this is slightly harder to justify in chiropractic education given the nature of the field.

The use of online video is a common component of balanced and/or blended educational designs. In this study, the addition of online procedural videos to the standard unit learning materials of a group of postgraduate chiropractic students appeared to have a small but positive effect on performance in an OSCE. The additional resources were perceived by the students as being helpful for preparing for an examination. Further research is required to establish whether this type of resource represents an essential component of a chiropractic education program or is simply a “nice-to-have” aspect of the overall educational design.

The research team would like to thank the learning and teaching support staff in the Faculty of Science and Engineering at Macquarie University for their help and advice regarding the filming and production of the learning aids and for making available the equipment required for filming.

This work was funded internally. The authors have no conflicts of interest to declare relevant to this work.