Differences in learning retention and experience of augmented reality notes compared to traditional paper notes in a chiropractic technique course: A randomized trial

Academic study often involves the memorization of various facts and figures, pushing students to find novel strategies to facilitate and improve their recall.¹  Basic repetition is one strategy but others include mnemonics, visual imagery, and multimodal association links.²  Use of visual imagery has been used for centuries³  and is associated with successful encoding of material into long-term memory.⁴  Although many possible memory strategies are available for students, there is a paucity of research available that studies their effectiveness.^5,6  Furthermore, most research emphasizes memory recovery in individuals with brain damage.^7,8 

This study is based on previous research into learning retention for students given written or visual aids and seeks to determine whether use of augmented reality (AR) notes in the presentation of diversified adjusting technique would improve student recall.⁹  Diversified chiropractic adjusting technique is a system used to describe and analyze a malposition of a single vertebra and the optimal way to correct said malposition.¹⁰  Correction includes the patient's and doctor's position, the point of contact on the affected vertebral segment, the direction of soft tissue pull, and the line of correction of any adjustive thrust (L. Baptista, class handout, February 2018). AR consists of the integration of virtual resources with real-world physical elements, in which computer-generated video excerpts are presented on users' electronic devices along with real environment elements.¹¹  In short, students hold their mobile phones over a particular page of their notes and a video of related content automatically plays. Demonstration learning is a great method of learning for teaching procedural skills in an engaging manner, allowing enhanced retention and reproducibility in learners.¹² 

Previous educational research into AR suggests AR does increase content understanding and knowledge retention,^13,14  supports collaborative learning,¹⁵  and helps with student motivation to learn.¹⁶  However, there are challenges in that AR itself is a new skill to learn¹⁷  and feeds into the ongoing debate as to whether technology is nothing other than a vehicle for information or a unique way to enrich a learning environment.^18,19  Research suggests that collaborative learning environments, like chiropractic technique teaching environments, help to reduce the cognitive loading from the extra information AR imparts.²⁰  Furthermore, a recent meta-analysis of the use of AR suggests that in a collaborative teaching and learning environment,²¹  especially for science-based learning, AR has a positive impact (d = .72) on learning.²⁰ 

If use of AR does improve student recall, this may provide the grounds to incorporate AR into daily teaching. Investigating new teaching and learning strategies is of particular importance, as clinical educators continually question the effectiveness of their teaching strategies and strive to improve students' learning environments. This study will give insight into how effective an AR tool can be and is part of an ongoing chiropractic education quality assurance program.

The aims of this study were to investigate if class handouts with an AR component improve student retention (and student experience) of specialized chiropractic diversified technique knowledge compared to a written guide. It also captures participants' impressions of the study in a poststudy questionnaire. The secondary aim of this study is to provide evidence of the use of AR in a teaching environment to inform budgeting or purchasing of AR technology or software.

A parallel group randomized controlled trial design was used for this study to test the effectiveness of 2 study aids for diversified adjusting technique recall. An AR guide was designed to help chiropractic students learn about diversified adjusting technique. To see if the AR guide enhanced memory and recall it used a combination of video and written word. This AR guide was tested against the current written guide used in the diversified course.

Students were asked to complete a free recall questionnaire for 3 diversified adjustment listings, as delineated in the outcomes section below, over 10 minutes. The time allowed to fill out the free recall questionnaires (10 minutes) was used, as it mirrored the task and time allotted in practical examinations. Students were then sequentially presented with the guides for each listing and were allowed to study the notes for 3 minutes each. Guides were then removed from students, and they were asked to fill out a second copy of the same free recall questionnaires. Students were again given 10 minutes to refill out the forms. Finally, students were asked to fill out a satisfaction survey. Figure 1 is a flowchart of the study.

Undergraduate chiropractic students who had previously completed (and passed) their diversified course were invited to participate in this study. This course is taught during the first year of the college curriculum. Only students from the year 2 group at the college were invited to attend as they were the most recent cohort to have completed the diversified technique course. All students spoke English as an entry requirement to the college.

The study was reviewed and approved by the New Zealand College of Chiropractic research committee. The Health and Disabilities Ethics Committee of New Zealand exempted this project from further ethics review, as this data collection was performed as part of course review processes.

The students were given 9 minutes to study either the original written teaching notes regarding diversified adjusting or the visual guides created for this study. The free-recall test was repeated after the 9 minutes of diversified adjustment study with the AR guide or written guide. The study period was used as it mirrors the length of reading time given for reviewing class notes before engaging in learning new motor skills.

Each student was asked to complete a free-recall questionnaire to test their knowledge of diversified adjusting (Appendix 1). They were allowed 10 minutes to complete the questionnaire. The document included responses for doctor stance, patient position, doctor contact, tissue pull, line of drive, and thrust required.²²  The questionnaire and time allotted were based on the practical examination marking guides for the diversified course that has been run for several years (Appendix 2). After the second free-recall questionnaire, participants were asked to fill out a poststudy questionnaire to capture their impressions of the study. Section 1 of the poststudy questionnaire included 10 questions. Responses for questions 1–8 were selected from a 6-point Likert scale to capture participants' impressions of the study (1 represented totally disagree, 5 represented totally agree. Questions 9 and 10 asked students' opinions about the aspect of the notes for which they were the most and least satisfied. The entire questionnaire is provided in Appendix 3.

Sample size calculations were performed using the R software²³  package pwr (R Foundation for Statistical Computing, Vienna, Austria).²⁴  Sample size calculations were based on a previous study relating to visual and written learning guides for memory retention. Improvement was defined as the mean improvement being 17.7%.⁹  To detect a true difference, a minimum sample size of 20 was recommended to reject the null hypothesis that both groups were equal with probability (power) of 0.8. The type 1 probability associated with the test of the null hypothesis was 0.05. To manage relative uncertainty about the effect sizes and to accommodate possible dropouts the minimum sample size was increased to 25.²⁵ 

Participants were randomly divided into 2 groups via a single application of the rock-paper-scissors game. The rock-paper-scissors game has been shown to be a useful tool for randomization of decision making, when only a single round is played.^26,27 

Data analyses were blinded by recording data with an associated participant identification code, then replacing the participant code with an unrelated alphanumeric code before analysis, thereby removing any associated participant identifiers.

Data collected from the participants were evaluated prereview and postreview of each guide. Descriptive statistics (unadjusted means, medians, standard deviations, and counts) were used to describe the characteristics of the study sample. The Shapiro-Wilk test was used to assess data from independent groups for normality. Wilcoxon-Mann-Whitney and t tests, for nonparametric and parametric data, respectively, were used to compare group differences in reevaluation scores. R software was used for all analyses.²⁸  Statistical significance for all tests was set at p < .05.

Thirty-eight students agreed to participate. The participants were randomized to review either a written guide (n = 18) or the AR teaching aid (n = 20). Not all participants completed all free-recall questionnaires as some participants ran out of time. Only 12 participants who studied the written guide and 7 who studied the AR guide completed the questionnaire for the PI ilium subluxation listing. Due to the number of incomplete questionnaires for the PI ilium listing, this questionnaire was removed before analysis. Table 1 shows the sample characteristics.

Shapiro-Wilk tests showed the C7 listing data violated normality for the AR (p = .02) and written guides (p = .01). However, data for the T6 listing was distributed normally (AR: p = .74; written guide: p = .57). Summary data for the listings are shown in Table 2.

A Wilcoxon-Mann-Whitney test showed that there was no significant difference (p = .088) between use of AR or written guide test scores for the C7 listing (Fig. 2). The median score for change in AR guide score was 0.75 points and the median score for change in written guide score was 3.0 points.

An independent-samples t test was conducted to compare difference in scores for the T6 listing when using AR or the written guide (Fig. 3). There was no significant difference in scores for the AR guide (M = −0.98, SD = 1.58) and the written guide (M = −0.92, SD = 1.50) conditions (p = .908).

All participants completed the poststudy questionnaire. Table 3 presents their responses. Overall responses were positive regarding the study task and its explanation, with 97.1% of participants in complete agreement that the study tasks were easy to understand, realistic to perform, and clearly explained. Results regarding AR and the written guide being straightforward and clear were similar, with 63.2% of participants in complete agreement. However, more participants (63.2%) fully agreed that the written guide gave more understanding of the nuances of the technique compared to the AR guide (55.3%).

A qualitative descriptive content analysis of participants' responses^29–31  for question 9 (“Please describe what aspect of the diversified notes have you been most satisfied with in this study?”) and 10 (“Please describe what aspect of the diversified notes have you been least satisfied with in this study?”) revealed a number of themes³²  running through participants' responses. Themes suggested are shown in Table 4.

Written guides were reported as being easier to read and more detailed when studying diversified theory (22.9%), while others (26.4%) thought the AR guide was more nuanced and engaging (10.5%). When asked if the written guides were sufficient when learning how to practically apply diversified technique, 56% of participants said the written guides were not detailed enough, 12% said it was difficult to visualize long complex written instructions, and 32% reported the wording could be confusing or inconsistent.

This study aimed to determine if a written or AR guide for diversified technique influences learning retention when teaching manual motor skills. Students' free recall of the C7 listing improved after using either the written or AR guide, and there were no significant differences in performance between the guides. However, free recall worsened in the AR and written guide for the T6 listing. These results suggest that the type of guide does not influence free recall, suggesting it is the act of study that affects recall. Taken together, these results suggest that there is no advantage to using AR over a written guide for free recall of information regarding C7 or T6 listing parameters.

The results of these analyses seem to support the argument that technology is just a vehicle for information rather than an improvement in pedagogy.¹⁹  However, meta-analyses indicate that a longer period of using AR might be needed before analyzing the effects AR has on learning.²⁰  The results also suggest that although 10 minutes may be an appropriate time for a student to complete a similar task when in a high-stress, high-stakes environment such as in examinations, it may not be enough time for a study such as this. Hence, the findings of a recall decline in the T6 task and removal of the PI ilium task. Any future study that uses AR might benefit from a longer intervention before analysis. Different analyses also could be used to include metrics of understanding, student motivation, and students' ability to use the technology.^13,16,17 

The poststudy questionnaire did reveal some useful findings that will allow educators to adapt and improve the diversified technique written guides that are currently used in teaching the course. Teaching diversified adjustment setups (patient position, doctor position, patient segmental contact, doctor contact, line of drive) is done in a collaborative, interactive way, but students often try to learn components by rote learning.³³  Students can lack the ability to remember the nuances of every adjusting setup after recalling through written notes. Such nuances are not easily translated into text; thus, many minor details are often missed. The poststudy questionnaire findings revealed that while written guides seem to be useful when teaching the theory of the diversified course, they have some limitations when attempting to teach the practical components of the course. These findings suggest adding more detail to the written guides may improve practical implementation. Furthermore, breaking down the written guides into more manageable chunks to allow students to visualize the practical components in discrete, easy stepwise components also may improve guides. Lastly, reviewing written guides in great detail to eliminate inconsistencies or confusion in language or layout also may improve students' learning or retention.

Limitations of this study include the type of randomization that was used and future studies should use a blockchain randomizing technique to ensure reliable randomization.³⁴  Furthermore, the AR used in this study is augmented video, but it is only 1 of the many types of AR tools available in education population.³⁵  The sample size was also small. Recommendations for future research could include investigating different types of AR or different styles of written guides when teaching complex, nuanced information such as that taught in the diversified technique course.

This study showed no difference in free recall after the use of a written guide or AR guide for diversified techniques listings. However, it did uncover some potentially useful strategies to improve the currently used written diversified technique guides. Further studies could be performed with a larger sample size and comparing written guides that have been adapted as per the findings of the poststudy questionnaire.