Objective

To investigate the influences of feedback on manual therapy skill acquisition as presented in the literature.

Data Source(s)

An electronic search was conducted across 4 databases: PubMed, EBSCOhost, SPORTDiscus, and CINAHL. The key words that were used in the search included manual therapy, physiotherapy, mobilizations, manipulation, education, instruction, feedback, intrinsic feedback, and extrinsic feedback. The Boolean phrases AND and OR were used to combine the search terms.

Study Selections

Studies that collected outcomes related to manual therapy skill acquisition from inception of the databases to September 2019 were included. Studies were excluded if they examined solely patient-rated or clinical outcomes of manual therapy or did not use feedback as the primary instructional intervention.

Data Synthesis

After quality appraisal with the Joanna Briggs Institute Critical Appraisal Checklist for Quasi-Experimental Studies, the articles included in the review were categorized according to generalized manual therapy skills. Joint distraction/traction was the skill examined in 2 studies. The effect of feedback on joint mobilizations was investigated in 5 studies. Studies examining joint manipulations represented the largest portion of the articles in this review, with 11 total studies being included. The primary forms of feedback that were examined in the literature included visual, verbal, and combined forms of auditory and visual feedback.

Conclusion(s)

Visual feedback that provides learners with graphical representations of their performance, such as force-time relationships, appear to have the greatest effects in improving force-related parameters. Visual feedback can be useful during the initial acquisition phases of manual therapy skills, as indicated by the concentration of significant findings immediately after use in training sessions. A limited number of studies examining outcomes at long-term follow-up reported that training effects decrease rapidly over time. Thus, future studies should investigate if optimal dosages or scheduling strategies exist to increase the retention of effects.

KEY POINTS

  • Real-time visual feedback in the form of graphical representations of a learner's kinematic parameters, such as force-time profiles, can produce short-term improvements in force-related parameters.

  • Novice learners should be provided regular and frequent feedback in the earlier stages of skill acquisition.

  • Educators should consider intentionally and progressively removing feedback once learners begin to demonstrate some degree of autonomy in a skill.

INTRODUCTION

Manual therapy is a broad term that is used to describe any technique that requires the skillful use of a practitioner's hands to induce therapeutic effects.1  Similar to other psychomotor tasks commonly used by health care professionals, manual therapy skills require learners to engage in deliberate practice in order to become competent.2  Although the use of manual therapy continues to increase in physical and sports medicine, it appears that the consistency of these skills is limited.3  Without adequate consistency, patient outcomes can be compromised because of the discrepancy of technique application among clinicians, or because of the inability to elicit the desired effects of a manual therapy technique.

Issues related to the limited consistency of manual therapy skills are most apparent in the literature regarding joint mobilizations.36  According to a series of studies published by Snodgrass et al,3,4  interclinician reliability of cervical spine mobilizations was found to be poor, with intraclass coefficients ranging generally from 0.17 to 0.58. Similar results were reported with mobilizations in other joints of the body.5,6  The uniformity of these results raises concerns in the current pedagogical strategies used to teach joint mobilizations and other forms of manual therapy to learners.

It is typical for manual therapy instruction to involve some form of demonstration, which is then followed by a period of student practice for which an instructor provides feedback.7  Feedback is regularly used to teach hands-on skills, and is a critical component in the preparation of health care professionals.8  As an instructional intervention, feedback can be defined as descriptive information of performance or understanding within a learner.9  The literature demonstrates that feedback can have significant impacts in learning and is particularly beneficial in the learning of technical skills.10  Feedback can benefit learners by increasing knowledge, facilitating accurate self-assessments, and providing a guide toward refinement of the task being performed.9 

The positive effects of feedback in the performance of psychomotor tasks within the health care professions have been shown in a variety of different skills. In the prehospital and ambulatory care settings, the effect of feedback in improving the depth and rate of chest compressions during cardiopulmonary resuscitation has been documented.11,12  Similar findings were demonstrated when feedback was provided to learners of surgical skills.13  Reiley et al13  examined the effect of visual force feedback in suture-tying tasks during robot-assisted surgery and found decreased suture breakage rates, lower peak force in knot applications, and increased consistency. Despite these positive influences on psychomotor skill acquisition, a more detailed evaluation of feedback shows that not all forms can bring about positive outcomes in all learners.9  Particularly, any medium or method of feedback that undermines the intrinsic motivation of the learners or fails to provide relevant information about a learner's performance has been shown to produce negative effects.9 

Other factors that may potentially contribute to the effectiveness of feedback include frequency and scheduling.14,15  Feedback that is provided during the practice trial is defined as concurrent feedback, whereas feedback that is provided after a performance is defined as terminal or delayed feedback.14  The literature is not exactly clear on which form of feedback is most beneficial, but the available evidence suggests that the previous experience of a learner may play a role in the effectiveness of timing.15  Novice learners of a skill may benefit from frequent concurrent feedback, as this form may help with facilitating learners towards better understanding a specific movement pattern.15  It is not exactly clear if this benefit to novice learners is a direct result of the scheduling (ie, terminal or concurrent) or the frequency of feedback.

Motor-learning theorists generally accept that there are 2 broad categories of feedback in skill acquisition: intrinsic and extrinsic feedback.14  Intrinsic feedback is understood as the ever-present sensory processing that occurs involuntarily with movement in individuals. Extrinsic feedback, also known as augmented or external feedback, is defined as information provided by external sources regarding a performed task.14  Extrinsic forms of feedback can be further categorized as auditory, visual, haptic, or mixed feedback. Although the literature regarding feedback and manual therapy acquisition focuses largely on extrinsic feedback, navigating the literature is difficult considering the rather heterogeneous nature of methodology used. Thus, the purpose of this systematic review is to investigate the effects of feedback on outcomes related to manual therapy skill acquisition as presented in the literature.

METHODS

Search Strategy

A systematic search of the literature was performed across 4 electronic databases to identify articles that examined the effect of feedback on outcomes related to manual therapy skill acquisition. The databases that were used were CINAHL, SPORTDiscus, PubMed, and EBSCOhost. Databases were searched from inception to September 2019. A combination of key words related to the research question was used to search the electronic databases along with the Boolean operators OR and AND (Table 1). The search was restricted to human studies research and manuscripts available in English.

Table 1

Search Terms and Articles Identified in Initial Search

Search Terms and Articles Identified in Initial Search
Search Terms and Articles Identified in Initial Search

Selection Criteria

The articles identified from the systematic search were screened for the inclusion and exclusion criteria. Titles and abstracts were screened by 2 investigators (I.C. and M.J.R.), with the full-text manuscript being assessed if the eligibility could not be determined initially. A third investigator (L.E.E.) was brought into the screening process to resolve disagreements between the 2 authors regarding the eligibility of articles.

Inclusion Criteria

The following criteria were used to determine if articles met the eligibility for this review: (1) the article examined the effect of feedback in outcomes related to manual therapy skill acquisition, (2) the article involved human participants, and (3) the article was a peer-reviewed, full-text publication.

Exclusion Criteria

The following criteria were used to exclude articles from this review: (1) the article examined patient outcomes only in patients receiving manual therapy, (2) the article was not published in English, and (3) the article did not assess the effect of feedback as an intervention in the outcomes of manual therapy skill acquisition.

Methodologic Quality

The Joanna Briggs Institute's Critical Appraisal Checklist for Quasi-Experimental Studies16  was used to appraise the quality of the studies that were included in the review. This appraisal tool is composed of 9 items, with each item being scored yes, no, unclear, or not applicable. Two independent investigators (I.C. and M.J.R.) scored each article that was eligible for inclusion. For each yes score, 1 point was given, whereas each no or unclear was given 0 points. When disagreements between scores arose, the reviewers met to discuss and come to a consensus. A cutoff summary score of 5 of 9 (55%) was used to determine if an article was appropriate for inclusion. Studies with summary scores below the cutoff value were deemed to be too low in methodological quality, necessitating exclusion.

Data Extraction

The data that were extracted included study aims, study design, inclusion criteria, participant characteristics, feedback types, statistical analyses, collected outcomes, conclusions, and relevant limitations from the articles remaining after the initial screening process. The included studies were categorized by types of feedback that were used. Studies that examined the effect of feedback provided by an external agent (ie, another individual or computer/screen) was categorized as extrinsic feedback. Intrinsic feedback studies were those that examined the effect of self-reflection and assessment while learning skills.

RESULTS

Search Results

The searches of the electronic databases yielded 1505 articles (Figure). After the removal of duplicate studies and the completion of the initial screening of titles and/or abstracts, 19 articles were deemed eligible for inclusion. The appraisal for methodological quality was then completed using the Joanna Briggs Institute's Critical Appraisal Checklist for Quasi-Experimental Studies, with 1 article being removed from review for not meeting the cutoff summary score of 5/9, leaving 18 articles remaining for inclusion.

Figure

Search diagram.

Figure

Search diagram.

Appraisal

The average summary score for methodological quality of the 18 studies included for review was 7 (Table 2). The 2 most commonly missed items of the quality appraisal tool were “Reliable measurement of outcomes” and “Participants included in comparison similar.” Most studies in this review report kinetic and kinematic data as their primary outcomes. However, given that there is no gold standard in measuring this type of data across manual therapy techniques, researchers largely relied on novel methods of measure, such as instrumented tables, or used methods proposed by the existing literature. Given this variability in measurement, it is crucial that researchers explicitly state the psychometric properties of their outcome-collection methods, which 10 studies failed to mention. Five studies did not mention adequate participant data (ie, age, weight, height), bringing into question if the comparison groups were similar to the experimental groups aside from the intervention. Because most of the studies examined manual therapy techniques (ie, mobilizations and manipulations) that are force dependent, it is important to have matched physical characteristics between the control and experimental groups for appropriate comparison.17  Without matched characteristics, it would be difficult to conclude if any observed differences were a result of the feedback interventions.

Table 2

Summary Scores of the Critical Appraisal

Summary Scores of the Critical Appraisal
Summary Scores of the Critical Appraisal

Type of Manual Therapy

Distraction/Traction

Only 2 studies18,19  examined the effect of feedback on distraction or traction (Table 3). Markowski et al18  examined the effect of real-time visual feedback via diagnostic ultrasound imaging on physical therapy students' performances of knee joint traction. Although there were no differences in students' confidence in their ability to perform traction, there was significant differences in joint space changes between the control and experimental groups postintervention (P = .04; 0.04 ± 0.13 [Mean ± SD]).18  The other study examined the effect of visual feedback on the ability of 5 novice doctor of chiropractic clinicians performing an unfamiliar lumbar distraction technique.19  Using force values obtained from 5 expert clinicians as the criterion, the learning participants were provided force graphs related to their performance of the skill in real time. Although no inferential statistics and mean differences between learners and experts were provided, the authors reported mean force values closer to the criterion at posttest.

Table 3

Articles Examining the Effect of Feedback on Joint Traction or Distraction

Articles Examining the Effect of Feedback on Joint Traction or Distraction
Articles Examining the Effect of Feedback on Joint Traction or Distraction

Mobilizations

Of the studies included in this review, 5 studies specifically examined the effects of feedback on the performances of joint mobilizations in learners (Table 4).2024  The effect of combined real-time auditory and visual feedback was investigated in 3 studies, with the other 2 studies examining the effect of real-time visual feedback alone. González-Sánchez et al20  observed significant changes in ankle-joint–mobilization outcomes (ie, total time to reach max amplitude, maximum angular displacement, maximum and average velocity to reach maximum displacement, and average velocity throughout the entire mobilization) both the control group and experimental groups (P ≤ .05). However, the experimental group receiving visual real-time feedback through graphs of the applied kinematic variables performed better than the traditional learning group, as demonstrated by greater measures of internal consistency (Cronbach α = 0.899–0.984 and 0.491–0.685, respectively).20  The second study to solely examine visual feedback was completed using a joint-translation simulator to aid students in the learning of generic grade II and grade III mobilizations according to the Maitland scale.21  Researchers also examined the effect of timing of the visual-feedback group, with one group receiving feedback concurrently with performance whereas the other received feedback at the end of their performance in each trial. Compared with the control group that did not receive visual feedback, the feedback groups both demonstrated similar significant posttest decreases in normalized error (P < .001).21  These findings were sustained through a retention test performed 5 days after posttesting.

Table 4

Articles Investigating the Effect of Feedback on Mobilization Performance

Articles Investigating the Effect of Feedback on Mobilization Performance
Articles Investigating the Effect of Feedback on Mobilization Performance

Examining the impacts on lumbar spine mobilizations in physiotherapy students, Snodgrass and Odelli22  saw the combined effects of visual and auditory feedback in improving peak force, force-amplitude accuracy, and oscillation-frequency accuracy (P < .001). However, significance was lost when outcomes were collected at 1-week follow-up. Similar results were also reported in a study examining the combined effect of auditory and visual feedback in the learning of cervical spine mobilizations.24  Although improved force application parameters were noted immediately after intervention (P < .001), the accuracy and the magnitude of the applied forces decreased significantly after 1 week (P = .008). Sheaves et al23  reported similar results, while also evaluating the role of frequency of this specific form of feedback. Students were placed into 1 of 3 frequency groups: intermittent, constant, and self-controlled. Those who were placed in the self-controlled feedback group chose when they received feedback; other groups received feedback for all trials or received feedback for only a third of their trials. With no significant differences between the intermittent and constant feedback groups, the self-control feedback group demonstrated longer retention effects, as some significance was maintained until follow-up approximately a week after posttesting.

Manipulations

Representing the largest proportion of the available literature, 11 studies examined the influence of feedback in performances of manipulations (Table 5).2536  Unlike the results reported by the literature for other manual therapy skills, the effects of feedback on manipulation is rather heterogeneous. Triano et al2528  released a series of studies from 2002 to 2014 examining the effect of visual feedback on cervical, thoracic, and lumbar manipulations using the Dynadjust Instrument (Labarge, Inc, St. Louis, MO). This apparatus provided learners with error messages when target force application was not achieved during a practice trial. Across these 4 studies, significant differences for various parameters related to force and speed of thrust were observed as a result of training with feedback (P ≤ .03). Comparable results were reported by other studies examining the impact of visual kinematic feedback, which was provided with computer graphics related to the learner's performance.

Table 5

Articles Investigating the Effect of Feedback on Manipulation Performance

Articles Investigating the Effect of Feedback on Manipulation Performance
Articles Investigating the Effect of Feedback on Manipulation Performance

Along with the changes in the force-related parameters as a result of visual feedback, there is also evidence to suggest that feedback increased the consistency and the accuracy of manipulations.29  In their study, Cuesta-Vargas and Williams29  found increased internal consistency of thoracic manipulations, with intraclass coefficients as high as 0.997 being reported. In their investigation of feedback on thoracic manipulation performances in learners, Enebo and Sherwood31  observed decreased force production error using a combination of visual and verbal feedback (P = .01). This study is unique compared with others in this review given that investigators examined the influence of practice scheduling in addition to feedback.31  Practice scheduling was manipulated by assigning students to a random or blocked practice group. Blocked practice entails the repeated performance of a specific skill, whereas random practice involves variations in when the skills is performed.15  The combined use of random practice and visual feedback produced significant results lasting into retention.31  In addition, Pasquier et al32  observed that previous student experience with a skill resulted in negligible differences when feedback was used in training.

Contrasting to other studies, Lardon et al33  observed no significant effect of feedback or the frequency of feedback in the performance of thoracic manipulations. Watson and Radwan34  also examined if student performance of thoracic manipulations would be affected by the timing of feedback. Feedback was provided during (concurrent) or after (delayed) the student trial verbally, with the control group receiving no feedback at all. No significant results were reported during skill acquisition, but the authors concluded that concurrent feedback provided the best results in retention despite not reporting statistical significance.

DISCUSSION

The purpose of this systematic review was to investigate the effect of feedback on outcomes of manual therapy skill acquisition as presented by the available literature. Most of studies examined manipulations as the primary manual therapy skill of interest, with combined extrinsic forms of feedback being the most commonly assessed within individual studies. There were no studies that examined the impact of intrinsic feedback. The dearth of studies examining intrinsic feedback could be due to the nature of this feedback being related to the sensory processing that occurs within an individual, making this a very difficult concept to study. Despite this rather large grouping of studies, the literature is still rather heterogeneous in the skills, feedback forms, and the outcomes that are collected.

Visual feedback seems to be an effective adjunct for teaching skills that rely on appropriate application of force-related parameters such as joint mobilizations and manipulations. This is consistent with the literature regarding procedural skills in other health care fields, such as cardiopulmonary resuscitation.11,12  Although not all aspects related to the skill are expected to improve, the evidence suggests that visual feedback can significantly influence the learner's application of force and/or the associated velocity with manipulations.2533,36  Although the literature is not as saturated in studies examining mobilizations, it is suggested that visual feedback may also result in improved outcomes for mobilizations in learners. In both mobilizations and manipulations, the greatest effects were found immediately after feedback; most studies that include retention assessments demonstrated a decline of significant findings with time.22,23  This contrasts with the feedback literature on cardiopulmonary resuscitation, as skill retention remained for some time. However, Miller et al36 , and Wik et al37  assessed cardiopulmonary resuscitation skill retention after allowing as many practice sessions with feedback as necessary until learners felt completely confident in their ability to perform the skill. The number of practice session were predetermined in the manual therapy feedback literature. It is possible that the number of practice sessions provided in these studies was not enough to instill confidence within learners to enable retention. Given that the results of visual feedback are optimal immediately postintervention, this form of feedback is most appropriate during the initial phases of skill acquisition.

Previous literature38  examining the guidance hypothesis for feedback concurs with this scheduling of feedback for learning motor skills. This learning theory suggests that feedback provided to the learner early on, in more regularly scheduled bouts, improves the retention of skill performance over 24 hours.38  Lastly, previous literature38  also found that gradually reducing the amount of feedback improved immediate skill performance as well as performance after a follow-up period during a test trial once the feedback was removed. This may help educators who are instructing students in manual therapy skill acquisition by structuring their feedback more regularly in early practice and gradually removing feedback as learners become more proficient and autonomous in the skill.

Although many of the studies examining visual feedback also combined other feedback forms such as verbal feedback specifically with manipulations, the limited number of studies and the paucity of convincing results seem to indicate that verbal feedback does not play a large role in influencing outcomes. In these studies, verbal feedback was provided through qualitative knowledge of a learner's performance of skills by the observing instructors.31,35,36  With no studies available that specifically examined the effect of auditory feedback aside from verbal forms, conclusions cannot be made on the exact impact of this form of feedback when combined with visual aids. More studies are needed to examine the exact influence of auditory feedback.

Only 2 investigations examined the effect of feedback on joint traction/distraction, which limits the ability to make strong conclusions regarding the usefulness of feedback on learning this specific type of manual therapy. However, the preliminary results show that visual feedback can potentially have positive effects.

A majority of the studies that examined visual feedback in the form of graphical representations of the kinematic and kinetic variables occurred as the learner was performing the skill. It is difficult to conclude if the timing of visual feedback would influence these outcomes given that no studies have assessed timing. With only a singular study explicitly stating that concurrent visual feedback is effective, this conclusion should be taken with extreme caution as no statistical significance was reported in support of this statement.

Traditional motor-learning theories state that the provision of feedback should be modified as learners progress through the phases of skill acquisition for optimal results. The guidance hypothesis, proposed by the motor theorist Richard Schmidt38,39  articulates the idea of frequent feedback leading to improved performance. However, a theoretical dependency on the feedback can also develop, potentially hindering performance once the feedback mechanism is removed.38,39  With only 2 studies investigating this specific variable of feedback, more studies are needed to confirm if this phenomenon exists when applied to the acquisition of manual therapy skills. If this phenomenon is confirmed to occur with manual therapy skill learning, then future studies should aim to investigate the optimal dosing and scheduling of feedback to best benefit learners without forming dependency.

Potential future investigations should also examine the interaction between feedback and focus of attention in the learning outcomes of manual therapy skills. The motor-learning literature is largely supportive of an external focus being more beneficial than an internal focus during skill acquisition.40  Feedback that describes or provides information about a learner's body movements are framed with an internal focus, whereas externally focused feedback forms allow for a student to concentrate on the effects produced as a result of the performed task.40  Emphasis on extrinsic focused feedback during instruction is said to have the benefits of facilitating the automaticity of movement, thereby possibly allowing for increases in movement accuracy and faster rates of learning.40  If found to be effective in manual therapy skill acquisition, extrinsic focused feedback can potentially be a useful instructional technique to improve the outcomes of manual therapy learners in the absence of methods to provide visual feedback.

Limitations

This review does have some limitations given the mixed nature of the methods, outcomes, and analyses used by the included articles. Therefore, clear conclusions and recommendations are difficult to synthesize. Another limitation is the search strategy used for this review as hand searches of reference lists were not performed, thereby potentially excluding relevant studies.

Educational Implications

Given the heterogeneity of the literature in the methodologies, manual therapy skills, and forms of feedback studied, strong recommendations on how to best incorporate feedback in the teaching of manual therapy are difficult to make. With a rapid decline in significance in the studies examining long-term retention,22,23  the effects of visual forms of feedback appear to be largely short-term. There are no studies that provide evidence on how to sustain effects into long-term retention. Given this current gap in the literature, educators should consider incorporating educational and motor-learning theories, such as the guidance hypothesis,38,39  to optimize the effects of feedback.

Clinical Bottom Line

The use of real-time visual feedback may potentially be beneficial in improving consistency and other force-related parameters, such as velocity and amplitude, in manual therapy learners. Improvements from the use of real-time visual feedback may not be uniform or predictable. With improvements appearing to be largely short-term, visual feedback may be most appropriate during the early stages of learning and skill acquisition. With the absence of strong evidence regarding other parameters of feedback, such as scheduling, frequency, and dosage, educators should consider incorporating educational and motor-learning theories.

CONCLUSIONS

Visual feedback that provides learners with graphical representations of their performance, such as force-time relationships, appear to have the greatest effects in improving force-related parameters. Visual feedback can be useful during the initial acquisition phases of manual therapy skills, as indicated by the concentration of significant findings immediately after use in training sessions. A limited number of studies examining learning outcomes at long-term follow-up reported that training effects decrease rapidly over time. Thus, future studies should investigate if optimal dosages or scheduling strategies exist to increase the retention of effects.

REFERENCES

REFERENCES
1. 
Donatelli
RA.
Physical Therapy of the Shoulder. 5th ed
.
St. Louis, MO
:
Elsevier;
2011
.
2. 
Bowley
P,
Holey
L.
Manual therapy education
.
Does e-learning have a place? Man Ther
.
2009
;
14
(6)
:
709
711
.
3. 
Snodgrass
SJ,
Rivett
DA,
Robertson
VJ,
Stojanovski
E.
A comparison of cervical spine mobilization forces applied by experienced and novice physiotherapists
.
J Orthop Sports Phys Ther
.
2010
;
40
(7)
:
392
401
.
4. 
Snodgrass
SJ,
Rivett
DA,
Robertson
VJ,
Stojanovski
E.
Cervical spine mobilisation forces applied by physiotherapy students
.
Physiotherapy
.
2010
;
96
(2)
:
120
129
.
5. 
Gorgos
KS,
Wasylyk
NT,
Lunen
BLV,
Hoch
MC.
Inter-clinician and intra-clinician reliability of force application during joint mobilization: a systematic review
.
Man Ther
.
2014
;
19
(2)
:
90
96
.
6. 
Tragord
BS,
Gill
NW,
Silvernail
JL,
Teyhen
DS,
Allison
SC.
Joint mobilization forces and therapist reliability in subjects with knee osteoarthritis
.
J Man Manip Ther
.
2013
;
21
(4)
:
196
206
.
7. 
Wise
CH,
Schenk
RJ,
Lattanzi
JB.
A model for teaching and learning spinal thrust manipulation and its effect on participant confidence in technique performance
.
J Man Manip Ther
.
2016
;
24
(3)
:
141
150
.
8. 
Shrivastava
SR,
Shrivastava
PS,
Ramasamy
J.
Effective feedback: an indispensable tool for improvement in quality of medical education
.
J Pedagog Dev
.
2014
;
4
(1)
:
13
20
.
9. 
Hattie
J,
Timperley
H.
The power of feedback
.
Rev Educ Res
.
2007
;
77
(1)
:
81
112
.
10. 
Fitzpatrick
JJ,
Carpenter
R,
Smith
MJ.
Encyclopedia of Nursing Education
.
New York, NY
:
Springer Publishing Company;
2015
.
11. 
Pozner
CN,
Almozlino
A,
Elmer
J,
Poole
S,
McNamara
D,
Barash
D.
Cardiopulmonary resuscitation feedback improves the quality of chest compression provided by hospital health care professionals
.
Am J Emerg Med
.
2011
;
29
(6)
:
618
625
.
12. 
Cheng
A,
Brown
LL,
Duff
JP,
et al.
Improving cardiopulmonary resuscitation with a CPR feedback device and refresher simulations (CPR CARES Study)
.
JAMA Pediatr
.
2015
;
169
(2)
:
137
.
13. 
Reiley
CE,
Akinbiyi
T,
Burschka
D,
Chang
DC,
Okamura
AM,
Yuh
DD.
Effects of visual force feedback on robot-assisted surgical task performance
.
J Thorac Cardiovasc Surg
.
2008
;
135
(1)
:
196
202
.
14. 
Sigrist
R,
Rauter
G,
Riener
R,
Wolf
P.
Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review
.
Psychon Bull Rev
.
2012
;
20
(1)
:
21
53
.
15. 
Akizuki
K,
Yamamoto
R,
Yamaguchi
K,
Yabuki
J,
Ohashi
Y.
The effects of feedback timing and frequency on the acquisition of cardiopulmonary resuscitation skills of health sciences undergraduate students: a 2 × 2 factorial quasi randomized study
.
PLoS One
.
2019
;
14
(7)
.
16. 
Critical Appraisal Tools
.
Joanna Briggs Institute
.
2020
.
17. 
Taylor
S,
Asmundson
GJG.
Internal and external validity in clinical research
.
In:
McKay
D,
ed.
Handbook of Research Methods in Abnormal and Clinical Psychology
.
New York, NY
:
Sage Publishing Inc;
2007
:
23
34
.
18. 
Markowski
A,
Watkins
MK,
Burnett
T,
Ho
M,
Ling
M.
Using real-time ultrasound imaging as adjunct teaching tools to enhance physical therapist students' ability and confidence to perform traction of the knee joint
.
Musculoskelet Sci Pract
.
2018
;
34
:
83
88
.
19. 
Gudavalli
MR,
Cox
JM.
Real-time force feedback during flexion-distraction procedure for low back pain: a pilot study
.
J Can Chiropr Assoc
.
2014
;
58
(2)
:
193
200
.
20. 
González-Sánchez
M,
Ruiz-Muñoz
M,
Ávila-Bolívar
AB,
Cuesta-Vargas
AI.
Kinematic real-time feedback is more effective than traditional teaching method in learning ankle joint mobilisation: a randomised controlled trial
.
BMC Med Educ
.
2016
;
16
(1)
;
261
.
21. 
Chang
JY,
Chang
GL,
Chien
CJC,
Chung
KC,
Hsu
AT.
Effectiveness of two forms of feedback on training of a joint mobilization skill by using a joint translation simulator
.
Phys Ther
.
2007
;
87
(4)
:
418
430
.
22. 
Snodgrass
SJ,
Odelli
RA.
Objective concurrent feedback on force parameters improves performance of lumbar mobilisation, but skill retention declines rapidly
.
Physiotherapy
.
2012
;
98
(1)
:
47
56
.
23. 
Sheaves
EG,
Snodgrass
SJ,
Rivett
DA.
Learning lumbar spine mobilization: the effects of frequency and self-control of feedback
.
J Orthop Sports Phys Ther
.
2012
;
42
(2)
:
114
124
.
24. 
Snodgrass
SJ,
Rivett
DA,
Robertson
VJ,
Stojanovski
E.
Real-time feedback improves accuracy of manually applied forces during cervical spine mobilisation
.
Man Ther
.
2010
;
15
(1)
:
19
25
.
25. 
Triano
JJ,
Rogers
CM,
Combs
S,
Potts
D,
Sorrels
K.
Developing skilled performance of lumbar spine manipulation
.
J Manipulative Physiol Ther
.
2002
;
25
(6)
:
353
361
.
26. 
Triano
JJ,
Rogers
CM,
Combs
S,
Potts
D,
Sorrels
K.
Quantitative feedback versus standard training for cervical and thoracic manipulation
.
J Manipulative Physiol Ther
2003
;
26
(3)
:
131
138
.
27. 
Triano
JJ,
Scaringe
J,
Bougie
J,
Rogers
C.
Effects of visual feedback on manipulation performance and patient ratings
.
J Manipulative Physiol Ther
.
2006
;
29
(5)
:
378
385
.
28. 
Triano
JJ,
McGregor
M,
Dinulos
M,
Tran
S.
Staging the use of teaching aids in the development of manipulation skill
.
Man Ther
.
2014
;
19
(3)
:
184
189
.
29. 
Cuesta-Vargas
AI,
Williams
J.
Inertial sensor real-time feedback enhances the learning of cervical spine manipulation: a prospective study
.
BMC Med Educ
.
2014
;
14
(1)
.
30. 
Cuesta-Vargas
AI,
González-Sánchez
M,
Lenfant
Y.
Inertial sensors as real-time feedback improve learning posterior-anterior thoracic manipulation: a randomized controlled trial
.
J Manipulative Physiol Ther
.
2015
;
38
(6)
:
425
433
.
31. 
Enebo
B,
Experience
Sherwood D.
and practice organization in learning a simulated high-velocity low-amplitude task
.
J Manipulative Physiol Ther
.
2005
;
28
(1)
:
33
43
.
32. 
Pasquier
M,
Cheron
C,
Dugas
C,
Lardon
A,
Descarreaux M. the effect of augmented feedback and expertise on spinal manipulation skills: an experimental study
.
J Manipulative Physiol Ther
.
2017
;
40
(6)
:
404
410
.
33. 
Lardon
A,
Cheron
C,
Pagé
I,
Dugas
C,
Descarreaux
M.
Systematic augmented feedback and dependency in spinal manipulation learning: a randomized comparative study
.
J Manipulative Physiol Ther
2016
;
39
(3)
:
185
191
.
34. 
Watson
TA,
Radwan
H.
Comparison of three teaching methods for learning spinal manipulation skill: a pilot study
.
J Man Manip Ther
.
2001
;
9
(1)
:
48
52
.
35. 
Descarreaux
M,
Dugas
C,
Lalanne
K,
Vincelette
M,
Normand
MC.
Learning spinal manipulation: the importance of augmented feedback relating to various kinetic parameters
.
Spine J
.
2006
;
6
(2)
:
138
145
.
36. 
Miller
MG,
Thompson
G,
Rice
A,
Endres
C,
Scholma
J.
The effects of visual feedback on CPR skill retention in graduate student athletic trainers
.
J Sports Med Allied Health Sci
.
2015
;
1
(2)
:
1
.
37. 
Wik
L,
Myklebust
H,
Auestad
BH,
Steen
PA.
Twelve-month retention of CPR skills with automatic correcting verbal feedback
.
Resuscitation
.
2005
;
66
(1)
:
27
30
.
38. 
Nicholson
DE,
Schmidt
RA.
Scheduling information feedback to enhance training effectiveness. In: Proceedings of the Human Factors Society Annual Meeting. Vol. 35, No. 19
.
Los Angeles, CA: Human Factors Society;
1991
:
1400
1402
.
39. 
Schmidt
RA,
Young
DE,
Swinnen
S,
Shapiro
DC.
Summary knowledge of results for skill acquisition: support for the guidance hypothesis
.
J Exp Psychol Learn Mem Cogn
.
1989
;
15
(2)
:
352
359
.
40. 
Wulf
G,
Shea
C,
Lewthwaite
R.
Motor skill learning and performance: a review of influential factors
.
Med Educ
.
2010
;
44
(1)
:
75
84
.