Can a Rescuer or Simulated Patient Accurately Assess Motion During Cervical Spine Stabilization Practice Sessions?

Although physical activity and sport are beneficial for health, some injuries sustained in sport can lead to temporary or permanent disability. In particular, spinal cord injuries that result in paraplegia or quadriplegia have devastating consequences for young people.1–5 Current estimated yearly catastrophic injury rates are 14 per 100000 National Football League players,3 6.5 to 10 per 100000 rugby athletes,2 1 per 100000 wrestlers,4 and 0.43 per 100000 basketball players.4 In addition, because health care providers must be prepared to manage all potential spine injuries as if they are unstable, most sport teams devote resources to training for sideline cervical spine (C-spine) emergencies.

When health professionals approach a patient with a suspected C-spine injury, their objective is to minimize C-spine motion in order to prevent the occurrence or exacerbation of spinal cord injury. Previous authors6,7 have shown that the lift-and-slide (L&S) technique limits C-spine motion more than does the log-roll (LR) technique. However, during both techniques, the lead rescuer has several options for stabilizing the C-spine. Two common methods for the supine patient are the head-squeeze8,9 and trap-squeeze methods.9 In brief, with the head-squeeze method, the lead rescuer simply holds the sides of the patient's head with both hands and follows the body as it moves.8 In the trap-squeeze method,9 the lead rescuer grips the patient's trapezius muscles on either side of the neck with his or her hands (thumbs anteriorly) and firmly squeezes the head between the forearms, with the forearms placed approximately at or slightly anterior or posterior to the ears.

Recently we10 studied the head-squeeze and trap-squeeze methods under different scenarios using inertial measurement units (IMUs) to measure C-spine motion objectively. The trap squeeze and head squeeze demonstrated similar efficacy for maintaining C-spine position during the L&S and LR,10 but the trap squeeze was much more effective in limiting motion when a confused patient tried to sit up (AGIT_SIT) and somewhat more effective when a confused patient tried to rotate the head (AGIT_ROT). In brief, the rescuer's forearms appeared to limit the patient's attempts to lift or rotate the head, and the rescuer's hands appeared to limit the patient's ability to raise either shoulder, thereby limiting trunk rotation.10 

In addition to the differences between the trap squeeze and the head squeeze, significant interrescuer variations strongly indicated that training is an important factor. Most training occurs without objective measures of motion; participants are evaluated only subjectively, probably because objective measures of C-spine motion require sophisticated and often expensive equipment that is not easily set up in the field. However, if subjective measures are unreliable or invalid, feasible solutions must be found. Therefore, the objectives of our study were to determine (1) how accurately rescuers and simulated patients can assess motion during C-spine stabilization practice and (2) whether providing performance feedback to rescuers influences their technique preferences. These objectives were accomplished via the following specific analyses: (1) the interrater reliability between the quality score for the rescuer (person stabilizing the C-spine during the maneuvers) and the simulated patient quality score, (2) the validity of rescuer and simulated patient scores compared with the C-spine motions that actually occurred using objective measures of motion (IMUs), and (3) the likelihood that objective feedback provided by the IMUs (in the forms of graphs and numbers showing objective measures of motion) would change rescuers' opinions about the comparative effectiveness of the different techniques. The latter analysis is important because the usefulness of objective feedback is limited if it does not convince the lead rescuer to opt for the most effective technique.

This study represents a complementary analysis of previously reported data on the relative effectiveness of the head-squeeze and trap-squeeze techniques.10 We used a crossover design: 12 lead rescuers performed C-spine stabilization techniques under different test scenarios. Two men (mass of approximately 70 kg and 77 kg) with regular neck lengths (according to the instructions for the Stiffneck Select collar; Laerdal Medical Corporation, Wappingers Falls, NY) served as simulated patients. The same simulated patient served across all scenarios for a given lead rescuer to eliminate within–lead rescuer variation. Lead rescuers were recruited from certified athletic therapists, certified athletic trainers, and physiotherapists with specific training in C-spine stabilization management, including prior knowledge of and training in both the head-squeeze and trap-squeeze stabilization techniques; they had received formal training only once, at least 6 months before the study, but they practiced the procedures frequently. The study was approved by the Ethics Review Board of the Health and Social Services Centre, University Institute of Geriatrics of Sherbrooke, and all participants gave informed consent.

The head-squeeze and trap-squeeze C-spine stabilization techniques were used in 4 test scenarios. The C-spine stabilization techniques and test scenarios were performed according to accepted recommendations followed by lead rescuers in their regular work.9,11 For the 6-person L&S, the lead rescuer was at the patient's head, 2 assistant rescuers were on either side, and an additional assistant rescuer placed the spine board (Dyna MED, Lexington, KY) under the patient, who was cooperative and wearing a Stiffneck Select collar. For the 5-person LR,7 the lead rescuer was at the patient's head, 3 assistant rescuers were at the side, and an additional assistant rescuer placed the spine board in position under the patient, who was cooperative and wearing a Stiffneck Select collar. To minimize within-lead rescuer variability, each lead rescuer worked with the same assistants for each scenario.

We also simulated C-spine stabilization in a confused patient who attempts to sit up (AGIT_SIT) or a patient who rotates his or her head (AGIT_ROT). For these scenarios, the sequence of movements (AGIT_ROT or AGIT_SIT) was randomized (ie, unknown to the lead rescuer) to ensure that the lead rescuer could not predict the patient's movement. In brief, the patient initially lay quiet and supine with the head and neck in the neutral position. The patient suddenly tried to sit up on the left or right side or rotate the head to the left or right side. The lead rescuer attempted to maintain the head in the neutral position without flexion or rotation in the sagittal, coronal, or transverse planes. During the head-squeeze technique for the AGIT_SIT maneuver, the lead rescuer limited motion by trying to match the patient's head movement to body movement. During the trap-squeeze maneuver for the AGIT_SIT, the lead rescuer firmly held the patient's head in place with the forearms and held the shoulders down by applying pressure to the anterior trapezius muscle or clavicle with the thumbs.

We conducted 5 trials each for the head squeeze and the trap squeeze in each condition (L&S, LR, AGIT_ROT, AGIT_SIT). Lead rescuers were allowed practice trials before each test scenario until they felt comfortable. For the L&S and LR, the practice trials were conducted with the same team of trained people acting as assistants as during the experimental recording. The order of the C-spine stabilization technique (head squeeze, trap squeeze) for each test scenario (L&S, LR, AGIT_SIT, AGIT_ROT) was counterbalanced across participants to minimize confounding due to fatigue. The order of L&S and LR was counterbalanced before AGIT_SIT and AGIT_ROT trials. Some rescuers completed all trials in 1 session (approximately 15 minutes for the agitated patient scenarios and approximately 30–45 minutes for the L&S and LR scenarios), whereas others participated in 2 sessions (L&S and LR in 1 session, AGIT_SIT and AGIT_ROT in a separate session).

After each trial for each scenario (L&S, LR, AGIT_SIT, AGIT_ROT) with each method (head squeeze, trap squeeze), the lead rescuer and simulated patient verbally provided an overall score between 0 and 10 (0 = best, 10 = worst) of how well the C-spine was stabilized during the trial. For logistical reasons, the rescuer and simulated patient were not blinded to each other's score.

We estimated C-spine motion during stabilization techniques in the sagittal plane (flexion, extension), coronal plane (right lateral flexion, left lateral flexion), and transverse plane (right rotation, left rotation) using IMUs attached to the forehead and trunk of the patient. Electronic devices with sensors (usually accelerometers, gyroscopes, and magnetometers), IMUs measure the orientation of an object or a body segment. (For a more detailed explanation of the principles behind IMUs, see Luinge and Veltink,12 Giansanti et al,13 and Giansanti and Maccioni.14) In brief, IMUs generally use a 9-axis sensor module, which contains 3 orthogonally mounted triads of angular rate sensors, accelerometers, and magnetometers. Angular orientation is determined by integrating the output from the angular rate sensors. The accelerometers measure the gravity vector relative to the coordinate frame of the sensor module. The magnetometers serve a similar function for the local magnetic field vector. Using these sensor signals and different signal-processing techniques, it is possible to estimate the orientation of the module with different degrees of accuracy. The accuracy of this estimation varies with the type of sensors used, the motion being tracked, the environmental presence of ferrous objects, and the performance of the fusion algorithm used. In this study, we used MotionPod IMUs (Movea SA, Grenoble, France), which incorporate a 3-dimensional accel-erometer and a 3-dimensional magnetometer. Sensor data were sampled at 100 Hz, converted from analog to digital on the module, and transmitted wirelessly to a computer using proprietary software (MotionDevTool; Movea SA). Median and low-pass filtered (10 Hz) orientation data (Euler angles for each module) were used to compute the relative motion of 1 module with respect to the other in all planes using MATLAB (Math-Works, Natick, MA). We tested the accuracy of the angles measured by the MotionPod using standard procedures with an optoelectronic motion capture system (Optotrak Certus; Northern Digital, Inc, Waterloo, ON, Canada). In brief, we fixed the MotionPod module to the center of a plastic rig, and the module's reference system was set to be the same as the rig reference system. Marker data and IMU data were collected at 100 Hz and synchronized in postprocessing using cross-correlation techniques. The accuracy of a single module was less than 0.6° in all directions under static test conditions (moving the rig from 0° to 90° in 5° increments). For dynamic conditions (rotating the rig around each orthogonal axis for 30 seconds) at approximately 60°/s, the accuracy was less than 0.51° for flexion-extension and rotation (sagittal and transverse planes) and less than 4.16° for lateral flexion (coronal plane). The lower dynamic accuracy for one of the planes of motion on the IMU (lateral flexion in the reference system) is well documented15–16 and occurs because of the IMU algorithm's reliance on magnetometer data for computer heading. Finally, we wrote software to provide rescuers with feedback on the motion in each direction immediately after they completed the study. Feedback was provided using graphs to illustrate C-spine motion for each trial and summary estimates of motion and variability of motion across all trials for a given scenario (Figure 1).

The quality of C-spine stabilization depends on limiting motion in every direction; excessive motion in even a single direction constitutes a failed trial. Therefore, we created a composite quality score from the IMU data for each trial. Because some movement is inevitable, we used mean values from the literature6 and our clinical judgment of what is reasonable to expect in terms of C-spine stabilization performance. We categorized each trial as a success if all objective motions were 5.0° or less, as a failure if any motion was greater than 10.0°, and as a partial success if at least 1 motion was between 5.1° and 10.0° but no motion exceeded 10.0°. As a sensitivity analysis, we also used categories of 10.0° or less, 10.1° to 20.0°, and greater than 20.0°.

In further exploratory analyses, we used principal component analysis (PCA) to create an overall index of motion based on the IMU data of motion (flexion, extension, maximum rotation, maximum lateral flexion). The PCA correlates variables into components and gives a single score for the correlated variables.17 In our context, the single score provided an overall index of motion. For simplicity, we combined the motions from all scenarios but also conducted a sensitivity analysis in which the PCA was performed on each scenario separately.

Before the study, each lead rescuer was asked whether (and why) he or she preferred the head squeeze or trap squeeze to stabilize the C-spine during the L&S, LR, and scenario of a confused patient. The same questions were asked after the lead rescuer completed the study and again after we provided the numerical results for C-spine motion measured with the IMUs (Figure 1).

We used the weighted κ to assess the lead rescuer–simulated patient scores for reliability of our ordinal data. We also report on the frequency of qualitative discrepancies between raters.

To determine the validity of the lead rescuers' composite quality scores, we first compared them with composite objective quality scores based on IMU data. As an exploratory post hoc analysis, we asked whether trials that were subjectively rated as near excellent by both lead rescuer and simulated patient score were indeed well conducted. We categorized each trial as excellent if the lead rescuer and simulated patient scores were both 0 or 1, as poor if either score was 4 or higher, and unsure otherwise (eg, lead rescuer scored 2, simulated patient scored 0). We then compared each trial composite categorization with the corresponding IMU trial categorization (success, partial success, or failure). In further exploratory analyses, we calculated the correlation between the lead rescuers' subjective scores and the overall index of motion (PCA method) using linear regression. In these analyses, we stratified by scenario because it is likely that both the lead rescuers' and simulated patients' subjective scores are based on relative motion. For example, the L&S results in less motion than does the LR. Some extra motion during the L&S might constitute a poorly conducted trial, even though the absolute motion was much less than during a well-conducted LR trial.

For preferences, we had only 12 lead rescuers for each scenario, and therefore we simply describe the absolute number of rescuers who changed their opinions. All analyses were carried out via open-source statistical software (version 2.9.1 R Statistical Package; Institute for Statistics and Mathematics, Vienna University of Economics and Business, Vienna, Austria).

All lead rescuers were certified as emergency responders and had professional certification as an athletic therapist, athletic trainer, or physiotherapist (Table 1). Their C-spine trauma management experience ranged from 4 to 42 years, and they were more familiar with the head-squeeze than the trap-squeeze technique. Of the 480 trials conducted, 25 had to be dropped because of large regions of missing data (because of problems with wireless data transmission) that could not be reliably imputed, leaving 455 trials for analysis. However, at least 4 trials for each context were acceptable in 11 of 12 rescuers performing the trap squeeze and 10 of 12 rescuers performing the head squeeze.

Although the lead rescuer-simulated patient reliability was acceptable for both of the simulated patients (κ_w=0.71 and 0.74), absolute variation between the scores was considerable (Figure 2). At each level of simulated patient score, the lead rescuer's score was up to 3 points different on a scale of 10 (eg, 4 simulated patients scored 3 when the lead rescuer scored 0). In addition, the lead rescuer and simulated patient scored 2 or more points apart in 10% to 15% of the trials. For example, of the 109 trials in which the lead rescuer scored 1 (sum of numbers in the column labeled 1), 16 trials were scored as 3 or 4 by the simulated patient (sum of rows labeled 3 and 4).

The validity of both the subjective lead rescuers' and simulated patients' evaluations was poor when compared with the objective IMU measures of motion (Figure 3). Although higher subjective scores (ie, poorer trials) were associated with a higher probability of objective failure (black bars represent a greater proportion of all trials), the proportion of correctly assessed trials was generally too low to be considered acceptable. For example, when the lead rescuer's subjective score was an almost-perfect 1 (Figure 3A), objective C-spine motion in at least 1 direction was sufficient to consider the trial a failure by objective measures 32% of the time (33 of 103). The simulated patients' assessments were similarly discrepant: 8 of 50 trials (16%) subjectively rated as a perfect 0 and 38 of 109 trials (34.8%) subjectively rated as an almost-perfect 1 (Figure 3B) were actually rated as failures according to objective measures. In our sensitivity analysis, when we increased the cutoffs from 5° to 10° for success and from 10° to 20° for failure, 15.7% of trials subjectively reported as almost excellent (0 of 10 or 1 of 10) by either lead rescuers or simulated patients were still considered failures by objective measures. In fact, only 51.4% to 55.2% of lead rescuers and simulated patients' scores of 2 of 10 were successful by objective measures.

Subjective scores were not adequate even when we denned success as a trial in which both the lead rescuer and simulated patient scored 0 or 1 (Table 2). For the L&S, of the 116 trials that were objectively successful or partially successful, 14 (12.1%) were subjectively rated as poor. In the LR, the subjective assessment was success in 13 trials, but none of these was actually successful; furthermore, 67 of 70 trials subjectively assessed as a success or partial success by both lead rescuer and patient (ie, unsure category) were also failures. Results for the confused patient scenario revealed similar problems with subjective assessments. When we used the higher cutoff levels of 10° and 20° for objective success, the results remained qualitatively similar, with 59 of 82 objective failures being rated as a success or partial success by both lead rescuer and simulated patient.

Our secondary analysis comparing the lead rescuers' subjective scores with the overall motion index calculated via PCA also showed very poor correlation, even when stratified by scenario (Figure 4). Although there was some correlation for head rotation, the r_adj² for each of the other relationships was less than 0.05. The graphs clearly illustrate that even when the L&S resulted in almost no motion, some lead rescuers and simulated patients scored trials as poorly done. For the LR and AGIT_SIT scenarios, lead rescuers and simulated patients scored some trials with excessive motion as almost excellent and some trials with almost no motion as poor. The results were qualitatively similar when we created separate overall motion indices for each scenario independently; that is, we calculated the PCA for each scenario separately, with r_adj² values of L&S =–0.01, LR=0.10, AGIT_SIT=0.04, and AGIT_ROT=0.40. The r_adj² values for the simulated patients' scores for each scenario separately were L&S =0.00, LR=0.04, AGIT_SIT=0.06, and AGIT_ROT=0.45.

The IMU feedback affected lead rescuers' preferences for LR and agitated patient scenarios but not for the L&S. For the L&S, only 2 lead rescuers changed their preferences after completing the trials but before seeing the IMU feedback (1 in each direction). After the IMU feedback, again only 2 lead rescuers changed preferences (again, 1 in each direction). The main reason given for preferring the head squeeze was ease of use, and the main reason for preferring the trap squeeze was the stability provided.

For the LR, again only 2 lead rescuers changed their preferences after completing the trials (1 in each direction). However, after seeing the IMU reports, 6 of 8 lead rescuers who preferred the head squeeze at baseline changed their preference to the trap squeeze; no shifts in preference occurred among those preferring the trap squeeze at baseline. Reasons given for preferring the head squeeze after the IMU feedback (n=3) were ease of use for 2 of 3 lead rescuers and stability for the remaining 1. Reasons for preferring the trap squeeze after IMU feedback were stability for 8 of 9 and ease of use for the remaining lead rescuer.

For the agitated patient, 2 of 5 lead rescuers who initially preferred the head squeeze preferred the trap squeeze after conducting the trials, and 1 was uncertain. When we then provided the IMU feedback, all lead rescuers preferred the trap squeeze, and the reason given was always stability. No rescuers switched preferences from the trap squeeze to the head squeeze at any time.

Our results suggest moderate interrater reliability between the quality scores from lead rescuers and simulated patients when evaluating C-spine stabilization trials but also notable discrepancies. More importantly, validity of the subjective scores was poor compared with objective measures of motion, even for our very experienced rescuers. Finally, our findings strongly indicate that objective feedback using IMUs is more likely to alter technique preferences than does simply asking lead rescuers for subjective assessments of quality during practice sessions.

The ability to stabilize the C-spine in suspected neck injury is considered an essential element of sport medicine professional development and is included in certification examinations of the Canadian Academy of Sport Medicine, the Canadian Athletic Therapists Association, and the National Athletic Trainers' Association. Although the L&S and LR methods have been compared in several studies,6,7 we are the first to assess the ability of individual lead rescuers and simulated patients to rate the quality of the stabilization technique.

Our results suggest that reliability between the lead rescuers' and simulated patients' scores was acceptable overall but that the groups' scores differed by 2 or more points 10% to 15% of the time across all quality levels. Because the lead rescuer and simulated patient were aware of each other's scores, our results represent a best-case scenario (and the situation that would occur in the context of actual training programs); reliability would probably be worse when the groups were blinded. At a minimum, any training processes should include communication between the lead rescuer and simulated patient, so that each can better understand what the other person is sensing.

The overall validity of the subjective assessments in our study was poor. In a recent review examining the learning of motor skills in the health profession, Wulf et al18 ^(p79) stated that “learners often have a relatively good feel for how they perform.” In examining our data in more detail, we found that lead rescuers and simulated patients were generally correct when they assessed the trial as poorly done. However, both groups often (incorrectly) judged the trial as excellent when the objective data suggested more motion than would normally be considered acceptable, even by our conservative estimates. When we created a combined quality score, so that both lead rescuer and simulated patient had to assign the trial as nearly excellent or excellent, we still found that the subjective categorizations were largely inaccurate (Table 2). We hasten to add that these results depended on the context of the simulation. For example, most of the L&S trials were actually successful by objective measures, but subjectively, the trials were often rated as poor. In contrast, most of the LR trials were failures, but the subjective scores were often excellent. These discrepancies suggest that lead rescuers and simulated patients were using relative criteria in their subjective assessments rather than absolute motion. In other words, if more motion occurred with the L&S than usual, it was considered poorly conducted, even if the absolute motion was very limited. Similarly, if there was less motion than usual during the more difficult LR, it was considered excellent, even if considerable motion still occurred at the C-spine. Although these results are understandable given the nature of human sensory discrimination, they underscore the limitations of subjective measures and the importance of using objective measures for C-spine stabilization training.

Because the validity of subjective assessment was poor, feasible methods to objectively measure motion and provide feedback must be developed. Furthermore, effective feedback during training requires that it change behavior when appropriate. In our study, the feedback changed preferences for behavior for the LR (6 of 8 from the head squeeze to the trap squeeze) and confused patient scenarios (5 of 5 from the head squeeze to the trap squeeze). The feedback was not effective for the L&S scenario because motion was limited with both techniques, and therefore feedback would not be expected to change preference. Together, these results strongly indicate that using feedback with objective measures is a promising tool for changing planned behavior in C-spine stabilization training or stimulating further training to improve technique (ie, if technique can be improved, changes in preferences may not be necessary). The use of simulations and feedback to teach medical skills is supported by several recent reviews,18–20 including areas of cardiopulmonary resuscitation, trauma management, and procedural skills.21 Indeed, feedback is considered the cornerstone of clinical teaching.22 The feedback system we used fulfilled several of the reported best educational practice requirements: It should be diagnostic, focus on a few critical performance issues, and include objective indicators of performance, and the delay between task performance and feedback should be as short as possible.19 

Our study should be interpreted in the context of the study design and population. Our lead rescuers were all very experienced in the management of suspected unstable C-spine injuries. We would expect even more extreme results for all categories with less experienced lead rescuers. Although we acknowledge that generalizability is always an issue in small studies (N = 12 in our investigation), the magnitude of our findings in an experienced group of lead rescuers strongly suggests that subjective assessments are problematic in general. Although we specifically counterbalanced the order of the scenarios to eliminate fatigue as an explanation for the overall results, it is possible that fatigue increased the variability of the results. We used only anchors of 0 and 10 when eliciting lead rescuer and simulated patient assessment of the trials; more detailed descriptions along the scale of 0 to 10 might have increased reliability. Rescuers were not told of the cutoffs used to define success and failure, and knowledge of cutoffs might have improved their judgment as to whether a trial was excellent. However, this is unlikely to change the results, because our sensitivity analysis using higher cutoffs of 10° and 20° gave qualitatively similar results.

We could elicit only stated preferences and cannot be sure that all lead rescuers were truthful, but we do not believe this is a major limitation because (1) clear differences were evident in the preferences for different techniques under different conditions at baseline; (2) some lead rescuers changed their preferences to prefer the head squeeze after the trials, and some changed their preferences to prefer the trap squeeze after the trials; and (3) some lead rescuers changed their preferences for some but not all conditions. These findings indicate that lead rescuers were providing honest responses overall. That said, the lead rescuers reported only preferences, and we cannot determine whether their behavior in real-life situations will change after participation in the current study.

Lead rescuers and simulated patients did not appear able to accurately assess the quality of stabilization in suspected C-spine injury scenarios without objective measures. Providing immediate objective feedback is a promising tool for changing behavior preferences in this context and for stimulating interest in further training. Future researchers should explore the effects in less experienced rescuers and across other rescue scenarios or techniques (eg, prone patient, water rescue).