Continuing Education: Improving Athletic Trainers’ Knowledge of Musculoskeletal Diagnostic Ultrasound Through a Physician-Led Longitudinal Ultrasound Course

• A 6-month longitudinal MSK diagnostic US course improved ATs’ abilities to acquire, label, and optimize US images of normal MSK structures. This course can be recreated in a setting with a physician US expert and ATs who are interested in assisting the physician with US evaluations.

• Athletic trainers who complete a diagnostic US course should spend time working directly with a physician who performs US as part of their job since spending more days or hours per week doing this was associated with higher image scores in the postcourse assessment.

• Understanding the basics of MSK diagnostic US will allow ATs to assist physicians when they are performing diagnostic and interventional US in the clinic, in the athletic training facility, and on the sidelines of sporting events. Athletic trainers may also play a role in the emerging field of “tele-US.”

Ultrasound (US) is an imaging modality that is increasingly being utilized by physicians practicing orthopedics and sports medicine. Diagnostic US can be performed as part of a patient evaluation in an outpatient clinic, in an athletic training facility, or on the sidelines of sporting events. Ultrasound can be utilized for diagnostic purposes to evaluate the muscle, tendon, ligament, bone, and bursae or guide interventional procedures. Ultrasound is an operator-dependent skill that requires a comprehensive understanding of anatomy, training in image acquisition and optimization, and hands-on practice. Hands-on,^1–4  instructor-led^5–7  US training has been shown to be superior to didactic or self-guided training alone. Ultrasound training has been shown to improve the accuracy of physical examination palpation skills.^8–10  Athletic trainers (ATs) work closely with physicians, taking histories, performing physical examinations, and assisting with the diagnosis and treatment of acute and chronic musculoskeletal (MSK) conditions. At our institutions, ATs work in the clinic, in the athletic training facility, and on the sidelines, assisting physicians with the capture, labeling, and optimization of US images during diagnostic US examinations and interventional US-guided procedures. If a physician is unable to be physically present, such as during an away game, ATs may perform “tele-US,” an emerging practice where US scanning is performed by a novice under the direction of an expert through video calls in real time.^11–15 

Given that many ATs work with sports medicine physicians who utilize US in their practice, an instructional MSK diagnostic US course was developed, offering free Board of Certification continuing education credits for the ATs working at our institutions. Previous studies have not evaluated MSK US training or ATs’ ability to acquire and optimize diagnostic images after taking part in an anatomy and US training program. The purposes of this study were to (1) determine if a longitudinal 6-month MSK anatomy and US course would increase ATs’ US image scores overall and by body region (to assess their ability to acquire, label, and optimize US images of normal structures that may commonly be injured by active persons) and increase their overall confidence in performing US, (2) determine if higher US image scores were correlated with higher self-reported confidence, and (3) determine if demographic factors such as AT practice location and practice setting or work-related factors such as years practicing as an AT, years performing US, days per week performing or observing US, hours per week observing or performing US, or previous formalized US training were associated with higher US image scores. We hypothesized that the course would result in improvements in ATs’ confidence and US image scores, that higher confidence would be correlated with higher image scores, and that weekly time spent observing or performing US would be associated with higher image scores.

Physicians invited all ATs with whom they worked (18 ATs) to participate in an MSK diagnostic US course for ATs that was conducted at the Mayo Clinic, Mayo Clinic Health Systems, and the University of Iowa. Seventeen ATs (10.8 ± 8.1 years of experience working as a certified AT) participated in the course. The ATs’ demographics are summarized in Table 1. We collected precourse, during-course, and postcourse self-reported confidence scores and US images for those who consented to participate in the prospective cohort study, which was approved by the Mayo Clinic (no. 20-008617-02) and the University of Iowa Institutional Review Board (no. 202008451).

Athletic trainers participated in an MSK diagnostic US course that met weekly for 1-hour sessions (virtual didactics with hands-on sessions that were primarily in person, with an occasional virtual session^16,17  due to participant mandatory quarantine secondary to the severe acute respiratory syndrome coronavirus 2 [SARS-CoV-2] pandemic, which occurred for 2 ATs for 1 week throughout the course) from November 2, 2020, through April 9, 2021. The course was taught by physicians trained in physical medicine and rehabilitation and sports medicine; the student-to-teacher ratio for the hands-on sessions was 2:1 to 4:1, depending on the site. A detailed timeline for the course is outlined in Figure 1.

Participants were asked to complete precourse surveys administered in REDCap (Research Electronic Data Capture)^18,19  assessing their years of experience as an AT; previous experience observing or performing US; weekly time spent observing or performing US in their current job; previous formal MSK US instruction; confidence on a 0 to 100 scale in identifying structures when a physician is scanning, identifying structures when an AT (AT completing the survey) is scanning, labeling the US image, and optimizing the US image (depth/gain); and demographic information (see Supplemental File 1, available online at www.nataej.org). After each weekly session during the course, participants were asked to complete a survey confirming attendance for anatomy, US video viewing, and hands-on scanning (see Supplemental File 2). The same questions assessing AT confidence were repeated weekly during the course to assess the ATs’ confidence for each body region and after the course to assess their overall confidence (see Supplemental File 3).

Athletic trainers acquired specific images designated on the image structure list (Table 2) that was developed by the physicians and included 6 structures for each body region that are commonly encountered in sports medicine settings. Athletic trainers obtained all images of the structure precourse and postcourse. Each week during the course, ATs obtained images of structures for that specific body region (Figure 1). Participants were made aware of the physician-selected key components of US image acquisition and interpretation, included in the image scoring criteria, and were presented with examples of optimal and suboptimal image acquisition (Figure 2). Three sports medicine physicians who completed a physical medicine and rehabilitation residency (at different institutions) and a sports medicine fellowship (2 at the same institution with a different fellowship program director and 1 at a different institution) with a focus on diagnostic and interventional MSK US (W.O.J., R.C.K., and A.N.S.) independently evaluated deidentified images of normal structures obtained by the ATs. Since there is no previously validated system to score MSK US images, a scoring system was created by the study authors. Images were scored, with 0 = absent and 1 = present, for the following criteria: structure label, centered images, surrounding structure label, directionality indicated (eg, medial or lateral), and quality of the overall image (correction of anisotropy and optimization of depth and gain). The consensus value (ie, needed 2 reviewers to agree) for each criterion was used in the analysis. Each image was given a score out of a maximum of 5 points. These scores were then averaged among all participants for each structure. An average score was calculated precourse, during the course, and postcourse for each body region and overall.

Demographic data were reported using standard descriptive statistics. We calculated Krippendorff α to assess the interrater reliability between US image scorers (W.O.J., R.C.K., and A.N.S.). Before the study, one set of sample images was scored with consistency among all scorers. Pre-, during-, and postcourse confidence values were compared using a repeated-measures analysis of variance (ANOVA) with an unstructured covariance matrix; if time was significant, post hoc tests with a Tukey correction were used to identify significant differences in least-square means at each time point. Athletic trainer image scores were compared pre-, during-, and postcourse overall and by body regions using Freidman’s ANOVA with a Tukey adjustment for post hoc comparisons. Spearman correlations were used to assess the correlation between AT confidence in identifying an image when the AT was scanning and their overall image scores pre-, during, and postcourse. Spearman correlations were also used to assess the correlation between AT overall postcourse image scores and the number of years working as an AT, the number of years spent observing US, the number of years learning or performing US scanning, and the number of days and hours per week that the AT spent observing or performing US scanning. Due to the categorical nature of the variables, Kruskal-Wallis tests were used to determine the relationship of ATs’ overall postcourse image scores with the study site, AT role, and formal US training. If a significant association was identified, pairwise comparisons were assessed, and P values were adjusted for type I errors using the Dwass-Steel-Critchlow-Flinger method.

All ATs reviewed the comprehensive anatomy lecture and the American Medical Society for Sports Medicine (AMSSM) lectures for each body region, except for 1 participant who did not review the comprehensive anatomy lecture for elbow/wrist/hand or watch the AMSSM videos for hip. All participants attended the hands-on sessions for each body region. For each of the 17 participants, a total of 90 images were scored by each of the 3 image scorers (W.O.J., R.C.K., and A.N.S.). The interrater reliability of image scoring was relatively high (Krippendorff α = 0.71, 95% confidence interval [CI] = 0.69, 0.73).

A significant difference in AT confidence pre-, during, and postcourse (Figure 3) was detected for identifying a structure when the physician was scanning, properly labeling an image, and adjusting depth/gain (all P < .001). No significant difference was detected in identifying a structure when the AT was scanning (P = .30). Post hoc comparisons revealed significant pairwise differences between all time points for confidence in identifying a structure when the physician was scanning (P values pre- to during course of <.001, pre- to postcourse of <.001, and during to postcourse of .03). Confidence in labeling an image and confidence in optimizing depth/gain pre- to during course, pre- to postcourse, and during to precourse were statistically significant (during to postcourse P = .04 for labeling; all other P < .001).

Overall, we found that the ATs’ image scores improved throughout the course from 1.34/5 precourse (range, 0/5 to 4.23/5), to 3.53/5 during the course (range, 2.6/5 to 4.37/5), to 3.83/5 postcourse (range, 2.33/5 to 4.67/5). There was a significant difference among the average precourse, during-course, and postcourse image scores overall (χ²[2] = 24.47, P < .001) and for each body region (all P < .01) (Table 3). Post hoc testing revealed that for knee and shoulder, all time points were significantly different (all P < .01). For ankle/foot, all time points were significantly different (P < .01 for all except during versus postcourse [P = .04]). For elbow and hip, pre- versus during course and pre-versus postcourse were significantly different (all P < .01), but during versus postcourse was not significantly different (P = .22 for elbow and P = .99 for hip). The highest scores were recorded for the knee (precourse = 1.94/5, during course = 3.98/5, and postcourse = 4.44/5), and the lowest scores were recorded for the hip (precourse = 0.85/5, during course = 3.23/5, and postcourse = 3.20/5) at all time points. The average postcourse image scores for each anatomic structure are displayed in Figure 4.

We found a positive correlation between ATs’ confidence in their ability to identify US structures when the AT was scanning and overall image scores. This was statistically significant precourse (r_s[17] = 0.64, P = .005) but was only marginally correlated but not significant postcourse (r_s[17] = 0.47, P = .06). During the course, there was a significant positive correlation between ATs’ confidence in their ability to identify US structures when the AT was scanning and image scores for the ankle/foot (r_s[17] = 0.57, P = .02) and shoulder (r_s[17] = 0.55, P = .02). No significant correlations between ATs’ confidence in their ability to identify US structures when the AT was scanning and image scores for the knee (r_s[17] = −0.17, P = .95), elbow/wrist/hand (r_s[17] = 0.44, P = .08), and hip (r_s[17] = −0.15, P = .57) were identified.

No significant associations between the average score of all images collected during postcourse and study site (P = .47), years working as an AT (P = .62), AT role (clinical, outreach, or other) (P = .44), the number of years spent observing USs before the course (P = .49), the number of years learning or performing hands-on scanning (P = .52), or US instruction before this course (none, didactic only, or hands-on scanning) (P = .79) were detected. There was a significant positive association of the number of days per week that the AT currently spent observing or performing US scanning (P = .008) and the number of hours per week that the AT currently spent observing or performing US scanning (P = .02) with the postcourse overall image scores, although both groups of ATs who spent time observing or performing US weekly (n = 6, average overall scores of 2.36 precourse and 4.25 postcourse) and ATs who did not spend time observing or performing US weekly (n = 11, average overall scores of 0.43 precourse and 3.61 postcourse) improved throughout the course.

Our results showed that during this 6-month, longitudinal MSK diagnostic US course, AT US image scores significantly improved over time. Although slight, the postcourse image scores were higher than the during-course image scores, indicating that the improvements made between precourse image acquisition and image acquisition during the course were retained, and even slightly increased, over time. This is likely due to the regular repeated assessments and longitudinal approach to the course as retention of acquired knowledge and skills are enhanced with continued exposure to content and iterative practice with more time with hands on the transducer.^20–22  Additionally, ATs who spent more days and more hours per week observing or performing US with a physician had higher scores on the postcourse images. This underscores the importance of iterative practice and exposure to US with frequent exposure to a physician US expert for on-the-job learning. Overall, the MSK diagnostic US course was successful in enhancing the ATs’ abilities to acquire, label, and optimize US images of normal structures, but having continual exposure to US as part of their job is significantly important in enhancing the ATs’ US abilities.

The ATs’ confidence in their ability to identify structures when an AT was scanning did not increase throughout the course, but their confidence in their ability to identify structures when the physician was scanning, their ability to properly label, and their ability to optimize depth and gain did increase. This may be due to the fact that most ATs working with a physician identify images while the physician is scanning, label images, and optimize depth and gain on a regular basis, but they rarely perform US scanning in their daily practice. Previous studies of physicians have revealed a correlation between confidence and US skill level, but one study assessed the interpretation, not the acquisition, of images by emergency medicine residents and physicians²³ ; one reported improvements in identifying normal images of the wrist but did not evaluate image optimization by radiology residents and MSK fellows²⁴ ; and another assessed performing and interpreting abnormal focused abdominal US examinations.²⁵  In our study, ATs were acquiring images of normal structures and were asked to optimize images rather than identify pathology. Overall, this MSK diagnostic US course improved ATs’ confidence in performing the tasks that they more commonly perform in our clinics and athletic training facilities: identifying structures when the physician is scanning, labeling images, and optimizing depth and gain. However, self-reported confidence in their ability to acquire or identify US images when they are scanning should be interpreted cautiously, as this did not correlate with the US image acquisition skill level.

There were several strengths to this study. This study evaluated a comprehensive, 6-month longitudinal MSK diagnostic US course that covered peripheral joints and surrounding soft tissue structures. All participants attended the hands-on scanning sessions, completed surveys, and submitted US images throughout the course. Previously, the effects of US training on a learner’s ability to acquire and optimize diagnostic US images have been limited in scope to isolated body regions^26,27  or typically evaluated for resident or fellow physicians.²⁴  To our knowledge, this was the first study to assess a learner’s ability to acquire US images of normal MSK structures and can serve as a model for future studies that have MSK US image acquisition as an outcome measure.

There were also several limitations to this study. This study encompassed only a single year, and it is unclear if the information learned during the course will be retained over subsequent years. Implementing a similar course at other institutions may present a challenge, as a physician skilled in MSK US is necessary to lead the course. It is unclear if a different US curriculum would produce similar results. In addition, completion of the course required a weekly commitment by the ATs at our institutions, which they did outside their typical work hours. Athletic trainers spent roughly 1 hour per week reviewing the anatomy lectures and the AMSSM US videos and roughly 2 to 3 hours per week during the hands-on session and acquiring US images. Obtaining pre- and postcourse images took several hours and, due to busy sports seasons, took some ATs the full 6 months to complete the image acquisition as part of the postcourse assessment. Finally, the overall improvement in posttest image scoring may have been influenced by factors outside the course, such as the time of exposure to US in clinical settings; however, given that ATs who observed and performed US while working with a physician on a weekly basis and those who did not both improved throughout the course, participation in the course likely contributed to the overall improvement in image acquisition scores that was observed.

The subjective image scoring system that we employed had not been previously described or validated; however, this system was developed to assess the key components of MSK US image acquisition and optimization. Measures to decrease the subjective nature of the scoring system were employed, including obtaining consensus among 2/3 of the image reviewers and having well-defined criteria. However, there was relatively high interrater reliability between the physician scorers for each, and future studies assessing learners’ ability to acquire MSK US images could utilize this scoring system. Additionally, the ability to acquire US images of normal structures was assessed, and any correlation with the ability to acquire US images of pathologic structures is unknown. Finally, although the image scoring system was utilized to assess key components of US image acquisition and optimization, it was not designed to determine criteria that would indicate that an AT is competent in performing US under a physician’s guidance.

Ultrasound is an imaging modality that is increasingly being utilized by physicians and the ATs who work with them in sports medicine clinics, in athletic training facilities, and on the sidelines. We have described a comprehensive longitudinal MSK diagnostic US course and found that participation may have resulted in significant improvements in ATs’ abilities to acquire, label, and optimize US images of normal MSK structures that are commonly injured by active persons. Athletic trainers who spend more time with physicians who utilize US had the best image scores. Developing a thorough understanding of MSK US will allow ATs to better assist physicians when they are performing diagnostic and interventional US in the clinic, in the athletic training facility, and on the sidelines of sporting events.

We acknowledge Jeffrey M. Payne, MD, and Shelby Johnson, MD, for their assistance with teaching the hands-on portion of the ultrasound course and Stephanie Kliethermes, PhD, for her assistance in performing the statistical analysis.