Effectiveness of Three-Dimensional Multiple-Object Tracking in Patients with Multiple Sclerosis: A Pilot Trial

More than 50% of patients with multiple sclerosis (MS) experience some cognitive decline throughout the progression of the disease.¹  Several domains of cognitive function are affected, including attention, memory, and executive function.²  Unsurprisingly, patients with MS experience significant limitations in daily functions (eg, maintaining focus while reading or driving) as a result of the cognitive impairment.^3,4  Attention, as one of the most central functions of the human brain, is frequently affected by the progression of MS.⁴  Impairments in attention may influence other fundamental cognitive processes (eg, memory, executive functioning)⁵  and may affect occupational and recreational functions.³  Several factors may cause impairments in attention in patients with MS, including demyelination of various brain circuits (eg, frontal, parietal structures) or axonal transection. Individual differences in demyelination may explain why the impairment in attention may vary in type and severity in patients with MS.⁶ 

Given the importance of attention in widespread functioning, it is surprising that few studies have focused exclusively on retraining this form of cognition. Plohman and colleagues^7,8  conducted two studies on the use of computer-based attention training with patients with MS. Both studies used software to train various domains of attention (eg, selective, distributive). Despite the small sample size (N < 25), the results indicated promising trends that domain-specific attention training may be beneficial for cognitive functioning and quality of life in patients with MS. Cerasa and colleagues⁹  conducted a randomized trial with 23 patients with MS examining the effects of computer-based cognitive training on neuroplasticity measured via functional magnetic resonance imaging. The study revealed that the training caused changes in cognitive function, which may be attributed to plasticity of involved neural networks. Last, Amato and colleagues¹⁰  conducted a study involving 88 patients with MS using computerized training to enhance attention. Although this study had the largest sample size, it revealed changes in selective attention only, whereas other domains (eg, divided, sustained) remained unaffected by the training. Taken together, results from current research provide mixed evidence of attentional training effectiveness in patients with MS. Notably, tasks used in previous research have not been generalizable to activities of daily life that are disrupted in patients with MS.

Recently, portable three-dimensional multiple-object tracking (3D-MOT) training software (NeuroTracker; CogniSens) was developed to improve cognitive perceptual skills (ie, selective, sustained, dynamic attention and processing speed).^11,12  3D-MOT trains the core deficits in patient with MS: domains of attention and information processing speed, which play a central role in a wide range of tasks/abilities.¹³  Furthermore, attention and processing speed are fundamental to decision making and other executive functions¹⁴ ; therefore, training these core functions should be advantageous for training cognition. Other advantages of 3D-MOT include¹¹  1) a large visual field and binocular 3D, which is more similar to the dynamic nature of attention required during activities such as driving than is 2D; 2) it uses an adaptive working load to improve cognitive function, which is particularly important for cognitive training for both healthy individuals and patients with MS; and 3) the task is entirely computerized so that the outcome measurement is objective and unbiased. Another important advantage of 3D-MOT is its accessibility. The technology can be displayed on any 3D-capable devices (eg, tablets, television). As such, 3D-MOT could function as an important remotely administrable and client-centered addition (because the patient can use the software from home) to existing rehabilitation protocols.

The evidence of the effectiveness of training with 3D-MOT has been sparse and mostly focused on highly functioning individuals (eg, medical students).¹⁵  Only a few studies have focused on clinical/aging populations. Legault and Faubert¹²  compared the performance of 30 younger (mean age, <28 years) and 29 older (mean age, >65 years) participants. The results indicated that younger participants were able to track objects at higher speeds than older individuals. However, older participants had a similar magnitude of training effect compared with younger individuals. From an external validity perspective, only one study has examined the transfer effect of 3D-MOT to real-life applications. Michaels and colleagues¹⁶  examined the prediction of driving ability by 3D-MOT in 155 drivers. The results revealed that 3D-MOT was a significant predictor of crash risk and mean driving speed, particularly in older participants. Although the studies were conducted with healthy older adults, it is possible that 3D-MOT training may show similar effectiveness in training skills in cognitively impaired populations (eg, patients with MS) and relate to real-life activities (eg, driving).

Hence, the primary purpose of the present study was to examine whether patients with MS would show similar cognitive improvements from 3D-MOT training compared with controls. Because of the lack of existing evidence of the effectiveness of 3D-MOT, a pilot study approach was chosen. Yet, based on research on other attention training protocols in patients with MS, we expect that the MS group will improve 3D-MOT performance at a similar magnitude compared to the control group. The secondary purpose was to examine changes in standardized cognitive tests caused by the 3D-MOT training. Although the evidence on the effects of cognitive training is mixed, we expect to see significant improvement in cognition due to the attentional training.

We performed a nonrandomized pilot trial with two intervention groups: patients with MS and controls. A pilot trial is a short-term study with a sample size of approximately 10% of a larger trial.¹⁷ 

Patients with MS were recruited through direct referrals from private neurologist offices in Regina, Saskatchewan, Canada, and through advertisements by the local chapter of the Multiple Sclerosis Society of Canada spanning 18 months. Potentially eligible patients with MS were screened for inclusion and exclusion criteria by the neurologist and the research assistant. To be included, patients with MS had to be aged 18 to 60 years and exhibit cognitive impairment assessed via the paper-based Symbol Digit Modalities Test (SDMT).¹⁸  An impairment was detected via a score of 1.5 SDs below the normative mean for the respective age group, as published by Walker and colleagues.¹⁹  This resulted in the following SDMT cutoff scores: 51.70 in those aged 18 to 35 years, 46.68 in those aged 36 to 50 years, and 43.35 in those aged 51 to 65 years. The paper version of the SDMT was used for screening purposes. Patients with MS were excluded from participation if they 1) had been diagnosed within the past 6 months, 2) had a relapse within the past 3 months, 3) received corticosteroid treatment within the past 4 weeks, 4) had any history of other neurologic or psychiatric disorders, 5) had an acquired brain injury or severe cognitive impairment, 6) had severe physical limitations (Expanded Disability Status Scale score >5), 7) already participated in a cognitive rehabilitation program, or 8) were unable to read 16-point Times New Roman font with visual correction (eg, required glasses). In addition, a group of controls matched for age, sex, and educational level were recruited from the community through advertisement e-mails and posters.

The present study received ethical approval from a health region in Western Canada. Before participation, all participants signed an informed consent form and completed a questionnaire with demographic information (eg, age, sex, education). Clinical information (eg, age at onset, disease duration, Expanded Disability Status Scale score) was recorded at baseline from neurologist medical records or self-report.

Participants performed the following tasks for baseline and post-test assessments: one 3D-MOT session, a driving readiness assessment, and a computerized cognitive battery. Each participant completed four 3D-MOT training sessions in approximately 2 weeks.

The 3D-MOT software was originally designed to train several domains of attention and processing speed in higher-functioning populations (eg, athletes).¹¹  In particular, the task mimics features of moving objects, which is similar to invasion games (eg, football, hockey) but also to several everyday activities (eg, crossing a busy street, driving). The versatility of the software permits adaptation of the task (eg, reduction of objects or speed) for cognitively impaired populations. Video examples of the 3D-MOT task are freely accessible on YouTube (https://bit.ly/30UwvXA).

In the present study, participants wore head-mounted displays (Sony HMZ-T3; Sony USA) with 3D capabilities. The 3D-MOT software works in the following five steps (see Figure 1 for an example). First, eight yellow spheres are projected into a 3D space. Next, a selected number of the eight spheres (three in the present study) are illuminated in red for a set time (2 seconds in the present study). The balls return to yellow and move around for a set time (6 seconds in the present study). Next, the balls stop and are numbered from 1 to 8. Last, the participants attempt to identify the spheres that were originally highlighted. Subsequently, the participant receives feedback on the correctness of the selection. If the participant identified the objects correctly, the balls will increase in speed on the next repetition. If the spheres are identified incorrectly, they will slow down. One training session consists of 20 trials spanning approximately 8 minutes. The software calculates an average tracking speed for all 20 repetitions. For the present study, participants completed three training sessions (ie, 60 repetitions) per visit. The average of these three sessions was used for analysis.

The Useful Field of View (UFOV)²⁰  assessment is a specific computer-based measure that requires identification and localization of suprathresholds that target the cognitive domains of information processing speed, attention, and susceptibility to distraction. Using a binocular device, the UFOV assessment measures the participant’s ability to rapidly process complex information while incorporating stimulus identification, divided attention, and selective attention via three subtests. Each subtest requires the participant to correctly identify a simplistic graphic (ie, a shape in the form of a car) after very short exposure. The complexity of the task is consistently increased with the introduction of distracting objects and shortened exposure time.²¹  Scores for each of the three subtests are expressed as the display duration at which the participant performed accurately on 75% of trials. Higher scores indicate greater impairment. Evidence suggests that the UFOV is a reliable and valid tool for clinical assessments of driving risk.²²  A meta-analysis by Clay et al²³  suggests a strong correlation between poor UFOV scores and poor driving performance in older adults, and patients with MS are impaired on this task.

The Brief Computerized Neuropsychological Battery (Sunnybrook Research Institute) was administered. The battery has been developed as a brief computerized assessment tool for cognition in patients with MS in clinical settings. Evidence suggests that the battery scores can be used to distinguish the progression of cognitive impairments in patients with MS, supporting its construct validity.²⁴  The battery consists of the following measures.

Computerized Stroop Color and Word Test. The computerized version of the Stroop Color and Word Test²⁵  is divided into three distraction tasks, where the participant is asked to identify a certain aspect of the stimulus while ignoring another salient feature. Each task is timed separately by the computer as the administrator strikes the space bar to start and finish each task. Each task includes reading a list of words or shapes with different variations, and the timer begins when the first word is read aloud and is stopped when the last word is read aloud. The first task is to read aloud a list of color names (red, green, yellow, and blue) as fast as possible while ignoring their print color. In the second task, the subject is presented with squares of various colors, and asked to state the color of each square. The final task is to name aloud the color that the words are written in as fast as possible while ignoring the actual names of the words. A recent meta-analysis showed that the Stroop test, among other tests, may be particularly useful to assess declines in cognitive functioning for patients with MS compared with other tests.²⁶ 

Computerized SDMT. The computerized SDMT is a version of the traditional SDMT¹⁸  that measures information processing speed. Participants see a screen that presents nine numbers in a row paired with unique symbols, and below the key are the symbols paired with empty spaces ordered in a pseudorandomized sequence. The objective of the computerized SDMT is for the participant to voice the matching number for each symbol as fast as possible. The total test includes eight trials, with one practice trial before the test begins. Participants are required to complete all eight trials regardless of how long they take. The computer program measures the time to completion for each trial, the total time for all trials, and the mean time per trial. The mean time per trial is the raw measure used. The test administrator is responsible for recording the number of errors made by the participant. Performance on the SDMT is significantly associated with markers of MS disease severity.²⁷  The SDMT is recommended to assess cognitive impairment in MS given its high sensitivity, specificity, and ability to accurately classify cognitively impaired and cognitively normal patients with MS.²⁸ 

Paced Visual Serial Addition Test. The Paced Auditory Serial Addition Test²⁹  is one of the most frequently used neuropsychological tests to assess processing speed and working memory. It has been readapted so that the test can be presented visually on a computer screen (ie, the Paced Visual Serial Addition Test). The Paced Visual Serial Addition Test presents a series of single-digit numbers on the computer screen, and the participants must add the number on the screen to the number seen on the previous screen and then state the sum aloud. Two trials are administered in the test, with 30 numbers given in each trial. The number of correct responses for each trial is recorded, and these numbers constitute the raw scores for the test.

All the variables were assessed for normality before parametric statistics were considered. All baseline and background data were compared via χ² tests for nominal data and independent t tests for continuous data. For the comparison of the effectiveness of 3D-MOT over time, a repeated-measures analysis of variance, with time as the repeated factor and condition as the between-subjects factor, was calculated. For the comparison of 3D-MOT and other outcome variables over time within each group, paired t tests or Wilcoxon rank tests were used. A significance level of P < .05 was set for all analyses.

In total, 16 patients with MS and nine controls were included in the present study (Figure S1, which is published in the online version of this article at ijmsc.org). Baseline characteristics of the MS group and control group are summarized in Table 1. Of the 16 patients with MS, three reported having a relapse in the 12 months before the study and two were currently not medicated for their condition. All the participants who started the study completed all the training sessions and finished the study. At baseline, the MS group performed significantly lower in 3D-MOT (t₂₃ = 5.06, P < .001, mean difference = 0.66, 95% CI = 0.31 to 1.01; Cohen’s d = 1.63, 95% CI = 0.67 to 2.56) and Stroop (t₂₃ = −2.33, P < .029, mean difference = −11.00, 95% CI = −20.78 to −1.21; Cohen’s d = −0.97, 95% CI = −1.82 to −0.10) tests at baseline. No other significant differences between the groups were found, but education approached significance (P = .051).

For the primary purpose of the present study, a repeated measures analysis of variance with a repeated factor (time) and a between-subjects factor (group) was conducted. A significant main effect for the repeated factor (F_5,95 = 7.75, η²_part = 0.29) indicated that both groups improved their 3D-MOT performance over time. A significant main effect for the between-subjects factor (F_1,19 = 18.08, η²_part = 0.47) indicated that the control group significantly outperformed the MS group in 3D-MOT. However, most notably, the MS group (t₁₅ = 5.06, P < .001, post hoc power = 0.99, mean difference = 0.33, 95% CI = 0.19–0.48; Cohen’s d = 1.46, 95% CI = 0.62–2.27; improvement from baseline = 38.33%) and the control group (t₈ = 3.24, P = .012, post hoc power = 0.81, mean difference = 0.32, 95% CI = 0.09–0.54; Cohen’s d = 1.08, 95% CI = 0.22–1.89; improvement from baseline = 20.19%) significantly improved their 3D-MOT performance from pretest to posttest in similar magnitude and trend (Figure S2).

To examine the secondary purpose of the study, dependent t tests and Wilcoxon rank tests were conducted. No significant improvements were found in the control group on any of the cognitive measures. For the MS group, the UFOV selective attention subtest showed a significant increase from pretest to posttest with a medium effect size (Cohen’s d = 0.65). All other cognitive tests indicated nonsignificant improvements over time, with small-to-medium effect sizes. Given the lack of power in the present study, the interpretation of the effect sizes should be taken with caution. All results, including significance values and effect sizes, are summarized in Table 2.

Developing accessible cognitive training strategies for patients with MS remains of interest given the mixed evidence on the effectiveness for current behavioral and pharmacological treatments.³⁰  The purpose of the present study was twofold. The primary purpose was to test whether 3D-MOT training would elicit similar effects in patients with MS compared with controls. The secondary purpose was to evaluate changes in standardized measures of cognition over the course of the 3D-MOT training.

As anticipated, the patients with MS performed significantly lower in 3D-MOT at pretest and posttest compared with matched controls. Both groups significantly improved their performance from pretest to post-test across four training sessions in 2 weeks. Most notably, the MS group improved substantially in 3D-MOT tracking speed, indicated by a large effect size (Cohen’s d = 1.36) and a 38.33% increase in performance. The improvements are similar to research in children with cognitive impairments,³¹  but the increase in 3D-MOT performance was achieved in shorter time and with fewer training sessions in the present study.

Compared with the control group, the MS group improved in similar magnitude (mean difference = 0.34 vs 0.31, Cohen’s d = 1.36 vs 1.08) from baseline to posttest. The findings indicate that patients with MS who are cognitively impaired may experience similar benefits of 3D-MOT training compared with controls. A longer period of training may yield stronger effects. These results align well with previous research. For example, Legault and Faubert¹²  showed that older adults (mean age, 66 years) performed significantly worse in 3D-MOT compared with a younger control group (mean age, 27 years). Yet, older adults experienced similar gains in 3D-MOT as the younger group during the study. In addition, 3D-MOT increases cognitive performance in other clinical populations (eg, children on the autism spectrum disorder, children with attention deficits).³¹  This evidence, combined with the present results, suggests that 3D-MOT training may improve cognitive performance in clinical populations (including patients with MS) in a similar manner as in healthy populations.

Changes in 3D-MOT performance with repeated training may indicate several neuroplastic adaptations in the brain. Parsons and colleagues³²  examined changes in cognitive performance and neuroelectric function in ten individuals receiving 3D-MOT training compared with a control group. In addition to improvements in cognition, they showed corresponding significant neuroplastic changes in the occipital cortex (ie, the brain region responsible for visual processing) measured using quantitative electroencephalography. Although these results are promising, the study has not been replicated with the use of other brain imaging techniques. Using functional magnetic resonance imaging, Cerasa et al⁹  demonstrated changes in the cerebellar lobule VI region (responsible for visuospatial skills and working memory) as a result of training with RehaCom rehabilitation software, which targets divided attention, concentration, and vigilance. However, research using functional magnetic resonance imaging and 3D-MOT has yet to be conducted.

For transfer to other cognitive domains, the most promising result is the change in UFOV selective attention performance observed from pretest to posttest in the MS group (P = .045, Cohen’s d = 0.65). Similar to the 3D-MOT task, the UFOV targets divided and selective attention as well as processing speed. Yet, the 3D-MOT software used in this study is more dynamic and uses a larger visual field, which makes it more relatable to real-life situations.¹²  The changes in 3D-MOT and UFOV may indicate important cognitive training benefits for patients with MS. Both tests have been strongly associated with real-life tasks demanding attention and processing speed (eg, driving) in nonclinical¹⁵  and MS¹⁹  populations. As such, 3D-MOT training may help patients with MS (re)train cognitive domains needed for important activities of daily living.

Note that only the selective attention subtest of the UFOV showed a significant improvement in the MS group. This may be a function of the relatively small sample size in the present study. The findings could also indicate that 3D-MOT training improves only certain domains of attention. Future research is needed to answer this question. Yet, with respect to the potential transferability to driving or other daily functions, selective attention has been identified as an important predictor of functional measures and driving in patients with MS.²¹  As such, the improvement in selective attention may yield important training effects of 3D-MOT for patients with MS. Future research is needed to answer this question.

The present study was not free of limitations. First, the sample size was small, yet it was sufficiently powered to detect the large training effect in 3D-MOT. Given the novelty of 3D-MOT training, the present study chose a pilot study approach. Multiple comparisons were not controlled for statistically. Because of the small sample size, the detected effect sizes and probability values may not be generalizable, which may diminish the findings of the study. Replication of the present study with more rigorous designs and a larger sample size is warranted. These research efforts should also include a no-training control group of patients with MS.

Another limitation is the way driving readiness was assessed. The UFOV is a computerized cognitive test that may not yield the same external validity as the use of other assessments (eg, driving simulators). As such, future research may examine the effect of 3D-MOT training on driving simulator performance in patients with MS. Preliminary evidence from such research efforts exists in other populations (eg, healthy adults).¹⁶  Last, the cognitive battery used in this study is only one way to test for cognitive changes. Other methods (eg, neuroimaging) may provide additional insights into the effectiveness of 3D-MOT training in patients with MS.

In summary, the results of this study indicate that 3D-MOT can be used to train cognitive skills in patients with MS. The training effect is similar to that for healthy individuals. Moreover, a promising trend for improvement in other cognitive measures (ie, UFOV) was detected. This is particularly important because the UFOV has been linked to important everyday activities in patients with MS. Although replication of the present study is warranted, the results indicate that 3D-MOT is a viable option to train cognitive skills in patients with MS, which may improve real-life functioning.

Dr Feinstein has received speakers’ honoraria from Novartis, Merck Serono, Roche, and Sanofi Genzyme. The other authors declare no conflicts of interest.

None.