Dear Editor:

We thank the letter authors for taking the time to review and analyze our work. We respect the points raised by the authors and agree that the method they discussed that could have been used for our study is an alternative to our established approach. However, as with many research questions, multiple valid methods are often appropriate for collecting, processing, and interpreting data. The letter authors indicated that they were primarily concerned that our methods may have increased the potential for type 1 error (ie, suggesting that areas of the brain were active when they were not). We provide an evidence-based justification for using whole-brain analysis (ie, exploratory analysis) and the subsequent secondary analyses13  to answer the research questions in our study.

First, suggesting that on the order of 130 000 statistical tests be performed indicates a lack of understanding of the actual dimensionality of the data. In a random-effects analysis of activation from a group or groups of participants, the number of unique tests is limited to (at most) the number of individuals in the study, essentially a number that is many orders of magnitude smaller. This is fundamental linear algebra based on the rank of the data matrix. Individual voxels do not represent uncorrelated variables but are aggregated due to strong spatial correlations among them. This underlying aggregation is visible in the group activation maps, as spatially adjacent voxels tend to have a stronger correlation and may materialize as clusters or blobs. Although correlations are strongest between adjacent voxels within a cluster of activation, correlations among physically remote regions of the brain exist as well, as evidenced by studies of functionally connected brain networks.4  Thus, it is not clear what may constitute unique tests and multiple comparisons in an image dataset. Bonferroni and Sidak corrections are valid in the case of independent uncorrelated variables but tend to be overly conservative for correlated variables.5 

Moreover, in Tables 2 and 3, our legends clearly stated that the P values were uncorrected, which provides transparency for readers.6  The analyses in our study were considered exploratory due to the novel application of these techniques in the field of sports medicine and the small sample size. It is also critical to reduce the potential for type II error as strict statistical corrections that increase this risk are often debated in the literature.7  When type II errors occur, viable treatment options and information that could help patients become unavailable and are potentially permanently eliminated as options for enhancing care.7  Additionally, as described in 1 of the letter authors' references, if exploratory data analyses are performed, the results must be presented with appropriate disclaimers, which we did.8  We were transparent about the exploratory nature of this study and recognized the barriers of obtaining a sample size needed for such rigorous analyses by using a novel and typically expensive measurement technique, such as functional magnetic resonance imaging (fMRI). Therefore, we opted to use uncorrected P values in Tables 2 and 3.6 

The letter authors are either misunderstanding or perhaps misrepresenting the referenced work on circularity. Kriegeskorte et al8  described circularity as using the same data to delineate regions of interest (ROIs) that will eventually be analyzed as part of the study (ie, functionally defined ROIs), in contrast to using a standardized atlas of identified brain regions. Unfortunately, a major limitation to using functionally defined ROIs is that this method lends itself to a self-fulfilling prophecy, as follows: determining a posteriori the probability of detection for a cluster of voxels that have already been identified by the investigator using the same data that demonstrated activation to the task. Now, that is “circular.” In contrast, our analysis was based on a priori-defined ROIs from an established atlas (automated anatomical labeling [AAL]).9  Therefore, the ROI data for the AAL atlas parcellation were extracted without any correction or thresholding. As a result, the ROI means simply represented the average of the activation measures by fractional signal change across all voxels within the previously established ROI.

We would argue that second-level analyses of activation based on a priori-defined anatomical ROIs, such as those of the AAL atlas, are in fact very informative. In using ROIs involving all voxels without thresholding, any effects that appear as statistically significant in a random-effects analysis are less likely to lead to a type I error, as this analysis expects additional random variation in the variables. An atlas-based approach also makes the results more readily interpretable, as the AAL atlas is included in most fMRI analysis and visualization packages. These ROIs are anatomically defined and do not adhere to functional region boundaries unique to the data being studied, thereby minimizing the risk of bias in the subsequent analysis.

Furthermore, a random-effects analysis of measures of activation (averaged across all voxels within an ROI) from individual participants was performed in a fully transparent manner offline using commonly available statistical software (SPSS; SPSS Inc). This method is an appropriate way to handle the data and allows us to begin to integrate traditional sports medicine statistical analyses that are more interpretable and digestible to our target audience (clinicians treating patients with anterior cruciate ligament reconstruction [ACLR]). As we analyzed between-groups and within-group differences in average fractional signal change by using traditional sports medicine statistical analyses (ie, independent and paired t tests), the “dead salmon” argument raised in the letter is not applicable to our work. Moreover, in the dead salmon argument, the authors performed a fixed-effects analysis of the time series response data from a single fMRI scan in a single (ie, the dead salmon) rather than a fully transparent random-effects analysis of activation in a group or groups of (living) participants, as in our study.6  In the 3-voxel cluster in the single dead salmon, type I error was, without a doubt, present. However, as eloquently argued by Lieberman and Cunningham,10  if 16 dead salmon were scanned and analyzed, the same false alarm would not be present in the same location; therefore, a group-level analysis will not show this effect as data aggregation allows for self-correcting of false alarms (ie, type I error).10  Conservative thresholding techniques, such as those proposed by the letter authors, can consequently increase type II error rates and limit our ability to advance patient care.10 

Lastly, we would like to state that the regions identified were not “cherry picked.” We selected regions that were previously identified by researchers as meaningful emotional regulation areas in patients with musculoskeletal injuries3  and areas specifically different for patients after ACLR.11  In fact, the development of a working hypothesis based on the most rigorous, available evidence and then testing the hypothesis are fundamental to the scientific method as first described by Aristotle. We note with strong emphasis that the regions we selected were also identified in a recent thesis12  and associated published conference abstract13  on which 1 of the letter writers was the senior author, further strengthening our rationale for selecting the specific brain regions included in our study. In this work, which identified neural activity associated with kinesiophobia after ACLR, the authors used the analysis proposed in their current letter and reported similar results to ours, even though they used a different task-based paradigm.12,13  Specifically, Kim et al12  observed greater activation in the left thalamus, precuneus, primary somatosensory cortex, primary motor cortex, lingual gyrus, superior parietal lobe, corticospinal tract, left cerebellum, and corpus callosum. These findings align very closely with our results presented in Figure 2 and Table 26  and further suggest the robust nature of our conclusion that injury-related fear may have led to altered neural patterns associated with emotional regulation in patients post-ACLR. Although the potential for type I error cannot be eliminated, we used rigorous methods to greatly minimize the potential and have since identified findings from researchers in other laboratories that corroborate ours.

We used a picture imagination task paradigm similar to that in a previous fMRI study3  of emotions in patients with chronic musculoskeletal pain that demonstrated comparable results. We wanted to (1) identify whether between-groups differences existed regardless of picture category and if they did exist (2) begin to identify whether we could quantify if it was because of sport-specific or activities of daily living images. We could have also completed this analysis in our healthy control participants, but it was unnecessary to answer our specific questions for patients post-ACLR. Our interest was not in whether this was present in healthy patients; therefore, we only conducted this in our population of interest. We invite colleagues in the field, including the letter authors, to replicate our study by using the proposed picture imagination task paradigm to examine whether differences are present in healthy individuals with respect to sport-specific images or activities of daily living images.

The purpose of our investigation was to provide further contextual evidence that injury-related fear should not be overlooked during rehabilitation. However, the letter authors attempted to nullify our findings when our methods were sound and our results suggested that an area of care is lacking in patients post-ACLR (ie, psychological evaluation and rehabilitation). Activation patterns similar to what we reported (ie, increased activation in the thalamus, cerebellum, and occipital regions and inability to suppress the default mode network) have been observed in patients with chronic musculoskeletal pain,3  with medial patellofemoral ligament deficiency,14  and after ACLR7  by using the analyses described by the letter authors. Subsequently, our findings support a growing body of literature in which researchers emphasize that injury-related fear may lead to objectively measured brain changes after musculoskeletal injury and warrant targeted interventions. We hope that the use of neuroimaging in sports medicine research continues to grow with both new and established investigators conducting novel hypothesis-generating studies aimed at solving significant clinical problems.

1. 
Kesler-West
ML,
Andersen
AH,
Smith
CD,
et al
Neural substrates of facial emotion processing using fMRI
.
Brain Res Cogn Brain Res
.
2001
;
11
(2)
:
213
226
.
2. 
Xu
M,
Fralick
D,
Zheng
JZ,
Wang
B,
Tu
XM,
Feng
C.
The differences and similarities between two-sample T-test and paired T-test
.
Shanghai Arch Psychiatry
.
2017
;
29
(3)
:
184
188
.
3. 
Taylor
AM,
Harris
AD,
Varnava
A,
et al
A functional magnetic resonance imaging study to investigate the utility of a picture imagination task in investigating neural responses in patients with chronic musculoskeletal pain to daily physical activity photographs
.
PLoS ONE
.
2015
;
10
(10)
:
e0141133
.
4. 
Sporns
O.
Networks of the Brain
.
MIT Press
;
2010
.
5. 
Conneely
KN,
Boehnke
M.
So many correlated tests, so little time! Rapid adjustment of P values for multiple correlated tests
.
Am J Hum Genet
.
2007
;
81
(6)
:
1158
1168
.
6. 
Baez
S,
Andersen
A,
Andreatta
R,
Cormier
M,
Gribble
PA,
Hoch
JM.
Neuroplasticity in corticolimbic brain regions in patients after anterior cruciate ligament reconstruction
.
J Athl Train
.
2021
;
56
(4)
:
418
426
.
7. 
Armstrong
RA.
When to use the Bonferroni correction
.
Ophthalmic Physiol Opt
.
2014
;
34
(5)
:
502
508
.
8. 
Kriegeskorte
N,
Lindquist
MA,
Nichols
TE,
Poldrack
RA,
Vul
E.
Everything you never wanted to know about circular analysis, but were afraid to ask
.
J Cereb Blood Flow Metab
.
2010
;
30
(9)
:
1551
1557
.
9. 
Tzourio-Mazoyer
N,
Landeau
B,
Papathanassiou
D,
et al
Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain
.
Neuroimage
.
2002
;
15
(1)
:
273
289
.
10. 
Lieberman
MD,
Cunningham
WA.
Type I and type II error concerns in fMRI research: re-balancing the scale
.
Soc Cogn Affect Neurosci
.
2009
;
4
(4)
:
423
428
.
11. 
Grooms
DR,
Page
SJ,
Nichols-Larsen
DS,
Chaudhari
AMW,
White
SE,
Onate
JA.
Neuroplasticity associated with anterior cruciate ligament reconstruction
.
J Orthop Sports Phys Ther
.
2017
;
47
(3)
:
180
189
.
12. 
Kim
H.
Indentifying Neural Activity Associated With Kinesiophobia After Anterior Cruciate Ligament Reconstruction
.
Master's thesis.
Ohio University
;
2020
.
13. 
Kim
HW,
Onate
JA,
Simon
JE,
Mischkowski
D,
Grooms
DR.
Identifying neural activity associated with kinesiophobia after anterior cruciate ligament reconstruction [abstract]
.
J Athl Train
.
2021
;
56
(6s)
:
S228
.
14. 
Kadowaki
M,
Tadenuma
T,
Kumahashi
N,
Uchio
Y.
Brain activity changes in somatosensory and emotion-related areas with medial patellofemoral ligament deficiency
.
Clin Orthop Relat Res
2017
;
475
(11)
:
2675
2682
.