The purpose of this study was to explore the cross-cultural validity of the Self-Determination Inventory: Student Report, a newly developed measure of self-determination grounded in Causal Agency Theory. The tool was translated to Spanish and administered to American and Spanish adolescents. The sample was structured to include adolescents with and without intellectual disability in both cultural contexts. More than 3,000 students in the U.S. and Spain aged 13 to 22 completed the assessment. Findings suggest that the same set of items can be used across cultural contexts and in youth with and without intellectual disability, although there are some specific differences in item functioning across students with and without intellectual disability in Spain that must be further researched. There were specific patterns of differences in latent self-determination means, with students with intellectual disability scoring lower in the U.S. and Spain. Implications for assessment research and practice in diverse cultural contexts are explored.
Promoting the development of self-determination has received attention nationally and internationally (Ginevra et al., 2013; Mumbardó-Adam, Guàrdia-Olmos, Giné, Shogren, & Vicente, 2018; Shogren & Ward, 2018) given the association between self-determination and positive school and postschool outcomes (Shogren, Garnier-Villarreal, Lang, & Seo, 2017). For example, researchers in multiple countries have found that enhanced self-determination leads to more positive employment and community participation outcomes after adolescents leave school (Shogren, Wehmeyer, Palmer, Rifenbark, & Little, 2015). Researchers have also found that self-determination impacts quality of life and life satisfaction across the lifespan (Nota, Soresi, Ferrari, & Wehmeyer, 2011; Shogren, Lopez, Wehmeyer, Little, & Pressgrove, 2006). For these reasons, assessment and intervention tools have been developed to support self-determination in school contexts, with a focus on enabling adolescents to develop self-determination skills during the transition from school to adult life. Increasingly, the importance of supporting self-determination for adolescents with and without disabilities in inclusive settings has been emphasized (Shogren, Wehmeyer, & Lane, 2016). An emerging body of research suggests that students without disabilities also benefit from self-determination interventions (Raley, Shogren, & McDonald, in press). However, until the recent introduction of the Self-Determination Inventory (Shogren, Wehmeyer, et al., 2017), there has not been a assessment that was developed to reliably measure the self-determination of adolescents with and without disabilities in inclusive environments.
The Self-Determination Inventory: Student Report (SDI:SR) was developed to address this need in the field, providing a self-report measure of self-determination for adolescents with and without disabilities, including those with intellectual disability (Shogren, Little, et al., in press). The SDI:SR was developed to align with Causal Agency Theory (Shogren, Wehmeyer, Palmer, Forber-Pratt, et al., 2015), a recently introduced theory to conceptualize the development of self-determination in adolescents with and without disabilities. Causal Agency Theory integrates past theoretical work and research on the development of self-determination in the fields of education and psychology. Causal Agency Theory defines self-determination as a “dispositional characteristic manifested as acting as the causal agent in one's life” (Shogren et al., 2015, p. 258) and identifies three essential characteristics of self-determined actions: volitional action, agentic action, and action control beliefs. Volitional action refers to the extent to which a person makes intentional, conscious choices based on preferences and self-initiates goal-setting using past experiences as a guide. Agentic action involves self-directing and managing actions towards a freely chosen goal, including the identification of different pathways to navigate encountered barriers, engaging in goal-directed action, and managing and evaluating actions taken. Finally, action-control beliefs relate to recognizing one's own strengths and weaknesses related to goal pursuits. One's self-knowledge shapes those beliefs and drives persons toward goal-directed actions. This conceptualization of self-determination has the potential to provide practitioners and researchers with valuable information to inform the decision-making process that shapes intervention selection and supports provided for all youth, so long as the self-determination construct can be assessed with reliability and validity.
One aspect of the validity of an assessment and the theory that undergirds it is construct comparability across cultures. Researchers have consistently asserted that there are universal elements of the self-determination construct, but that self-determined actions may be expressed differently across cultures (Shogren, 2011). For this reason, the goal of this study was to explore the cross-cultural validity of the SDI:SR by examining its use in American and Spanish adolescents. This represents the first examination of a translated version of the SDI:SR, and its cross-cultural comparability. Our specific focus was examining similarities and differences at the measurement level (i.e., can the same set of items be used) and the construct level (i.e., are there differences in overall self-determination scores across cultures) providing direction for assessment in diverse cultural contexts. We were also interested in exploring differential impacts of the presence of an intellectual disability across cultural contexts.
Our research questions were as follows:
Can the same set of items be used across the SDI:SR and SDI:SR Spanish Translation (SDI: SR Spanish)?
Does measurement invariance hold (e.g., can the same set of items be used) across the SDI:SR and SDI:SR Spanish in adolescents without disabilities and with intellectual disability?
Are there differences in overall self-determination scores across adolescents with and without intellectual disability on the SDI:SR and SDI:SR Spanish?
Sample and Procedures
In this study, a subset of the sample used to validate the SDI:SR in the United States was utilized (see Shogren, Little, et al., in press, for additional details on the validation of the SDI:SR). To generate the sample for the SDI:SR validation, a sampling plan was devised to enable analyses of the reliability of items and validity of scores across groups stratified by disability and age. The sampling plan targeted adolescents aged 13 to 22 years with diverse disability labels (i.e., no disability, learning disabilities, intellectual disability, other health impairments, autism spectrum disorders, emotional and behavioral disorders, and sensory disabilities) in the United States. The sample was stratified into two-year age bands crossed by disability label. Within the age and disability groups, efforts were made to further stratify the student sample based on other personal characteristics, including race/ethnicity and gender. The overall sample consisted of 4,741 adolescents from 39 states, representing all regions of the United States. To generate the sample, after receiving university institutional review board (IRB) approval to conduct the research, a multi-pronged recruitment effort was undertaken. School districts (including general and special education administrators and teachers) were approached, as were postsecondary institutions in rural, suburban, and urban areas across the United States. Information was also disseminated through local, state, and national organizations' email listservs and social media accounts.
In the present study, we used a subset of the overall validation study sample, focusing on adolescents that did not have disability labels (no disability group) and adolescents that received special education services under the educational classification of intellectual disability (intellectual disability group). The decision to focus on these two groups was made to align with the focus of the recruitment efforts in the Spanish context (described subsequently). As such, the U.S. sample utilized in the present study included 2,668 adolescents aged 13 to 22 (M = 16.5; SD = 2.35). Approximately half of the U.S. sample (n = 1,305) were in grades 9 to 12 and 16.5% were enrolled in a college or university. The sample contained youth identifying as Black, Hispanic, White, and Other races/ethnicities. Further demographic information, including counts for all education levels, is provided in Table 1.
To generate the Spanish sample, adolescent participants were recruited from 31 schools or college and universities across regions of Spain. In total, 620 adolescents completed the SDI:SR Spanish. Of the 620 participants, 249 (40.2%) were students without disabilities enrolled in general education schools or universities and 371 (59.8%) were students with disabilities enrolled in inclusive or segregated settings. Despite efforts to promote the inclusion of students with disabilities in inclusive settings in Spain, students with disabilities, especially students with intellectual disability, are still educated in segregated secondary schools. Significant reforms are being promoted in several regions, for example, the Catalonia Government recently decreed that all students must be enrolled in inclusive settings and that supports should be provided using a multi-tiered system of supports framework to promote access to the general education curriculum, however the Spanish data collection efforts reflected the continued, high use of segregated settings. Only 30 out of 371 students with disabilities in the sample were educated in inclusive settings. Perhaps more illustrative of the segregation of students with intellectual disability, of those 30 students with disabilities enrolled in inclusive settings, only one was diagnosed with intellectual disability. The remaining students had other labels, such as sensory and hearing impairments. However, it was hard to generate a sufficiently large sample of students with other disability labels in inclusive schools as they are not identified in the same ways as students that attend a segregated school. Disability information was requested of teachers in inclusive settings, however, rarely is such information available as youth rarely identify as having a disability in these settings. However, teachers in segregated settings were able to provide information on the presence of intellectual disability as well as levels of deficits in intellectual functioning as this is information is collected for students that attend such settings.
Therefore, the SDI:SR Spanish sample consisted primarily of adolescents without disability labels in typical school settings and adolescents with intellectual disability labels in segregated school settings. For this reason, we only focused on students without disabilities and with intellectual disability in the U.S. sample, as noted previously. The 29 students who had other disability labels in the Spanish sample were also excluded, resulting in a total sample of 591 Spanish adolescents. Spanish teachers reported that 34.8% of the sample of youth with intellectual disability had mild intellectual impairments; 43.6%, moderate; and 21.6%, severe impairments.
The 591 Spanish adolescents included in the present analysis ranged in age from 13 to 22 years old (M = 16.95; SD = 2.01), the majority being male (57%). More adolescents with intellectual disability (58%) were included in the sample, compared to adolescents without disabilities. More than half of the students were enrolled in 9th (20.1%) or 10th grade (29.3%). Students enrolled beyond compulsory education programs (in Spain compulsory education extends through the 10th grade) were either in 11th or 12th grade (5.9%), in vocational training programs (25.4%), universities (13.9%), or in transition to adult life programs for students with intellectual disability (5.4%). Table 1 provides additional demographic information on the Spanish sample. It should be noted that racial/ethnic information was not collected in the Spanish sample. Race and ethnicity constructs have different implications in the Spanish context and asking about race or ethnicity can be seen as disrespectful, and thus such data are not usually collected in the Spanish context. Further, these constructs are often confused with the person's place of origin.
To address the research questions, the U.S. and Spanish samples were combined and the overall sample (n = 3,259) was utilized in the analyses. The U.S. and Spanish sample included roughly equal numbers of adolescents with intellectual disability; however, the U.S. sample included a much larger number of youth without disabilities. The steps taken to address this in the analyses are described in the Research Question 2 analysis description below. In terms of other sample characteristics, the percent of males and females was similar in the two countries and the average age of the youth was very similar, although the standard deviation was larger in the U.S. sample (σu.s. = 2.35 versus σspain = 2.01).
Self-Determination Inventory: Student Report
As described, the SDI:SR was developed to provide a new self-report measure of self-determination aligned with Causal Agency Theory for use with adolescents with and without disabilities aged 13 to 22. The final pilot version of the SDI:SR consisted of 45 items (Shogren, Wehmeyer, et al., 2017) and the number of items was reduced to 21 during the validation study (see Shogren, Little, et al., in press, for a full description of item refinement). During the item refinement process, the items with the strongest loadings and association with the constructs defined in Table 2 were identified. An overall self-determination score, as well as subscale scores for the three essential characteristics and seven components constructs of self-determined action, can be calculated. The factor structure of the scale and sample items are provided in Table 2. In this study, we explored a single factor model (overall self-determination score) as well as a three-factor model representing the three essential characteristics of self-determined action (volitional action, agentic action, and action-control beliefs). Previous research has demonstrated that the 21 items can be reliably used in U.S. samples of youth with diverse disability labels (i.e., no disability, learning disabilities, intellectual disability, autism spectrum disorders, other health impairments) from varying racial/ethnic backgrounds (i.e., White, Black, Hispanic/Latino(a), Other) (Shogren, Little, et al., in press). Researchers have also established that latent scores are influenced by personal factors, including disability label and race/ethnicity in U.S. samples, as well as by environmental factors, including socioeconomic status (Shogren, Shaw, Raley, & Wehmeyer, in press).
The 21 item SDI:SR is delivered using an online platform, with users indicating responses on a slider scale ranging from 0 to 99 with anchors of “Disagree” and Agree.” The slider scale was adopted to reduce discrimination errors and allows the data to be treated as continuous (Ahearn, 1997; Rausch & Zehetleitner, 2014). The average reading level across all items is approximately third grade, and the online system provides accessibility features, including audio playback and in-text definitions.
Self-Determination Inventory: Self-Report – Spanish Translation
The SDI:SR Spanish utilized in this study consisted of the 45 items aligned with the SDI:SR pilot version (Shogren, Wehmeyer, et al., 2017). However, the purpose of Research Question 1 was to determine if the same 21 items identified in the SDI:SR validation study could be reliably used to measure self-determination in the Spanish sample to promote alignment of the SDI:SR and SDI:SR Spanish and enable them to be delivered through the same online platform.
To translate the SDI:SR to Spanish, the following steps were taken based on Tassé and Craig's (1999) guidelines for international translation of scales in the disability field: (a) translating and back translating the original scale; (b) experts revising the items based on clarity, importance, and relatedness within the Spanish language and culture and applying their comments and suggestions to the preliminary version of the scale; (c) pilot testing the preliminary version to explore psychometric properties, followed by necessary modifications of the scale; and (d) administering the modified measure with a broader sample to further analyze psychometric properties through structural equation modeling approaches. The pilot version of the SDI:SR Spanish has demonstrated acceptable psychometric attributes, consistent with the U.S. version (Mumbardó-Adam et al., 2018). Construct validity analysis confirmed the alignment with Causal Agency Theory and concurrent validity was established through score correlations with a previously validated measure of self-determination (Mumbardó-Adam et al., 2018).
Research Question 1
To address the first research question we compared the 45-item pilot version of the SDI:SR Spanish with the 21-item version developed in the SDI:SR validation study in the U.S. to determine the feasibility of using the reduced set of items for the SDI:SR Spanish. We examined a three factor model, based on the alignment of the SDI:SR items on the 21 and 45-item versions with the three essential characteristics of volitional action, agentic action, and action-control beliefs. We also explored a single factor model, representing overall self-determination, as analyses in the U.S. sample have suggested the appropriateness of a single factor model (Shogren, Shaw, et al., in press). Factor loadings and model fit were evaluated to determine how the confirmatory factor model changed with the reduction of indicators from 45 to 21 for both the one and three factor model to inform decisions about the viability of the 21-item measure in the Spanish context and the appropriateness of the one factor model for further exploring cross-cultural validity. For the 21-indicator model to be considered acceptable for the following analyses, we expected model fit to either change very little or improve when moving from the 45- to 21-indicator Spanish model.
Research Question 2
Assuming there were only minor differences in model fit across the 45 and 21-indicator model in the overall Spanish sample, our plan was to move forward with the 21-items and explore the use of the items across the U.S. and Spanish sample. Our analytic plan called for the creation of a four-group model (intellectual disability-Spanish, intellectual disability-U.S., no disability-Spanish, no disability-U.S.) to evaluate cross-cultural measurement and latent invariance using the 21 items across youth with and without intellectual disability. The data for the Spanish sample were complete except for one missing response on two separate action-control belief indicators. The U.S. validation study, however, implemented a three-form planned missing design (Little & Rhemtulla, 2013) in which all youth provided demographic information and then responded to approximately 75% of the remaining questions to reduce the time to administer the measure as additional measures to evaluate construct validity were included in the validation study (see Shogren, Little, et al., in press). This method introduced missingness under the missing completely at random (MCAR) mechanism while reducing respondent burden due to fewer questions for respondents to answer. Missingness due to MCAR enabled the recovery of unbiased estimates through multiple imputation (Enders, 2010) in R 3.4.2 (R Core Team, 2017) with PcAux package (Lang, Little, & PcAux Development Team, 2017). The imputation process generated 100 imputed data sets. The Spanish sample was then merged with each of the 100 imputed data sets and exported for confirmatory factor analysis and measurement invariance testing in Mplus 7.2 (Muthén & Muthén, 1998–2015). Merging of data sets was completed in R (R Core Team, 2017) and exported with the mitml R package (Grund, Robitzsch, & Luedtke, 2017).
To evaluate the four group model, we used a confirmatory factor analysis method developed to evaluate invariance and norm a standardized scale based on a translation of the original measure (see Seo, Shaw, Shogren, Lang, & Little, 2016). The group size for U.S. youth with no disabilities (n = 2,369) was 9.5 times larger than the smallest group in the model, Spanish youth with no disabilities (n = 249). Because this size imbalance may mask items that perform differently in one of the smaller groups (Chen, 2007), measurement invariance testing was conducted first on just the Spanish validation sample in a two-group model (intellectual disability vs. no disability) so that changes to the Spanish groups due to partial invariance could be incorporated into the four-group model.
Measurement invariance testing in the two group Spanish model and in the four group U.S. and Spanish model testing that followed used the same process. Specifically, the factor structure, factor loadings, and indicator intercepts for configural, weak, and strong invariance were evaluated (Brown, 2006). Configural invariance examined whether the same factor structure could be applied to the two or four groups. For acceptable fit, root mean square error of approximation (RMSEA), a measure of absolute fit, should be less than 0.08 with good fit around 0.05. The comparative fit index (CFI) and non-normed fit index (NNFI) should exceed 0.90. If model fit is acceptable, the second stage equates factor loadings across the groups, and change in model is examined. The primary criteria for passing weak invariance is a change in CFI (ΔCFI) < .01 (Cheung & Rensvold, 2002) from the configural to the weak model. The same ΔCFI < .01 criteria is used to evaluate change in model fit between the weak and strong model in which the indicator intercepts are constrained in addition to the equated factor loadings. Change in RMSEA is expected to fall within the previous model's 90% confidence interval. If all stages of invariance testing pass, then the model is considered invariant at the measurement level.
If ΔCFI < .01 criteria is not met during the measurement invariance testing process, nested model testing is used to identify which parameters cannot be equated across groups. Because the data for the Spanish validation sample was complete except for one missing response on two separate action-control belief indicators, full information maximum likelihood managed missingness and nested model testing based on change in χ2 was used to identify estimates that could not be equated across groups. Those variant individual parameters are freed in the model, one at a time, until ΔCFI falls below .01. If some, but not all, parameters can be equated at each stage of invariance testing, the model is referred to as partially invariant.
Testing for the four-group model based on the U.S. and Spanish validation samples followed the same steps as those used for the Spanish two group model with the Cheung and Rensvold (2002) change in CFI criteria (ΔCFI < .01) utilized to evaluate invariance. If the Spanish groups did not pass measurement invariance but were partially invariant, the final partial invariant model for the Spanish groups were included (see Seo et al., 2016) throughout the measurement invariance testing process. In the case that a stage of invariance testing failed due to ΔCFI > .01, nested model testing with imputed data was conducted with the Wald test, which evaluates the effects of constrained parameter estimates rather than overall model fit because multiple imputation methods cannot pool the χ2 for nested model testing. Once the parameters that could not be equated were identified, then individual parameters were freed in the four-group model until ΔCFI falls below .01.
Research Question 3
After examining measurement invariance, the next step was to explore latent differences across the U.S. and Spanish contexts in the disability groups. Specifically, we examined latent means for the self-determination construct to see if there were differences across the four groups. Differences were evaluated statistically with the Wald test in which the first test constrained the self-determination latent mean to equality across all groups using an a priori α = .05. If this test led to the null hypothesis being rejected, then pairwise comparisons with a Bonferroni adjusted α = 0.008 were conducted based on 6 comparisons. Latent d was also calculated to obtain a standardized effect size of group differences, interpreted as the distance between latent means on the normal curve. The formula is
where K refers to the latent mean, n refers to the group size, ψ refers to latent variance, and the subscripts for 1 and 2 refer to the groups. Latent variances were also evaluated to determine if there were differences among groups with the same α levels used to evaluate latent means.
For the two-group model based on the Spanish subsample, the youth with no disabilities served as the reference group. For the four group model, the largest group, U.S. youth with no disabilities, was set as the reference group. Throughout the analysis, latent variables were identified and scaled with fix factor scaling. This method places the factor on the normal distribution with mean of 0 and standard deviation of 1 (Brown, 2006). After completion of all testing, the strong model with equal factor loading and indicator intercepts was re-estimated with effects coding to identify and set the scale. Model fit does not change with the implementation of different scaling, but the metric of the latent variable is placed on the metric of the original questions (Little, Slegers, & Card, 2006) to improve interpretability of the results.
Research Question 1
The first step of the analyses was to examine if the 21-item version of the scale that was refined from the 45-items version during the validation study in the U.S. fit the data well in the Spanish sample. Therefore, we modeled the 45 indicator and the 21 indicator model with 3 factors defined by items loading on the essential characteristics of volitional action, agentic action, and action-control beliefs (see Table 2). Based on model fit statistics shown in Table 3, overall model fit improved in the 21 indicator model over the 45 indicator model, as would be expected because of the reduced number of items loading on each construct. The RMSEA for the 45-indicator model fit was 0.055 with 90% CI [0.053, 0.058]; the statistic was larger but still in the range of acceptable (< .08) at 0.060 with 90% CI [0.060, 0.071] in the 21 indicator model. Larger RMSEA was expected in the 45 indicator model because it contained more degrees of freedom (Kline, 2011). The comparative indices of CFI and NNFI improved slightly in the reduction to 21 indicators because some of the items that were removed to create the 21 indicator model shared less variance with the other items on the same factor. For example, CFI increased from .830 to .889 and NNFI increased from 0.822 to 0.874. Although values < 0.90 are considered an indicator of poor fit on these indices, they were influenced by the degree to which the indicators correlated. The standardized root mean also improved with a reduction from 0.050 to 0.047 for the 45 and 21 indicator models respectively. Based on the minor changes in model fit between the 45 indicator and 21 indicator models, we found the 21-indicator model to be an acceptable model for comparison against the same 21 indicators in the SDI:SR version for the U.S. We also estimated a single factor (overall self-determination) model with all items loading on a single factor, as this was found to best fit the data in the U.S. validation sample (Shogren, Shaw, et al., in press). The model fit was slightly worse in the one-factor model with increased RMSEA and decreased CFI and NNFI as seen in Table 3. However, these values were still in the acceptable range, and because of the U.S. findings we chose to move forward with the one-factor model for comparisons between the U.S. and Spanish sample in Research Question 2.
Research Question 2
After deciding to move forward with the single factor model, we followed the process outlined by Seo et al. (2016) for cross-cultural comparisons of translated scales. We first evaluated measurement invariance in the Spanish sample before it was combined with the U.S. sample. The first model estimated the same single factor structure for the two groups (Spanish – no disability; Spanish – intellectual disability) to create the configural model. The factor loadings were equated to estimate the weak model. As shown in Table 3, based on ΔCFI =.002 between the configural and weak model, factor loadings could be equated across groups meaning the items load on the self-determination factor in the same way across groups. After the weak model, intercepts were equated across groups to create the strong model. The groups differed on the indicator intercepts in the SDI:SR Spanish Translation based on ΔCFI = .025, indicating not all intercepts could be equated across groups. Nested model testing, with a Bonferroni adjusted α = .002, identified four intercepts that were statistically significantly different across the no disability and intellectual disability groups in the Spanish sample that when freed resulted in ΔCFI = .009. The four questions that had to be freed across youth with and without intellectual disability in the Spanish sample were from each of the three essential characteristics: volitional action (“I consider many possibilities when I make plans for my future.”), agentic action (“I set my own goals”), and action-control beliefs (“I know what I do best.” and “I am confident in my abilities.”). As such, the two-group model was considered partially invariant at the strong level of invariance. Because of the partial invariance, we also examined the three factor model for invariance within the Spanish sample, to see if similar patterns were obtained. The three factor model showed the same pattern of partial invariance at the strong stage with final model fit of χ2(404) = 989.564, RMSEA = 0.07, 90% CI [0.065, 0.076], CFI = .864, and NNFI = 0.859. This model fit suggested that it was reasonable to move forward with the one factor model.
The next step was to combine the U.S. and Spanish samples for testing measurement invariance in the four group model: U.S. adolescents with no disabilities, Spanish adolescents with no disabilities, U.S. adolescents with intellectual disability, and Spanish adolescents with intellectual disability. The configural model of equal structure contained no constraints on the parameters except those needed to set the scale on the factors. Model fit was acceptable based on RMSEA < .08 and both CFI and NNFI > 0.90. After the factor loadings were equated, ΔCFI between the configural and weak model was .002, the same amount of ΔCFI observed in the two group model for the Spanish sample data. Factor loadings could be equated across all four groups. Prior to equating indicator intercepts for the strong model, the four indicators that were identified in the measurement invariance testing for the Spanish sample were freed in the group for Spanish youth with intellectual disability. Based on change in model fit for the model with the partially invariant structure for the Spanish youth with intellectual disability, all other intercepts could be equated across groups. As shown in Table 3, ΔCFI equaled .01 instead of < .01, but given the simulation study that provided the recommended threshold for ΔCFI was based on a two group model (Cheung & Rensvold, 2002) and this model was a four group model, the level of change in CFI was considered minimal enough to pass invariance. Final model fit on the strong model for the four groups was χ2(872) = 4415.578, RMSEA = 0.071, 90% CI [0.069, 0.073], CFI = .925, and NNFI = 0.928. This meant that the same set of items could be used across the four groups, so long as the intercepts of the four items in the Spanish sample with intellectual disability were allowed to vary from the other groups.
Research Question 3
After establishing partial invariance across the four groups, latent means and latent variances in overall self-determination scores across the four groups could be examined. The values reported in Table 4 are the estimates from the partially invariant strong model. As would be expected from the latent variances, the omnibus test to equate latent variances across all groups was statistically significant, Wald(1) = 212.12, p < .001. Based on a Bonferroni-adjusted α = .0083, pairwise testing indicated that latent variances differed in every pair of groups. Latent variance was larger in the U.S. groups with the largest variance in the group of U.S. youth with intellectual disability. The group with the smallest variance was Spanish youth without disabilities.
Self-determination latent means could also not all be equated (Wald(3) = 35.96, p < .001), but as shown in Table 4 not all groups were statistically different from each other. Again, for comparing each pair of latent means, α was adjusted to control the Type I error rate. With the Bonferroni-adjusted α = .0083, testing showed the self-determination latent mean could be equated between Spanish youth with intellectual disability and the two U.S. groups, yet the two U.S. groups could not be equated to each other. No other groups were equal. Self-determination was highest for Spanish youth with no disabilities, followed by U.S. youth with no disabilities, then Spanish youth with intellectual disability and lastly U.S. youth with intellectual disability. The larger variance in the two U.S. groups created wide distributions that resulted in the overlap with the Spanish youth with intellectual disability.
Latent d was computed to obtain a standardized effect size so mean differences could be interpreted independently of sample size. In this model, the statistic was primarily useful in understanding differences between the largest group, U.S. youth without disabilities, and the other 3 groups, which were approximately equal in size. As shown in Table 5, the largest effect size (d = 0.45), was between the Spanish youth with no disabilities and the U.S. youth with intellectual disability, a difference that is considered medium with the latent means approximately one-half of a standard deviation apart. The difference between U.S. youth without disabilities and Spanish youth without disabilities was −.13, a value that indicated the latent mean was larger for the Spanish adolescents but the mean difference was small. The latent mean difference was also small (d = 0.11) between U.S. youth without disabilities and Spanish youth with intellectual disability. All other group differences were moderate in size with absolute effect sizes that ranged from 0.21 to 0.34.
The purpose of this article was to explore the cross-cultural validity of the SDI:SR in American and Spanish adolescents, informing future research and practice in the assessment of self-determination across cultural contexts. There were several key findings. First, at the measurement level, we established that the same set of 21 items identified in a large, validation study in the U.S. (Shogren, Little, et al., in press) could be utilized in for the SDI:SR Spanish. This enables comparability in measurement, particularly as the scales will be available through the same online delivery platform (see www.self-determination.org). Second, we were able to establish partial measurement invariance across the U.S. and Spanish samples of youth with and without intellectual disability, suggesting it is reasonable and justified to use the same set of items across these contexts. Such a finding is critical to enable the comparison of the self-determination construct across cultures, examining factors that may impact overall self-determination. However, it is important to note that a small subset of items (n = 4) had to be freed within the Spanish sample across youth with and without intellectual disability. The implications will be discussed further in a subsequent section, but this suggests specific differences in measurement across youth with and without intellectual disability in the Spanish context (this finding was not replicated in the U.S. context) that must be further examined. Third, the findings suggest differences at the construct level across youth with and without disabilities in the U.S. and Spain that have implications for future research and practices conceptualizing interventions to promote self-determination.
Implications for Future Research and Practice
In considering implications for future research and practice, an important next step will be further examining and researching the four items that needed to be freed within the Spanish sample. First, the experiences of the youth with intellectual disability in Spain likely differed from their peers without disabilities given their education in segregated settings. A limitation was that specific details on education placement were not collected for the U.S. sample, and further details should explore the impact of education placement on self-determination in U.S., Spanish, and other cultural contexts. The level of inclusion likely shaped the ratings and perceptions of Spanish youth on the four items that differed. Specifically, the volitional action item was related to considering many possibilities when making plans for the future. However, future plans might not have the same meaning and possibilities for students with and without intellectual disability in the Spanish educational context. Students with intellectual disability in segregated settings may have limited options for future education. The focus is typically on employment and job-related skills, however, even the range of possibilities for employment tend to be more restricted than those for their peers without disabilities. Regarding the two agentic action items that were freed, these items focused on being confident in own abilities and knowing one's strengths. The differences on these items across groups align with previous research on the 45 items version of the SDI:SR Spanish that reported differential item functioning on these items (Mumbardó-Adam, Guàrdia-Olmos, & Giné, 2018a). It is possible that Spanish adolescents with and without disabilities might be conceptualizing their strengths and abilities differently. In students without disabilities strengths and abilities might be linked to academic skills, which are traditionally taught in secondary settings at the expense of other social or behavioral skills in the Spanish context. For students with intellectual disability, strengths may be more strongly linked to positive personal characteristics or transition related skills. The final item that was freed from the action-control beliefs domain focused on settings one's own goals. Again, this item may reflect the different implications for adolescents with and without intellectual disability based on education setting. Students with intellectual disability are rarely involved in formal goal setting activities; for example, students with intellectual disability are never present at IEP meetings and are rarely aware of even having them. Researchers have found that students with intellectual disability have fewer opportunities to engage in self-determined actions at home in Spain, compared to their peers without disabilities (Mumbardó-Adam, Guàrdia-Olmos, & Giné, 2018b), perhaps further limiting goal setting opportunities. Interestingly, similar differences were not found in the U.S. sample in this or previous studies (Shogren, Little, et al., in press) as measurement invariance could be established across all items. This suggests, perhaps, a greater focus on self-determination for students with intellectual disability in the U.S. context in ways increasingly aligned with expectations for peers without disabilities, although significant work remains to be done (Shogren & Ward, 2018). This may also have implications for practice as Spain continues to promote inclusive education and higher expectations for students with intellectual disability. In the Spanish context, educators and others utilizing self-determination assessments may want to particularly focus on these items in adolescents with intellectual disability and explore ways to promote instruction aligned with these areas. At present, when interpreting scores on the SDI:SR Spanish Translation, educators should consider that these four items' scores are impacted by the presence of intellectual disability and consider this when designing interventions and comparing results across groups of students. Further, in developing norms in the Spanish contexts, the need for different norms across populations should be explored.
Although the findings suggest that the same set of items can be used across the Spanish and U.S. context for adolescents with and without intellectual disability, there were interesting patterns of latent differences in the self-determination construct that must be further explored. As would be predicted (see Table 4), American and Spanish adolescents without disabilities scored higher than their counterparts with intellectual disability. This suggests that youth without disabilities across contexts likely have greater opportunities for self-determination, consistent with the wide-body of literature suggesting disability-related differences in self-determination opportunities. Interestingly Spanish youth with and without intellectual disability scored slightly higher than their counterparts in the U.S. These differences should be further explored, with a focus on examining the factors that lead to higher self-determination scores in adolescents with and without intellectual disability in Spain. Additional analyses, however, are also needed to examine other personal factors (e.g., gender, age) that might impact scores. For example, in the U.S. sample, we had a highly diverse group in terms of race/ethnicity, and other analyses have suggested differences in scores based on race/ethnicity as well as other environmental factors, such as socioeconomic status (Shogren, Shaw, et al., in press). Although the race/ethnicity construct is not directly relevant in the Spanish context, there is a need to collect information that can be utilized across cultural contexts to examine factors that might lead to disparities based on differences in expectations and access to resources and opportunities. For example, Shogren, Shaw et al. (in press) suggest that expectations and restrictions in opportunities likely lead to adolescents from diverse backgrounds scoring lower in self-determination in the U.S., and that part of these differences can be explained by environmental factors, including increased risk for poverty and lower quality education services and supports. Exploring if similar factors impact self-determination in the Spanish context is an important direction for future research, as is determining if these factors can be aligned in some way and meaningfully researched across cultural contexts. For example, we note that the scores of the Spanish sample without disabilities in this sample is similar to scores in the White sample without disabilities found in other studies (Shogren, Shaw, et al., in press), suggesting that the differences between Spanish and U.S. adolescents without disabilities found in this study may be influenced by the diversity present in the U.S. sample. Similarly, in research with the American sample, intellectual disability and race/ethnicity have been found to interact in influencing self-determination scores and diverse youth with intellectual disability are at high risk for poverty and restricted access to high quality education services and supports (Shogren, Shaw, et al., in press). The intersectionality of disability, race/ethnicity, and other cultural factors across countries and regions of the world deserves further attention, particularly to provide guidance for promoting equity in access to supports for self-determination that are culturally responsive across the world.
Limitations and Future Directions
Compromises were made in designing the analysis for this study to enable the cross-cultural comparison that must be considered in interpreting the findings and planning for future cross-cultural research. First, there are differences in how personal factors that may influence self-determination scores are defined and the implications for education services across U.S. and Spain. For example, in the U.S. there has been a push toward access to inclusive settings. Although students with intellectual disability in the U.S. are still at risk for segregated placements (Kurth, Morningstar, & Kozleski, 2014), these placements are still more likely to be in segregated classrooms in public schools, rather than in wholly segregated schools. This likely impacts the experiences of adolescents with intellectual disability across the U.S. and Spain, making it difficult to generate a directly comparable sample because of the different education experiences. And, we did not collect specific data on educational placement in the U.S., focusing instead on other personal factors that could be easily reported on by students and teachers. Further, there is less identification of other disabilities that are identified in the U.S. in school contexts, leading to additional differences in the samples that can be recruited. For example, it may be that some of the youth in the Spanish no disability sample may have disabilities that would be identified in the U.S (e.g., learning disabilities). Further research is needed on the implications of these differences in classification and the provision of services in the U.S. and Spain, and the impact on outcomes, including self-determination outcomes. Further, the differences in compulsory education across the U.S. and Spain must be considered. In Spain compulsory education only extends through the 10th grade, and additional analyses should explore differences based on the differences in the type of education that adolescents and youth adults without disabilities receive across the U.S. and Spain. For example, could the earlier transition from school to postsecondary education or employment influence self-determination in Spanish adolescents? Future research is needed that explores age and education status related differences across cultures.
Additionally, the results from Research Question 1 indicated that a three-factor model was a slightly better fit to the data than the one factor model in the Spanish sample, but the data from the U.S. sample does not support a three factor model because the indicators are too highly correlated to support more than one factor (see Shogren, Shaw, Raley, & Wehmeyer, 2018). Future research should explore the factors that influence these differences across cultural contexts. Finally, as mentioned previously, the lack of consistency in demographic information both in terms of what is available from schools (e.g., consistent information on level of intellectual deficits for students with intellectual disability in educational systems is lacking in the U.S., but more widely available in Spain) as well as the need to identify other factors that could influence scores within and across contexts (e.g., personal and environmental factors that have cross-cultural relevance and shape opportunities and experiences for self-determination) will further enhance cross-cultural research.
Overall, the findings suggest that the SDI:SR has cross-cultural applicability and that researchers and practitioners can feel confident in using the 21-item version of the SDI:SR and SDI:SR Spanish to assess self-determination. The findings also suggest directions for future research and practice both within the Spanish and U.S. context, particularly related to the impact of educational placement on self-determination outcomes and the degree to which educational experiences and expectations shape item functioning in the Spanish context. Ongoing cross-cultural research exploring common factors that influence self-determination, its development, and effective supports for its promotion are needed and this research provides preliminary findings that can guide this work.