Meaningful relationships with others are often elusive for people with intellectual and developmental disabilities, but no less desired for their full inclusion and participation in society. It is well documented that people with disabilities are victims of interpersonal violence at higher rates than peers without disabilities. This article presents a formative evaluation of the Friendships and Dating Program (FDP). The FDP was designed to teach the social skills needed to develop healthy, meaningful relationships and to prevent violence in dating and partnered relationships. Thirty-one adults were recruited by 5 community agencies in Alaska to participate. The results showed the size of the participants' social networks increased and the number of incidents of interpersonal violence was reduced for participants who completed the FDP, and outcomes were maintained 10 weeks later.
Research has documented the value of interpersonal relationships and demonstrated the strong connection between physical and mental health and the quality of an individual's social network (i.e., friendships, relationships, etc.; Cohen, 2004; Kawachi, Subramanian, & Kim, 2008). Participation in social networks has been linked to lower rates of depression, stress, anxiety, and even mortality (Berkman & Glass, 2000; Brissette, Cohen, & Seeman, 2000). People with a broad range of relationships develop a greater sense of identity, belonging, purpose, meaning, and self-worth (Cohen, 2004). However, very little attention has been paid to this core human need in people with disabilities, particularly those with intellectual and developmental disabilities (Emerson & McVilly, 2004).
Romantic relationships are important in the lives of adults with intellectual and developmental disabilities (Siebelink, de Jong, Taal, & Roelvink, 2006; Ward, Bosek, & Trimble, 2010). Unfortunately, these relationships are fraught with high rates of interpersonal violence (Copel, 2006; Smith, 2008; Ward et al., 2010). Although people with intellectual and developmental disabilities have consistently expressed the desire for opportunities to make friends and spend time with them (e.g., Knox & Hickson, 2001; Ward et al.), they experience rates of victimization 1.5 to 10 times higher than people without disabilities (Petersila, 2001; Rand & Harrell, 2009; Sobsey & Doe, 1991; Sobsey, Lucardie, & Mansell, 1995). Furthermore, people with disabilities are also less likely to have received sex education or any type of education or training in assertiveness, healthy relationships, or how to avoid sexual abuse (Barger, Wacker, Macy, & Parish, 2009). Research has repeatedly shown that interpersonal violence negatively impacts the abilities of people with intellectual and developmental disabilities to work, to live independently, and to maintain their health (Hughes 2009). It has been stressed that restricting appropriate social and sexual interactions leaves people with disabilities more vulnerable to exploitation and inappropriate sexual relationships (Bruder & Kroese, 2005; Sobsey & Varnhagen, 1989).
Studies have consistently documented that people with intellectual and developmental disabilities tend to have small social networks comprising primarily family members and staff (Emerson & McVilly, 2004; Robertson et al., 2001). In addition to having a small network, some individuals experience little or no social connectedness to others. In a qualitative study about romantic relationships, Ward et al. (2010) found that of 24 people with disabilities who experienced interpersonal violence, 38% had no one they felt they could turn to for help. Despite the well-known increased risk, little research attention has been given to preventing interpersonal violence and/or promoting healthy relationships in this exceptionally vulnerable population (Barger et al., 2009).
There is a paucity of empirical research documenting the effectiveness of approaches to teach sexual abuse protection skills to people with intellectual and developmental disabilities. No known evaluation studies have examined the behavioral impact of teaching healthy relationships skills or combining these skills with sexual abuse protection. The purpose of this formative-pilot study was to assess the acceptability and efficacy of a new, multi-session Friendships and Dating Program for adults with intellectual and developmental disabilities delivered by community agency personnel serving this population.
Friendships and Dating Program
The Friendships and Dating Program (FDP) was developed at the University of Alaska Anchorage (UAA) Center for Human Development (CHD) by a team of professionals who work with individuals with intellectual and developmental disabilities. A comprehensive meta-evaluation of the literature failed to identify interventions (Bruder & Kroese, 2005) or programs that used effective learning strategies for individuals with intellectual and developmental disabilities (Blanchett & Wolfe, 2002). Based on review and recommendations found in the literature, the following guidelines were used to develop the FDP: (a) The intervention should focus equally on protecting oneself and on creating opportunities and helping people to acquire the skills necessary to experience sexuality in a positive way (Bruder & Kroese, 2005); (b) the intervention should include three core elements: providing information and instructions, modeling and rehearsing skills in role-play, and testing and rehearsing behaviors in situ (Bruder & Kroese; Egemo-Helm et al., 2007); and (c) the intervention should include collaboration among facilitators, care providers, and FDP participants (Blanchett & Wolfe, 2002).
The FDP was created both to teach social skills necessary to develop healthy, meaningful relationships and to prevent violence in relationships for adults with intellectual and developmental disabilities. It was designed as a train-the-trainer model to build capacity in community agencies that serve individuals with intellectual and developmental disabilities to fill a gap in services related to the development of healthy relationships. The goal was for agency personnel to conduct training sessions independently. Personnel working in community agencies serving individuals with intellectual and developmental disabilities were recruited and trained as facilitators to deliver the FDP. The program consisted of 20 sessions taught twice per week over a 10-week period in small coed groups, ideally with six to eight participants. Concepts and skills were taught in odd-numbered class sessions and practiced in even-numbered community-based (in situ) sessions. Each session was approximately 1.5 hr for a total of 30 program hr.
The content of the FDP included salient concepts and skills delivered and practiced by participants in a sequential order: feelings, types of relationships, personal boundaries, communication, meeting people and first impressions, planning social activities, the dating process, personal safety, sexual health, and gender differences. The FDP introduced relationship-building methods for class sessions combining several multimodal approaches including discussions, role rehearsals, games, worksheets, handouts, videos, DVDs, slides, posters, drawings, guest speakers, and modeling. Community sessions focus on experiential activities in community settings such as malls, coffee shops, and parks to reinforce concepts taught in previous sessions by enabling participants to apply new knowledge and skills in natural settings.
The Friendships and Dating Manual (Center for Human Development, 2009) organized materials for facilitators to easily conduct sessions. Specific protocol for each session delineated learning and behavioral objectives, materials needed, session activities, scripts for teaching each concept and skill, and community in situ activities. Suggested time sequences were provided for each activity to help facilitators accomplish the session objectives. See Figure 1 for a sample of an FDP class session's objectives and activities. Facilitators received a CD with all session materials and additional resources such as DVDs, slides, posters, and games.
Facilitators attended a 2-day, 12-hr, face-to-face training program. The training introduced the purpose of the program, underlying values, teaching strategies, and learning modalities and reviewed the mandatory reporting laws, participant recruitment guidelines and materials, how to conduct the care provider orientation, how to implement the sessions, and data collection techniques. After the training, technical assistance was provided to facilitators through conference calls two times per month and on an as-needed basis.
Prior to beginning the FDP sessions, facilitators held a care provider orientation. Care providers were identified as those who provided support or spent a significant amount of time with participants. The orientation introduced the content of the program and encouraged care providers to support participants to use the skills taught during the FDP. During the delivery of the FDP, care providers received weekly guidance from facilitators about session topics to reinforce learning in day-to-day settings.
The FDP study methods and protocols were reviewed and approved by the UAA Institutional Review Board. Given the sometimes-sensitive nature of the FDP content and assessments, facilitators were trained to deal with the disclosure of violence and the emotional responses of the participants. Further, facilitators were given resources and contact information for additional assistance such as clinical debriefing or referral to a mental health service provider. The consent forms included information about the facilitator's mandated responsibility to report abuse, as well as contact information for CHD staff who were available to answer questions for care providers, parents and guardians, and participants. CHD also operates a specialized clinic for victims of violence and sexual abuse who have intellectual and developmental disabilities. Mental health clinicians were available to answer questions and provide clinical consultation.
Five community agencies serving people with intellectual and developmental disabilities throughout Alaska were recruited to provide facilitators for this study. Communities included Anchorage, Fairbanks, Juneau, Kodiak, and the Matanuska-Susitna (Mat-Su) Valley: a combination of urban and rural communities from different regions of the state. Some of these agencies participated in earlier phases of the FDP development. At least two facilitators for each group were nominated by agency directors based on their interest in the topic and their experience working (at least 1 year) with adults with intellectual and developmental disabilities. Eleven direct service personnel from these agencies were trained as facilitators to deliver the program.
Facilitators were responsible for recruiting program participants. During the facilitator training they were provided with recruitment guidelines, materials, including an introductory letter describing the program for participants and care providers, and strategies for successful recruitment. Program participants had to (a) be at least 18 years of age, (b) have a documented intellectual or related developmental disability, and (c) not have a history of inappropriate sexual behaviors. Given the explicit nature of some of the material, individuals with inappropriate sexual behaviors (e.g., paraphilia, pedophilia, assault) were excluded because they could be potential perpetrators and present a risk to other participants. Facilitators contacted the researchers to discuss individuals who had such a history to determine if there was a potential risk. Thirty-one adults who met the eligibility criteria, 14 women and 17 men, were recruited for 5 groups. Group sizes ranged from three to seven participants. All groups were mixed gender. Informed consent was obtained from the participants and their guardians when appropriate.
Process Evaluation Methods
Process evaluation methods and data provide insight about the types of participant assessment and program components that can (and cannot) be routinely delivered for specific settings, behaviors, types of providers, and program participants. In the development of a new intervention it is important to tailor program content, methods, and structure to the capacity of providers and participants. Confirmation of treatment integrity, or acceptability by participants and fidelity of implementation, is needed to establish the feasibility and replicability of a program (Hart, 2009). The literature has consistently indicated a need for evaluations of abuse prevention programs for adults with intellectual and developmental disabilities and has stressed the importance of providing empirical evidence to confirm program integrity. A meta-evaluation of the literature indicated that no specific methodological guidance had been presented about how to conduct a process evaluation for a program involving this population (Doughty & Kane, 2010).
A Process Evaluation Model (PEM) and methods applied by Public Health Programs were selected for use in this formative evaluation to monitor fidelity or how closely program procedures were implemented to the planning model (Windsor, Boyd, Clark, & Goodman, 2004; Windsor et al., 2000). The FDP process evaluation had two primary objectives: (a) to document the level of the implementation integrity for each session and the program (i.e., participant exposure) and (b) to document the comprehensiveness of the content delivered. An examination of participant exposure rates, the amount of content delivered, and the comprehensiveness of procedures accomplished provided insight to the level of fidelity achieved. Participant exposure rates were documented weekly to monitor the implementation of the FDP assessments (O) and intervention (X) sessions. In the PEM, each assessment and session of the FDP was referred to as a procedure (P). Facilitators used online surveys to report the number of participants exposed to each procedure, the amount of content delivered at each session, and detailed information about the length, location, and if modifications were made to program sessions.
Exposure and comprehensiveness rates for each procedure were divided by a performance standard, the expected level of exposure to a procedure or the expected level of content delivered, to produce an implementation index. Indexes for each procedure were aggregated to produce a program implementation index (PII) for both implementation and comprehensiveness. An overall PII ≥ .90 provided empirical data that a high level of both implementation success and comprehensive content delivery was achieved for all assessment and intervention procedures of the FDP (Windsor et al., 2004).
Feedback from the facilitators gathered through the PEM surveys was used to make modifications and revisions to the program. If program components were not delivered as intended, facilitators made suggestions on how to improve the delivery of the program content. These ideas and suggestions were incorporated into the program and helped to strengthen the program usability for facilitators and the program acceptability by participants. A detailed discussion of the Process Evaluation Model applied to the FDP is presented in Ward, Windsor, and Atkinson (2012).
Two measures were used to assess participant outcomes and to determine if the social skills taught in the FDP resulted in more friends and opportunities to develop healthy, meaningful relationships and if the program prevented violence in relationships: the social networks measure (SNM) and the interpersonal violence interview (IVI). The SNM was designed to measure the size and composition of participants' social networks, and the IVI was designed to measure the number of incidents of interpersonal violence. Facilitators conducted face-to-face interviews to collect data at baseline, after the completion of FDP (post), and 10 weeks following the end of the program. Each data collection point examined the previous 10-week time frame. Because facilitators were more familiar with the participants from their agencies than the researchers were, they collected outcome data because of the sometimes-sensitive nature of the questions on the assessments. Additionally, it was more efficient and logistically easier for the facilitator to conduct the interviews rather than a researcher located in another part of the state.
Social networks measure (SNM)
The project research team developed the social networks measure (SNM) to assess the size and composition of participants' social networks. The purpose was to measure the number of friends and potential dating relationships in a participant's life. Facilitators administered the SNM as an interview. They asked participants to identify individuals within their social network by answering the question, “Who would you do something fun with?” Once identified, participants were also asked which area of life the person belonged to, such as family, staff, friend, coworker, and so forth. Facilitators continued to inquire about members of the social network until the participant could no longer identify network members.
Interpersonal violence interview (IVI)
The interpersonal violence interview (IVI; Atkinson & Ward, 2012) was developed by the project research team to measure the number of incidents of interpersonal violence among adults with intellectual and developmental disabilities. A comprehensive search of published literature found no evidence-based measures that examined the number of incidents of interpersonal violence for adults with intellectual disability. A modified Delphi technique was employed in the development of the IVI to enhance the content, concurrent, and face validity of the instrument (Caves, 1988; Goodman, 1987; Hasson, Keeney, & McKenna, 2000; Powell, 2002; Williams & Webb, 1994). A nine-member expert panel was used to develop the IVI including researchers, professionals, and self-advocates from multiple University Centers for Excellence in Developmental Disabilities (UCEDDs) and other research institutes across the United States. Panelists reached consensus on 30 dichotomous yes/no questions as necessary components of an assessment of interpersonal violence for individuals with intellectual and developmental disabilities.
The IVI is a semi-structured interview questionnaire designed to solicit information about incidents of interpersonal violence over the previous 10 weeks. Participants were given a time anchor (e.g., “since your birthday,” “since the start of the Friendship and Dating classes”) to help them gauge the time frame. IVI questions solicited information related to several types of violence and abuse, such as physical, sexual, emotional, financial, neglect, and exploitation. For example,
6. Has anyone you know told you who you can and cannot visit?
15. Has anyone you know told you s/he would kill you?
20. Has anyone you know held or tied you down so you could not get away?
25. Has anyone you know made you have sex with her/him when you did not want to?
Participants response choices included: yes, no, don't know, refused to answer, or not applicable. If participants answered yes to any question, facilitators followed up with additional questions to determine if a violent event occurred.
The World Health Organization's definition of intimate partner violence (Taft et al., 2009) describes behaviors that can occur in significant relationships and was used to define interpersonal violence for this study. In general, it includes any behaviors that cause physical, psychological, or sexual harm. It involves a variety of systematic abusive behavior, typically in combination, including threats of and acts of physical violence, sexual violence, emotionally abusive behaviors, economic restrictions, and other controlling behaviors (Taft et al., 2009). Interpersonal relationships were perceived for this study as encompassing all significant human relationships (e.g., intimate partners, friends, personal care assistants, care providers, and family members).
Prior to implementation with the FDP, the IVI was pilot tested with 28 individuals with intellectual and developmental disabilities. Because interviewers make judgments regarding whether or not an incident of violence occurred, interrater reliability was assessed between two facilitators who administered the IVI. A Krippendorff's α (Krippendorff, 1970, 2004) was calculated to examine interrater reliability using an SPSS macro (Hayes & Krippendorff, 2007) resulting in a coefficient of 0.94. This is high considering that a coefficient 0.90 or greater is acceptable in all situations, and 0.80 is acceptable in most situations (Neuendorf, 2002). A minimum coefficient ≥ 0.70 is recommended for use in exploratory research (Lombard, Snyder-Duch, & Bracken, 2002). Approximately 1 week later, the IVI was administered again with 11 individuals, rendering a good test–retest correlation coefficient of 0.86 (Hartley & MacLean, 2006).
Process data were collected for all sites and participants in the FDP, providing the opportunity to examine the fidelity of program delivery. The total program implementation index (PII), reflecting success at all 5 sites, all 20 sessions, and all 31 participants, for the FDP was 0.98 (Ward et al., 2012). This was higher than the recommended rate PII ≥ 0.90. The total PII for FDP comprehensiveness for all sites was 0.96; again, greater than the recommended rate (Ward et al., 2012). The FDP implementation indexes documented that the intervention sessions were delivered with a very high degree of fidelity to our planning model. The PIIs provided specific evidence that direct service personnel can be trained to use the FDP and to deliver it over a 10-week period to a group of adults with intellectual and developmental disabilities. Whereas programs for almost any target group delivered over an extended period of time tend to see participation rates decrease, this was not observed by the FDP process evaluation.
Outcome Data Analysis
When individuals receive an intervention or treatment in groups, as is the case with the FDP, members of the same group have shared experiences that may result in positive intraclass correlation (ICC; Murray, Varnell, & Blitstein, 2004). ICC is the proportion of the total variance in the outcome variable that is between groups. When ICC is present, the assumption of independent observations is not met, and general linear models do not apply (Kreft & de Leeuw, 1998). In this study, the mean calculated ICC value for the SNM was 0.45 over the three units of time: baseline, posttest, and follow-up. Thus, to account for the hierarchical structure of the outcome data from the SNM, multilevel linear models were used in the analysis. However, the ICC equaled zero for the IVI indicating there was no extra variation to be accounted for at the group level. Thus, a repeated measures analysis of variance (ANOVA) approach was used to analyze the data. All statistical analyses used SPSS version 18.0 software.
Starting with the intercept-only null model that incorporated data from 31 participants, additional parameters were added one at a time with significant improvement determined by a chi-square difference test that compared changes in the −2 Restricted Log Likelihood goodness of fit measure. The most parsimonious, best-fitting model incorporated the repeated measure Time as a fixed factor along with two random effects: (a) the effect of the highest level grouping variable Site (e.g., Anchorage, Juneau) and (b) the effect of the grouping variable Person, which was nested within Site. The variable Gender did not significantly improve goodness of fit and was therefore not included in the model. See Table 1 for estimates of the fixed effects.
The default reference category for the fixed effects output in SPSS is the category with the highest numerical value, which in Table 1 corresponds to the estimated mean social network size at the 10-week follow-up (i.e., Time = 2) represented by the intercept. The estimates for the other modeled means are deviations from the intercept. Thus, the results showed a modeled average social network size of 4.14 (6.48 − 2.34) at baseline, 7.06 (6.48 + 0.58) at the conclusion of the FDP, and 6.48 at the 10-week follow-up. From baseline, the average social network size increased by 2.92 (71%) with a subsequent slight reduction of 0.58 at the 10-week follow-up (see Figure 2). The p values in Table 1 are based on comparisons with the reference category (i.e., 10-week follow-up). Compared to the mean social network size at follow-up, the mean at baseline was significantly different (p = .002) whereas the mean at the end of the FDP was not (p = .470). Reverse coding the times (i.e., Time = 2 for baseline, Time = 0 for follow-up) with baseline as the reference category showed that the mean social network size at the end of the FDP differed significantly from the mean at baseline (p = .001; estimates of fixed effects not shown).
The results of a repeated measures ANOVA showed that the number of interpersonal violence incidents was significantly affected over time throughout the FDP, F(1.21, 19.42) = 7.84, p = .008. Post hoc comparisons were made to determine where significant differences existed. The sequential Holm-Bonferroni correction (Holm, 1979) was used to control for the family-wise error rate. At baseline, post, and follow-up, there were an average of 2.53, 0.94, and 0.12 incidents of interpersonal violence, respectively, during the 10-week period preceding each measurement. As noted in Table 2, there were significant differences between mean values for all three pair-wise comparisons. This indicated a significant decline in interpersonal violence from each assessment point to the next (see Figure 3).
The FDP is based on the principle that safety training alone is not enough to prevent interpersonal violence. Adults with intellectual and developmental disabilities also need opportunities to acquire and practice skills necessary to engage in meaningful relationships. The combination of these skill areas seems to be an effective strategy to promote healthy social relationships. Although the number of sites and sample of participants were small, the results of this study were encouraging. The FDP appears to have the potential for sustained engagement with the target population, to increase the size of social networks, and to reduce interpersonal violence for participants.
Both outcome measures produced results that were statistically significant. As reported by the SNM, participants were able to identify more people in their social networks after the FDP. The statistically significant increase occurred between baseline and post. It appears participants acquired the skills necessary to expand their social relationships. There was no statistically significant change between post(assessment) and the 10-week follow-up point, which means participants were able to maintain the increased size of their social networks even 10 weeks after the program had ended. Thus, the FDP appeared successful in helping the participants to increase the size of their social networks and maintain those relationships. It is important to note that participants rarely (n = 4) identified facilitators as part of their social networks at the post and 10-week follow-up point.
Participants experienced a statistically significant decrease in the number of incidents of interpersonal violence between the 10 weeks before baseline and the 10 weeks preceding the end of the training postassessment. This change continued to decline at the 10-week follow-up point and resulted in a statistically significant difference between post and follow-up. The combination of abuse-prevention skills with sociosexual relationship skills resulted in significant decreases with regard to the number of incidences of interpersonal violence.
The results of this study were strengthened by the high level of implementation success by the FDP facilitators and participants (Ward et al., 2012). Due to the fact the 20 sessions were delivered as planned, one of the most plausible explanations for observed changes on the SNM and IVI was the FDP. The process evaluation data confirmed the delivery of the FDP with a high level of fidelity at all sites for all participants. Adults with intellectual and developmental disabilities participated in the FDP twice per week for a 10-week period and were available, with few exceptions, for all assessment procedures.
This study also demonstrated community capacity to provide evidence-based services to help people with intellectual and developmental disabilities develop healthy, safe relationships. Community agency personnel can be trained to deliver the FDP with positive outcomes. The FDP was comprehensive, relatively inexpensive, easy to implement, and required only a modest level and intensity of facilitator training. The FDP is a viable option for community service agencies to produce meaningful social outcomes for their clients.
The process and outcome results of this formative evaluation were encouraging. It is important, however, to be cautious about the study because of the small sample size and lack of a control group. Researchers were also not able to examine changes in social network size within specific categories (e.g., friends, family). It was also unclear whether the number of friends at postassessment included fellow participants in the FDP. It was encouraging that the increase in overall social network size was maintained throughout the 10-week follow-up period. Additionally, the SNM measure needs to be broadened. Future research should gather information relating to social participation such as the frequency of interactions with network members and types of activities engaged in with friends. These data will help provide greater understanding of participant's social networks.
Another limitation is that only two outcome measures were used. Unfortunately, very few measures existed for this population. A measure of change in knowledge and attitudes should be added in the future. Assessment of Sexual Knowledge Tool (Butler, Leighton, & Galea, 2003), a short and easy-to-administer tool, was used in the initial roll-out of the FDP. However, participants were not comfortable with some of the questions and, as a result, chose not to participate in the program. Other measures of sexual knowledge such as the SSKAAT-R: Socio-Sexual Knowledge and Attitudes Assessment Tool—Revised (Griffiths & Lunsky, 2003) or Sex Ken-ID: Sex Knowledge, Experience, and Needs Scale for People with Intellectual Disability (McCabe, 1994) take considerable time to administer (e.g., at least 1.5 hr, and sometimes in multiple interviews). A related limitation was the use of facilitators as data collectors. Although this decision was made to help participants feel more comfortable answering sensitive questions with someone they knew, as well as for efficiency, bias might have been introduced into the study because facilitators also delivered the intervention. Future evaluation studies with the FDP need to address these limitations. Future work should also involve larger sample sizes and control groups, which will likely produce results with narrowed confidence intervals around the estimates of fixed effects and provide a better opportunity to adequately and more accurately address the issue of effect size.
Meaningful relationships with other people are often elusive to people with intellectual and developmental disabilities, but they are no less desired and no less necessary for overall health and well-being as well as, ultimately, full inclusion and integration in society. The FDP was an initial response to the near absence of a program that focused on the basic human need for relationships with other human beings. Further research is needed to explore the issues of sexuality and the development of healthy relationships for adults with intellectual and developmental disabilities.
This research was supported by a grant from the Alaska Mental Health Trust Authority (AMHTA). AMHTA had no involvement in the study design.
Editor-in-Charge: Glenn T. Fujiura
Karen M. Ward (e-mail: firstname.lastname@example.org), UAA Center for Human Development, 2702 Gambell Street, Suite 103, Anchorage, Alaska 99503, USA; Julie P. Atkinson and Curtis A. Smith, University of Alaska Anchorage; Richard Windsor, George Washington University.