The Delphi technique was used to obtain expert panel consensus to prioritize content areas and delivery methods for developing staff grief and bereavement curriculum training in the intellectual and developmental disabilities (IDD) field. The Delphi technique was conducted with a panel of 18 experts from formal and informal disability caregiving, nursing, and hospice and bereavement service provision. Results showed that training should help staff identify and support service users experiencing grief. Importantly, staff also needs help in managing their own grief. Organizational policies and resources should be instituted to support the grief processes of both service users and staff. Practice-based applications are discussed, and research implications are presented for training evaluation.
Because service users with intellectual and developmental disabilities (IDDD) are living longer (Coppus, 2013; Janicki, Dalton, Henderson, & Davidson, 1999; Patja, Livanainen, Vesala, Oksanen, & Ruoppila, 2000; World Health Organization & The World Bank, 2011), there is greater likelihood that they will grieve the deaths of significant others, such as family members and friends, as well as experience other losses in adulthood (Perkins & Moran, 2010). Staff can play a significant role in assisting service users through the grief process, although staff's training in this area is often lacking (McEvoy, Guerin, Dodd, & Hillery, 2010).
Compounding this situation, staff members also experience personal bereavement challenges when a service user dies, although they may not receive the training nor support to navigate the bereavement process. Once staff receive adequate support to manage their personal losses, they can function more effectively as caregivers (McEvoy et al., 2010). Because service users' and staff's grief processes are often interconnected, grief and bereavement training should incorporate the needs of both service users and staff.
In this study, the Delphi technique (de Villiers, de Villiers, & Kent, 2005; Hasson & Keeney, 2011) was used to obtain expert panel consensus to prioritize content areas and delivery methods to be included in staff grief and bereavement curriculum training in the IDD field. Such training can help staff support service users as well as themselves during times of bereavement. This article provides an overview of the Delphi technique and details findings and lessons learned from the current study.
Existing Training and Staff Training Needs
Existing bereavement training in the IDD field has not consistently been evaluated nor consistently reported in peer-reviewed literature ( Blackman & Todd, 2005; Clearbrook, 2010; Luchterhand & Murphy, 1998; Steinman, 2006; Stern, Kennedy, Sed, & Heller, 2000). Gaps in staff training in the IDD field include the need for more background information on grief and bereavement, staff assistance to better identify and communicate with service users experiencing loss; staff assistance to help service users move on postloss; staff assistance in managing personal challenges with grief and bereavement; and the need for more practical application of grief theories.
To support service users adequately, staff can benefit from training on how service users understand or experience loss (Morgan & McEvoy, 2014; Gray & Abendroth, 2015). Service users' understanding of loss can vary, due to multiple factors, including cognitive ability and personality (Brickell & Munir, 2008; Dodd, Dowling, & Hollins, 2005; McRitchie, McKenzie, Quayle, Harlin, & Neumann, 2014; Wiese, Stancliffe, Dew, Balandin, & Howarth, 2014). Staff may not recognize service users' needs and abilities to grieve, thus increasing the likelihood that service users will experience disenfranchised grief, defined as grief not recognized by others (Handley & Hutchinson, 2013). In some cases, staff may misunderstand service user manifestations of grief as being psychosocial problems ( Fahey-McCarthy, McCarron, Connaire, & McCallion, 2009). Staff may encounter challenges in knowing how much information should be disclosed to service users to help them understand the loss (Hahn & Cadogan, 2011; Reynolds, Guerin, McEvoy, & Dodd, 2008).
Although some existing training helps teach staff about service users' experiences of loss and how to support service users experiencing grief (Borsay & Critoph, 2012; Hahn & Cadogan, 2011; Reynolds et al., 2008; Watters, McKenzie, & Wright, 2012), staff have indicated the need for more training in these areas (McEvoy et al., 2010). For example, staff have desired more background information on bereavement and guidance on how to talk about death with service users of different cognitive levels and communication needs (McEvoy et al., 2010).
Additionally, staff can benefit from training on helping service users move on postloss, though no specific postloss bereavement training has been identified in the IDD field. Restoration processes, for example, present in grief theories such as Stroebe and Schut's Dual Process Model (Stroebe & Schut, 1999, 2010), focus on the bereaved individual's adopting new roles, making new friends, and learning new skill sets in healing from a loss (Stroebe & Schut, 1999, 2010). Restoration-focused staff training can help service users build a new life posloss, considering that many service users are dependent on others (Brickell & Munir, 2008).
In addition to supporting service users with their grief, staff also need training on how to manage their own grief, though few trainings include such guidance or are not evaluated in the peer-reviewed literature (Clearbrook, 2010). The benefits of providing staff with support regarding their personal grief issues is evident. Staff reported challenges in balancing their professional responsibilities and personal grief feelings (McEvoy et al., 2010) and expressed a need for self-awareness and support in navigating the coping process (McEvoy et al., 2010). Bennett (2003) developed a loss and bereavement educational intervention for staff supporting service users in a community group home. The intervention included staff sharing grief experiences, a theoretical review of Worden's (1991) four tasks of grieving, a focus on care applications, and helped staff to reflect on their feelings of loss (Bennett, 2003).
Organizational factors, such as policies and resources, also are important for staff and service users to heal from their grief (MacDermott & Keenan, 2014). Examples include policies allowing staff to attend a funeral or organizing an informal memorial service in-house. Staff might need time off from work, for example, to grieve such losses (McEvoy et al., 2010).
Furthermore, little focus has been placed on the utility of different types of training methods for bereavement training in the IDD field. Existing training generally reviews the theoretical models of grief, but focuses less on practical applications (Bennett, 2003). Staff have acknowledged the need for more hands-on tools, including video clips and handouts (McEvoy et al., 2010).
In summary, greater emphasis needs to be placed on training staff on how to support service users during and postloss. Few trainings teach staff on managing their own grief issues. Furthermore, limited attention has been placed on the effect of organizational resources and policies on service user and staff grief processes, as well as what training delivery methods might be most effective. Such gaps warrant a comprehensive review of what content areas and delivery methods should be incorporated in a staff bereavement training for the IDD field. In this study, the Delphi technique was exercised with an expert panel as informants about such content areas and delivery methods (de Villiers et al., 2005; Hasson & Keeney, 2011).
The Delphi technique can be used to obtain consensus on a particular topic of interest on the basis that experts' knowledge can be pooled with respect to the topic (de Villiers et al., 2005). In the current study, the Delphi technique utilized a panel of experts and online survey data collection to solicit consensus and feedback on staff grief and bereavement training development in the IDD field. Classical and e-Delphi designs were employed. With the classical Delphi design, an expert panel prioritizes information through responding to several rounds of survey data collection (de Villiers et al., 2005; Hasson & Keeney, 2011); the e-Delphi design is administered through an email or online web surveys (Hasson & Keeney, 2011). Central features of the classic Delphi design include participant anonymity, feedback provided to the participants, opportunities for participants to alter their responses over the course of the rounds, and statistical analysis of the group data (Rowe & Wright, 1999). Key aspects of the e-Delphi design are its convenience, allowing for participation from remote locations, and participant privacy in responding (Hasson & Keeney, 2011).
Although there is no agreed-upon sample size of expert panel members, the final sample size does impact the data quality (Hsu & Sandford, 2007). The sample's expertise affects the overall validity of the Delphi process (de Villiers et al., 2005). The Delphi technique should have ample follow-up to assist the consensus-building process. Ten to 15 people may be sufficient from a more homogeneous group, or five to ten from selective professional classifications (de Villiers et al., 2005; Linstone & Turoff, 1975). The Delphi technique does not require expert panels to be representative for statistical purposes (Powell, 2003).
Although opinions of too few expert panel members may inadequately represent the key issues, too many expert panel members can lead to low response rates and difficulty reaching consensus; groups larger than 30 members generally have not been efficacious (de Villiers et al., 2005; Hsu & Sandford, 2007). Panel members are experts selected on specific criteria, including knowledge of the topic area, good communication skills, research and field-based experience, and ample time for participation (Adler & Ziglio, 1996; Hsu & Sandford, 2007). The present Delphi study panel included 18 expert panel members with expertise in formal and informal disability caregiving, nursing, or hospice and bereavement service provision, meeting the criteria of five to 10 members from a professional classification (See Table 1). Additionally, four members currently work in academic research settings. Panel members were chosen mainly for their field-based and research expertise in the given areas. Individuals with such expertise who expressed interest in the research study were asked to participate. Recruitment occurred through professional contacts with local service providers and at professional conferences.
The majority of the members were female (n = 14, 78%) and White, Non-Latino race/ethnicity (n = 15, 83%). Most members had earned a graduate professional degree (n = 10, 55%). Geographically, they represented a few U.S. states and one international country. Their tenure working in their respective fields was approximately 18 years.
All expert panel members were asked to respond to three online questionnaires related to prioritizing training content areas and delivery methods. The response rates for the three rounds of data collection were as follows: 100% (18/18) in round one, 83% (15/18) in round two, and 100% (18/18) in round three. To achieve a high response rate, the second author sent out reminders to encourage expert panel members to respond. The study was approved by the University's Institutional Review Board. All expert panel members were required to review a consent form prior to study participation, and confidentiality of the expert panel members was respected.
Delphi questionnaires can be based on a review of the literature or an expert panel's responses to open-ended questions (de Villiers et al., 2005). For this study, a thorough review of the literature and focus groups with direct care workers in the IDD field guided the questionnaire development (Gray & Abendroth, 2015). The content areas included (a) service users' understanding and experience of loss, (b) ways that staff can help service users grieving a loss, (c) ways that staff can help service users to move on postloss, (d) staff professional challenges with grief and bereavement, (e) organizational factors (e.g., policies and programs, such as organization-sponsored memorial services), and (f) preferred training delivery formats. See Table 2 for a complete listing of content areas and items.
In each round, expert panel members were provided with project updates. More than three rounds have been found to be unsuccessful due to time, cost, and panel member fatigue (Hasson, Keeney, & McKenna, 2000; Powell, 2003; Skulmoski, Hartman, & Krahn, 2007). In this study, expert panel members were asked to respond to a four-point Likert-type rating scale (not very important, not important, important, very important) to gauge their level of agreement.
Although literature identifies a variety of ways to reach consensus, convergence should be achieved, or at least approached, by the final round (Powell, 2003). There is no single, agreed-upon interpretation of consensus indicated in the literature (Gibson, 1998; Hartley, 1995). Statistical central tendency measures (e.g., mean, median) and indicators of dispersion (e.g., standard deviation) tend to be preferred (Hasson et al., 2000). In the current study, decision rules were determined to be a mean of 3.5 or greater (de Villiers et al., 2005) and a median of 3.25 or greater (Hsu & Sandford, 2007). The term “full consensus,” used in the results section, refers to consensus being achieved for both the mean and median measures. As recommended, responses across the three rounds were monitored to determine their stability (Scheibe, Skutsch, & Schofer, 1975).
In addition to the measures of central tendency, feedback from panel members is reported to provide additional context. Within each of the broader topic areas, feedback quotations were analyzed for emerging themes and subthemes (Richards & Morse, 2007).
The concept of “trustworthiness” (i.e., credibility, dependability, confirmability, and transferability) (Lincoln & Guba, 1985) can gauge the rigor for the Delphi technique (Day & Bobeva, 2005; Hasson & Keeney, 2011). In this study, the iterative nature of the Delphi process and feedback techniques provided to participants positively affected the credibility of this study (Engles & Kennedy, 2007). Dependability was established by compiling a representative panel of experts (Cornick, 2006). Participants' job experience and credentials were queried to determine if the participants were qualified to serve on the expert panel. Confirmability was achieved through detailing the Delphi process for each of the three rounds with expert panel members through email and telephone correspondence. Transferability was established by demonstrating the applicability of the study findings (Kennedy, 2004; Powell, 2003).
Study results can be found in Table 2.
1. Service Users' Understanding and Experience of Loss
Consensus was achieved on only one item in this section (information provision on what information can or cannot be provided to the service user about loss) ( = 3.61, M = 4.00), but it did not maintain full consensus in rounds two or three. In response to feedback from this section (round one), an additional item was added: service users need to know the what, where, when, why of the loss. This item did not achieve consensus in follow-up rounds according to the mean and median criteria.
Feedback gathered from rounds two and three indicated that service users should have the option of learning about their loss. Delivery of such information should be tailored to the cognitive level of each service user. Euphemisms, for example, should be avoided.
2. Staff Helping Service Users Grieving a Loss
Several items reached consensus in all three rounds of this section (allowing service users to vent emotions due to loss (; = 3.78, M = 4.00), helping service users deal with difficult behaviors precipitated by loss ( = 3.56, M = 4.00), providing creative means for service user healing ( = 3.53, M = 4.00), and providing rituals for service user healing (organizational and individual) ( = 3.72, M = 4.00)). The items helping service users denying loss ( = 3.60, M = 4.00), helping service users re-experiencing loss (e.g., at anniversaries) ( = 3.50, M = 3.50), and helping service users with avoidant behaviors to loss ( = 3.53, M = 4.00) achieved full consensus in round two, although the latter two items did not maintain full consensus in round three. One item that did not achieve consensus in any round was helping service users break bonds with the deceased. Expert panel members cautioned against this, as one expert panel member expressed, “I don't want them to break any bonds with the deceased—maybe build new ones instead.” Expert panel members spoke of “grief and loss” as “highly personal experiences” and emphasized staff developing relationships with service users experiencing the grief process.
3. Staff Helping Service Users to Move on Postloss
Two of the four items in this section reached consensus. The item making new relationships achieved consensus in round three ( = 3.67; M = 4.00) with mean criteria and consensus in round one ( = 3.39; M = 3.50) with median criteria. The item learning new things achieved consensus in round three ( = 3.50; M = 3.50) with mean and median criteria. Neither of the other two items (developing new roles or learning a new job or skills) achieved consensus.
Expert panel members emphasized the importance of service users developing relationships, such as through group activities or various formal activities (e.g., bowling). Learning new things also seemed to be a priority.
4. Staff Challenges Regarding Grief and Bereavement
All three items in this section achieved consensus in round one. This included the items staff coping skills regarding loss and grief ( = 3.83, M = 4.00), lack of information on grief and bereavement ( = 3.67, M = 4.00), and lack of staff opportunity to participate in rituals (e.g., funerals, informal memorial services) ( = 3.50, M = 4.00). The latter two items did not maintain full consensus, however, for rounds two and three.
It was recommended that staff receive proper mental health training regarding their experience of loss and that staff complete a “loss assessment” to enumerate the range and types of losses weathered. Rituals were viewed as necessary for gaining closure and moving on.
5. Organizational Factors
Two of the four items achieved clear consensus under the category of organizational factors, such as organizational memorial services and policies assisting staff and service users with the grief and bereavement process. Both the items professionals available to assist staff and service users with the grief process ( = 3.56, M = 4.00) and staff trained to guide service users through the grief process ( = 3.61, M = 4.00) achieved consensus in all three rounds. The item policies to facilitate staff and service users with the grief process (e.g., staff time off, organization-sponsored memorial service) did not achieve consensus using mean measures, but did achieve consensus using median measures in rounds two and three (M = 4.00 [round two]; M = 3.50 [round three]). The item policies to facilitate service users' transition to a new life, identity, etc., postloss (e.g., new job opportunities) did not achieve any consensus.
Expert panel members spoke of the importance of staff receiving help from grief counselors to assist with grief challenges. They also recommended that staff receive training to support service users through the grief process. Those in supervisory roles voiced concern that staff might take advantage of such policies, such as taking time off for non-bereavement purposes.
6. Preferred Training Delivery Formats
Items achieving consensus on training/tool delivery format for staff included interactive exercises, both for in-person format in all three rounds ( = 3.71, M = 4.00) and the online format in round three ( = 3.67, M = 4.00). In-person case studies gained consensus only in round two ( = 3.40, M = 4.00), with the median calculations, but online case studies did not achieve consensus. Delivery format modalities using lecture, self-tests, or discussion boards did not achieve consensus.
Expert panel members spoke of the interactive exercises as a format to “engage [the] learner.” Case studies were perceived as helpful, with stories that staff could relate to without having to directly deal with personal issues and concerns. Concern was raised for staff with limited reading and writing skills being inhibited from using discussion board activities. With respect to online modalities, expert panel members indicated that it may not be an adequately supportive nor safe learning environment for teaching about grief and bereavement.
The purpose of this paper was to use the Delphi technique (de Villiers et al., 2005; Hasson & Keeney, 2011) to obtain expert panel consensus to prioritize the content areas and delivery methods for grief and bereavement training development in the IDD field. Findings can be used for the purpose of instruction when designing grief and bereavement training in the IDD field.
In particular, expert panel members arrived at consensus on the content areas of ways to help service users who are grieving and ways in which they can heal from loss. For example, staff could benefit from training on what information to tell and not to tell service users about the loss. Inherent in this process is how staff can learn to identify service users grieving a loss and how best to support them. Additionally, panel members agreed that staff need help to manage their own grief. Doing so also will help them to best support the service users. Importantly, findings show that organizations must institute policies and provide resources to facilitate the grief processes of both service users and staff.
Though consensus was not reached on the content area of to what extent a service user understands loss, it was reached on the content area of what information should or should not be provided to service users with respect to the loss. This can help to ease the service user's experience of loss. Additionally, to support a service user grieving a loss, proper communication skills could help service users vent emotions or deal with difficult behaviors precipitated by loss. Creative modalities or rituals for healing, which can manifest in multiple forms such as storyboard formats (Read, Nte, Corcoran, & Stephens, 2013) also were identified (Tuffrey-Wijne & Bernal, 2003).
Regarding the content area of supporting service users to move on post-loss, expert panel members agreed that staff could help service users develop new relationships and learn new things. Such modalities were viewed as being relatively easy to incorporate as part of the healing process, unlike adopting a new role or learning a new job skill.
Although staff experiences with grief and bereavement are addressed in several studies (Handley & Hutchinson, 2013; McEvoy et al., 2010; Watters et al., 2012), this content area tends not to be prioritized in bereavement training. Staff may not be supported adequately regarding their emotional difficulties regarding bereavement (Handley & Hutchinson, 2013). Expert panel members agreed that staff need help in developing adequate coping skills to manage their grief. Recommendations included staff completing a “loss assessment” to enumerate the range and types of losses staff have experienced on the job: this would help to determine what types of training and resources staff need.
Although much training has an individual or micro-level focus, organizational policies at the macro-level also are critical to help service users in managing their grief (McEvoy et al., 2010). For example, expert panel members agreed that staff and service users should receive support from mental health professionals in managing grief; similarly, organizational policies and protocol should be utilized to help staff and service users manage the grief process.
Expert panel members concurred that interactive exercises and case studies are good ways to engage staff as directly as possible in the training materials. Similar formats have been recognized in other studies (McEvoy et al., 2010). Expert panel members raised concerns regarding the reading level of the curriculum materials being accessible as well as not isolating online learners with heavy subject matter.
Despite the benefits of using the Delphi technique, limitations must be considered. The Delphi design incorporates variability in expertise among panel members. Some members may not be knowledgeable or possess experience in some content areas (Hsu & Sandford, 2007). Time constraints and the fact that some panel members did not respond in all three questionnaire rounds impacted the response rates.
Future Directions for Research and Practice
Study recommendations have been summarized in Table 3.
This study suggests a variety of directions for future research and practice. To help service users grieve a loss, staff need to know how to identify which service users are experiencing grief. Developed training can make use of existing grief assessment instruments (Blackman, 2008) to assess a service user's level of grief postloss (e.g., death of a family member or staff member). Training tools (e.g., checklists, fact sheets) also must be developed to help staff identify service users who are experiencing grief due to events related to losses (e.g., an anniversary). Training can help staff become knowledgeable about how service users understand grief and the best ways to communicate with them about a loss, such as through creative ways to express grief and communicate feelings (Read et al., 2013; Tuffrey-Wijne & Bernal, 2003).
In addition to supporting service users, staff often experience loss in the workplace. There is a clear need to support staff in navigating their own feelings of grief. Additional organizational supports are necessary, including policies that might allow staff to take time off following a loss. Bereavement time off could give staff the option of taking time off from work to process their grief. This study, however, did identify that there may be diverse opinions among staff members regarding this issue, depending on their managerial level. For example, managers voiced concerns that direct care staff may abuse opportunities to take time off for bereavement-related purposes, whereas direct care staff did not. Furthermore, it is important that measures are taken to ensure the care and safety of participants in online training when learning about the difficult topics of grief and bereavement in a rather isolated environment. One way to do so could be through adding interactive group discussions or one-on-one coaching in the online learning format.
Additionally, further assessment of staff bereavement training in the IDD field is needed. Not only should pretests and posttests be completed, but longitudinal assessments of training effectiveness also should be conducted (Watters et al., 2012). This will help to assess the longer term effects of the training. Additionally, adequate assessment is needed regarding the effectiveness of different types of training modalities. Training formats might be tested to determine whether online training formats are as effective as in-person formats. Training exercises can be assessed on whether they assist with staff information retention and in-the-field applications.
This study was funded by a Northern Illinois University College of Health and Human Sciences grant. This study was approved by the Institutional Review Board of Northern Illinois University (HS13-0149). The funding body has placed no restrictions on free access to or publication of the research data. The content is solely the responsibility of the authors and does not necessarily represent the official views of Northern Illinois University. There are no author conflicts of interest related to this manuscript. We would like to acknowledge the support and time of Jinsook Kim, Aimee Prawitz, Jarrett Wolske, and the entire expert panel.
We would like to acknowledge the support and time of Jinsook Kim, Aimee Prawitz, Jarrett Wolske, and the entire expert panel.