Improving Safety for Gulf Oil Spill Responders: Individual and Organizational Factors Impacting the Effectiveness of Health and Safety Training

On April 20, 2010, the BP Deepwater Horizon oil rig exploded in the Gulf of Mexico causing a massive oil release. The event directly affected communities in four Gulf States: Louisiana, Alabama, Mississippi, and Florida. In response to the disaster, the National Institute of Environmental Health Sciences (NIEHS) facilitated the presentation of health and safety training and development of training materials for over 147,000 cleanup workers (e.g., on-shore and off-shore volunteers, technical specialists) (NIEHS, 2010, 2012). Of those trained, approximately 47,000 workers engaged in cleanup activities. One of the key aspects of the Gulf of Mexico oil spill (GOS) training effort was an emphasis on evaluating its effectiveness and impact (NIEHS, 2012). This manuscript highlights the evaluation process and reports findings concerning the effectiveness and impact of the training.

The health and safety training literature has long recognized the need for comprehensive, systematic evaluations of the effectiveness of training with respect to increasing both safety knowledge and exhibiting safe work behaviors, while reducing outcomes such as accidents, illnesses, and injuries (e.g., see Ford & Fisher, 1994; Burke and Sarpy, 2003). In relation to such calls in the literature, Sarpy and colleagues (see Sarpy, Chauvin, et al., 2003, 2005) have developed a rigorous process for evaluating the effectiveness of health and safety training programs that includes: (1) an integrated programmatic assessment incorporating elements of both process and impact evaluations; (2) qualitative and quantitative data from the major program stakeholders; (3) identification of lessons learned and best practices for continuous quality improvement; and (4) a standardized process documenting training program effectiveness and impact. In addition, the GOS Training Evaluation effort delineated that training be evaluated according to a four-level framework (Kirkpatrick, 1996) that examines training outcomes by measuring: reactions to the training program (Level 1); learning during training (Level 2); behavior following training (Level 3); and results of the training program in meeting intended outcomes (Level 4).

The GOS Training Evaluation effort was grounded in these frameworks so as to provide an accurate and thorough evaluation of the training as a whole. In this way, information could be synthesized to determine the extent to which the trainees perceived the training as useful, learned the relevant information, transferred this learning to cleanup sites via the exhibition of safe work behavior, and, ultimately, whether the training led to intended outcomes of the training (reductions in accidents, illnesses, and injuries). The integration of information from the four levels provided critical information for ensuring quality assurance and quality control of the training (Alliger, Tannenbaum, et al, 1997). As such, the present study centers on the workers who received GOS training and assesses their subsequent reactions, perceived learning, on-the-job performance, and health and safety outcomes.

Participants. Of the more than 147,000 workers who received training across four Gulf States affected by the oil spill (i.e., Alabama, Florida, Louisiana, Mississippi), approximately 47,000 workers were actively involved in the cleanup activities. All workers (i.e., on-shore volunteers, off-shore volunteers, technical specialists) who received GOS trainings and/or worked in the GOS environmental cleanup activities were potential study participants.

Training Intervention. Several types of training were developed and delivered in modules according to the recommendations discussed above. The course offerings provided workers with information regarding safe work practices, personal protective equipment, decontamination, heat stress and other common hazards for cleanup work (NIEHS, 2010; OSHA, 2010). A complementary booklet highlighting the general training concepts also was developed and translated into Spanish and Vietnamese.

More specifically, Module 1 was the 30–45 minute BP health and safety orientation training for workers in non-contaminated environments (i.e., no contact with hazardous materials; pre-oil landfall beach cleanup). Module 2, a 45-minute orientation on BP procedures, was geared toward contracted employees who were engaging in any labor/work not involving spill-contaminated materials (e.g., site health, safety, and environmental orientation). The 4-hour Module 3 off-shore awareness training was developed for workers conducting cleanup offshore; whereas the 4-hour Module 3 on-shore awareness training was developed for workers cleaning contaminated shoreline environments. Module 3 off-shore is designed for captains or crewmen (i.e., marine cleanup workers) without previous clean-up experience who were working on Vessels of Opportunity involved in skimming, oiled boom, or controlled burns. Module 3 on-shore is designed for workers without prior clean-up experience who were picking up tar balls and other oil-contaminated debris on land. Module 4, which was the last training to be developed, was an expanded off-shore cleanup training that targeted vessel workers. Finally, the Hazardous Waste Operations and Emergency Response (HAZWOPER) training, a 5-day (40-hour) course, offered through NIEHS and OSHA, was offered to workers and supervisors with direct contact with contamination.

GOS Evaluation Survey (Worker Version). A 63-item worker version of the GOS Evaluation Survey was developed to evaluate training relative to four levels. Along with demographic information, workers were asked to indicate all GOS trainings attended, if certifications were received for those trainings, the state and location (on-shore, off-shore) in which they performed cleanup activities.

With respect to Level 1 outcomes, the survey required workers to rate their perceived satisfaction with various aspects of the training on a scale ranging from 1 ‘Strongly Disagree’ to 7 ‘Strongly Agree.’ The respondents also were asked to provide ratings on overall effectiveness of the training, content, format, and instructor. For Level 2 outcomes, the workers were asked to rate their perceived attainment of each of the training objectives on a scale ranging from 1 ‘Strongly Agree’ to 7 ‘Strongly Disagree.’ In addition to the ratings, workers were asked to indicate the content and/or skills learned that were most and least valuable to them as well as to provide suggestions for improvement to the training. With respect to Level 3 outcomes, the respondents were asked to rate the extent to which they demonstrated each of the safety-related training behaviors during their GOS cleanup activities on a scale from 1 ‘Never’ to 7 ‘Always’. The safety-related behaviors were classified into four major categories consistent with measures of general safety performance (Burke, Sarpy, et al, 2002): (1) Using Personal Protective Equipment (5 items); (2) Engaging in Work Practices to Reduce Risk (7 items); (3) Communicating Safety and Health Information (4 items); and (4) Exercising Employee Rights and Responsibilities (3 items). In addition to the ratings, workers were asked to provide explanations as to why they did not perform any of the safety behaviors while they participated in the GOS cleanup activities. Finally, to gather Level 4 information, workers were asked to provide any examples of accidents, injuries, or illnesses that they and/or their co-workers experienced during the GOS cleanup activities.

Along with English, the GOS evaluation survey was translated into Spanish and Vietnamese versions. Due to issues regarding accessibility of the training roster databases, a convenience sample was gathered using select NIEHS grantees (e.g., Boat People SOS, Bayou Interfaith Shared Community Organizing) to facilitate face-to-face administration of the surveys for workers who remained in contact with the community-based organizations. Approximately 650 surveys were distributed.

Data analysis. Five hundred seven completed surveys were returned to the evaluation team. Descriptive analyses (i.e., means, standard deviations, percentages) were calculated for the ratings using SPSS, with the data being analyzed separately to provide for comparisons between the worker sub-populations (i.e., self-reported race/ethnicity). Content analysis was conducted on comments to identify emerging themes and to make recommendations for improvement.

Worker Demographics. Among those workers responding, 74% of the survey participants were male whereas 26% were female. Among survey respondents, the following ethnicities were reported: Black 27%, White 23%, Asian (Vietnamese) 22%, Isleno 20%, Hispanic 6%, and Native American 2%. Vietnamese and Islenos are emigrants to the U.S. Gulf Coast region. The Vietnamese community developed as many individuals fled South Vietnam in the midi-1970s to escape the incoming Communist regime. The Islenos community developed as thousands of Canary Islanders (referred to as Islenos) left Spain in the late 1700s to build the Province of Louisiana and guard against the expected British invasion.

Worker Trainings. Module 1, orientation training for beach cleanup volunteers working in non-contaminated environments, was presented to 34% of the workers. Module 2, geared toward contracted employees not engaging in work involving spill-contaminated materials, was attended by 31% of the workers. The Module 3 off-shore awareness training for workers conducting cleanup offshore was attended by 45% of the workers whereas the Module 3 on-shore awareness training for workers cleaning contaminated shoreline environments was attended by 43% of the workers. Both of these trainings are directed towards individuals with no previous experience with this activity. Module 4, the expanded off-shore cleanup training targeted at vessel workers, was attended by 21% of the workers. Finally, nearly 60% of the workers reported attending the Hazardous Waste Operations and Emergency Response (HAZWOPER) training.

Worker Ratings of Training Effectiveness. The majority of workers indicated that the training was effective in each of the major facets of training, and in an overall sense. That is, respondents, on average, indicated that they ‘Agreed’ with each of the statements associated with excellence in training. The average item ratings also suggest that the workers felt the training content and format as well as the instructors presenting the training were effective (see Table 1).

Comparison of Ratings across Worker Subpopulations. As shown in Table 1, across Reaction items, Isleno workers, and Vietnamese workers to a lesser extent, rated the training less favorably than their White, Black, Hispanic, and Native American counterparts. In particular, the Isleno workers reported the greatest divergence from the other groups with respect to effectiveness of content and training overall.

Worker Acquisition of Training Knowledge and Skills. The majority of workers indicated that the training was effective in ensuring that they had the appropriate knowledge and skills as defined by the Gulf Oil Spill health and safety training objectives (see Figure 1). That is, the majority of respondents indicated that they ‘Agreed’ that they had attained the requisite knowledge and skills for safely engaging in the cleanup activities. More specifically, these items included: (1) Knowledge I: “I was knowledgeable about potential hazards and mitigation strategies associated with the Gulf Oil spill activities I performed”; (2) Knowledge 2: “I was knowledgeable about how to minimize potential exposure to hazards associated with the Gulf Oil Spill activities to myself, my co-workers, and the public”; and (3) Knowledge 3: “I was empowered to identify and avoid hazards while performing the Gulf Oil Spill activities.”

Comparisons of Training Knowledge and Skill Acquisition Across Worker Subpopulations. As depicted in Figure 1, the analyses revealed that workers' acquisition of training-related knowledge and skills did significantly vary according to the various worker subpopulations. In general, the Isleno workers, and Native American and Vietnamese workers to a lesser extent, rated their learning gains less positively than their White, Hispanic, and Black counterparts. These finding are consistent across the training-related knowledge and skills.

With respect to the content of the training, qualitative comments revealed that for the majority of respondents, safety awareness was the most valuable aspect of the GOS training (e.g., safety awareness, use of personal protective equipment and booms). However, the Isleno workers cited content issues related to specific applications of the training. For example, they reported that training provided examples of spills that were not applicable to the Gulf of Mexico oil spill clean-up (e.g., refinery examples) and that instructors lacked knowledge of local and regional issues. In addition, while a few workers cited particular curricular elements of the format (e.g., hands-on; use of visuals; instructor pacing) that created a positive learning environment conducive to their learning, issues centering on literacy and English as a second language were often reported as creating a barrier to training effectiveness for Vietnamese workers and, to a lesser extent, for the Hispanic workers.

Workers' Performance of Training-related Behaviors on the Cleanup Site. Collectively the workers most often reported that they ‘Always’ demonstrated the appropriate health and safety behaviors, as needed, with respect to: (1) Using Personal Protective Equipment (5 items); (2) Engaging in Work Practices to Reduce Risk (7 items); (3) Communicating Health and Safety Information (4 items); (4) Exercising Employees Rights and Responsibilities (3 items) (see Figure 2). Moreover, across these four performance dimensions, behaviors associated with the category of Engaging in Work Practices to Reduce Risk were among those training-related behaviors that were most often exhibited during the cleanup activities whereas those behaviors associated with Exercising Employees Rights and Responsibilities were among those least frequently reported during the GOS clean up activities.

Comparisons of Training-Related Performance across Worker Subpopulations. In general, the Isleno workers, and Vietnamese workers to a lesser extent, rated their safety performance, on average, less positively than their White, Hispanic, Black and Native American counterparts. To illustrate these differences, when asked to rate the extent to which the worker “practiced safe spill handling procedures”, the White (Mean=6.51), Black (Mean=6.75), Hispanic (Mean=6.50), and Native American (Mean=7.0) workers indicate that, on average, they ‘Always’ performed this training-related behavior, with the Vietnamese workers, on average, (Mean=6.0) responded that they ‘Almost Always’ performed this behavior. On the other hand, the Isleno workers (Mean=4.69) often reported that they ‘Sometimes (i.e., about 50% of the time)’ demonstrated this behavior.

Workers' Qualitative Comments Regarding Training-related Behaviors Performed During Cleanup Activities. Respondents indicated several reasons why they did not perform the training-related behaviors when participating in the cleanup activities. The largest percentage of respondents who indicated they did not perform the safety behaviors during the cleanup activities stated that they were not physically on-site to perform them (e.g., boat sat idle; did not get called into work), but that they had received training sufficient to ensure their ability to safely perform the clean up activities. A smaller set of workers reported that, although they participated in the cleanup activities, they did not have the opportunity to perform the safety behaviors because they did not come into contact with the oil or did not encounter an incident or accident on the work site. Lack of available resources accounted for the inability to safely conduct the cleanup activities for many of the respondents. This lack of resources ranged from a complete lack of safety equipment, to lacking various components such as masks, to receiving safety equipment after the cleanup activities had already started. Related, a number of respondents reported that they did not use the equipment because it was too cumbersome and that it interfered with their activities (e.g., gloves stuck to oil). Another subset of responses centered around supervisors and the influence of their behavior on the workers safety. For many of these workers, communication issues on the clean-up site interfered with safe performance (e.g., lack of emphasis on safety issues). Further, many of these workers felt that if they voiced their concerns, they would be fired (i.e., fear of reprisal). A second related category was lack of supervisory support, where safety did not take precedence over getting the work completed.

Worker Accidents, Illnesses, and/or Injuries. The vast majority of respondents did not experience any accidents, injuries, or illnesses (see Table 2). Of those reporting incidents, the most frequently cited outcome was self-reported illnesses. The respondents were typically able to link their unsafe behavior such as not using the appropriate Personal Protective Equipment with the illness such as breathing problems.

Taken together, the results generally suggest that the workers reported that the training was developed and delivered in an effective manner with respect to content and format and, in turn, positively enhanced their training-related knowledge, skills, attitudes, and environmental cleanup behaviors. Further, the vast majority of workers reported that that they did not experience or witness any accidents, injuries, and illnesses resulting from cleanup activities.

However, when considering results stratified by ethnicity/race of the workers, results suggest that workers from the sub-populations most vulnerable to the effects of the oil spill were not as positively influenced by the training. That is, the training was relatively less effective for those who workers who associate by cultural heritage (i.e., Isleno workers) as well as for those who associate by race (i.e., Vietnamese workers). To be salient for those workers who work and reside in the contaminated region, the training content must reflect local and regional examples and application. For workers with English as a second language, training format appears particularly important. Along with translators in the classroom, use of hands-on activities supplemented with discussion as well as greater emphasis on visuals and less emphasis on reading and written materials (unless translated) are suggested. Such training interventions could incorporate hands-on training activities and exercises supplemented with practice and follow-up discussions that provide specific examples relevant to the disaster work they will be engaging in within their communities. These recommendations are consistent with the literature on safety training (e.g., see Burke, Sarpy et al., 2006; Burke & Sockbeson, 2015; O'Connor, Flynn, et al., 2014).

Cultural influences including language and literacy are becoming increasingly recognized as important factors that should be taken into account in the development and delivery of worker safety training programs (Carruth, Levin, et al., 2010; Samples et al., 2009). In addition, researchers and public health agencies are prioritizing development of partnerships with community-based organizations to reach worker sub-populations, which are often non-English speaking and have low-educational levels (e.g, see Forst, et al., 2013). However, little is known about how such safety training affects the performance of workers within these sub-populations. This point is important as workforce mortality and severe injury rates are disproportionately distributed among foreign born and Hispanic workers in the United States (Byler, 2013), and the actions that workers engage in are recognized as direct antecedents to adverse health outcomes (Christian et al., 2009). The present study provides insight into the effect of safety training on the safety performance of various worker sub-populations in the Gulf South region.

Related, the results suggest that the social and organizational aspects of the worksite are an important factor of the process when engaging in the cleanup activities and should not be overlooked. For example, the disaster response included use of volunteers in addition to technical specialists for the environmental cleanup activity. As such, role expectations of workers and supervisors must be clear and explicit. When examining the workers' training performance it is also evident that the particular work environment and related safety climate/culture either supports or inhibits the use of safety behaviors taught in the training intervention. Managerial and organizational support were not consistent across work sites. Likewise, the emphasis on safety, availability of equipment and fostering of upward communication regarding safety issues without fear of reprisal were necessary to ensure that cleanup activities were performed safely.

The GOS Training Evaluation effort points to the need to consider worker health and safety training in a broader context; one that considers the entire training system (i.e., training needs, course objectives, delivery method, etc.) to ensure training effectiveness across worker populations. In this way, the systematic approach to training interventions can used for other natural and man-made disasters to be responsive to the changing needs and conditions in the recovery efforts and better ensure safety of the workers involved in those efforts. Collectively, the results of the GOS evaluation process can be used to ensure continuous quality improvement of safety training efforts and mitigation response.