ABSTRACT

A robust preparedness framework is essential to maximize response effectiveness and safeguard success. Exercises are an integral part of oil spill preparedness and emergency management. Exercises enhance the capability of an organization to respond by validating plans and procedures and seeking affirmations on assumptions and stakeholder expectations. Since planning and preparing for exercises can be costly and time consuming, it is important that lessons learned are accurately captured, widely communicated and implemented in a timely manner so that they can lead to measurable improvements in the way we prepare and respond to incidents.

On average, international Oil Spill Response Organizations (OSRO) may be involved in thirty to forty oil spill response exercises in a given year, ranging from small scale exercises testing communication between various stakeholders to highly complex, multi-agency, multi-jurisdictional scenarios. Participation in multiple exercises in various capacities ranging from planning, designing and facilitation to participation, performance and capability assurance as an Oil Spill Response Organization, presents varied opportunities to view and analyze exercises from both ends of the spectrum, as a planner as well as a participant. A meta-analysis of these exercises offers a unique opportunity to systematically identify recurring areas for improvement and to examine new avenues to explore and test capabilities with operators who have established robust exercise programs. A key outcome of this would be to translate learnings into important enhancements for industry’s collective response and preparedness capability.

This paper presents a meta-analysis view of recent OSRO-industry exercises with a focus on a comparative assessment of specific After Action Reviews (AAR). The purpose is to identify common themes in terms of lessons learned and areas for improvement, especially when designing and planning future exercises. It also considers whether exercises have evolved to incorporate changing response capabilities and realistically testing any increased information requirements of the stakeholder community. Finally, it will also examine how successfully companies have sought out emerging technologies that will positively impact preparedness, response and recovery. It is anticipated that the recommendations put forward from this meta-analysis will assist operators in realizing the full benefits of the time and effort invested in exercise programs.

INTRODUCTION

While prevention and the mitigation of risk are key guiding principles relating to potential safety, security, health and environmentally-related (SSHE) incidents, emergency preparedness and response (EP&R) efforts are equally important aspects of a holistic view of the risk-management world that considers both the antecedents and potential consequences of an incident. (Figure 1).

Figure 1:

Schematic of Prevention and Response Aspects of a Potential SSHE Event

Figure 1:

Schematic of Prevention and Response Aspects of a Potential SSHE Event

An important part of the EP&R focus is on regular and meaningful response exercises as a means to help improve and sustain organizational capabilities, especially as personnel move into different roles. With this in mind, and especially with respect to many large organizations, it is necessary to ensure that timely and effective response capabilities are maintained and that response personnel proficiency is supported through an ongoing, recognized training process, e.g., an approach that is consistent with an Incident Command/Management System (ICS/IMS) or International Maritime Organization (IMO) framework. (IPIECA-IOGP 2015; FEMA 2011) For example, the IMO has been working to revise training modules to “provide up to date guidance for preparedness and response to marine oil spills.” (IMO 2017) A number of key components associated with training can be effectively obtained during a variety of exercises, and depending on the scope of the exercises, they can be used to: (API, 2014)

  • Clarify team roles and responsibilities

  • Improve response coordination and communications internally and externally

  • Assess and validate policies, plans, procedures, training, equipment, assumptions, and contract or mutual aid agreements

  • Identify potential gaps in the resource supply chain

  • Identify various opportunities to improve planning efforts and enhance future exercise programs

A variety of potential response exercise objectives for a broad suite of response personnel are provided in Table 1 below. (IOGP-IPIECA, 2014)

Table 1:

Examples of Programme Objectives

Examples of Programme Objectives
Examples of Programme Objectives

Focused on the Emergency Response Process & Organizational Structure

  • Utilize National Oil Contingency Plan

  • Utilize company Contingency Plan for Abatement of Marine Oil Spills

  • Employ incident management team (IMS) & organization structure

  • Develop & implement site safety & health plan

  • Develop & implement incident action plan (IAP)

  • Ensure Incident Commander clearly states incident response priorities

  • Implement improved planning session & meeting effectiveness processes

  • Enable effective response team intra-section communications

  • Enable effective national inter-agency communications

  • Optimize the use of oil industry & government agency personnel diverse skills

  • Demonstrate effective integration of industry representatives and government agency personnel

Focused on Oil Spill Response Effectiveness

  • Activate & operate on-scene incident command center

  • Identify & prioritize protection of environmentally sensitive areas (ESA’s)

  • Identify & prioritize protection of economically important areas (EIA’s)

  • Implement the national oil spill dispersant use approval process

  • Exercise oil spill dispersant delivery logistics, deployment and monitoring capabilities

  • Exercise ability of Oil Spill Response Organization (OSRO) to obtain response equipment customs clearances in a timely fashion

  • Develop effective oil spill response containment, recovery, and cleanup work plans

  • Develop effective oil spill response equipment logistical, deployment and demobilization plans

  • Identify & address incident-related security concerns

  • Implement claims management & settlement process and establish hot line

  • Manage & document daily and cumulative oil spill cleanup costs

Focused on Information Management & Communication Effectiveness

  • Ensure Incident Commander clearly communicates key external messages to Public Affairs and the Public Information Officer and that press releases are consistent with those messages

  • Conduct initial & periodic response team incident status update briefings

  • Implement process for documentation of key response team actions & decisions

  • Document key response team decisions using appropriate tools

  • Communicate exercise status through use of Public Affairs internet site

  • Manage media & community expectations and concerns as a joint local and regional effort

  • Document exercise learnings, next steps, and gap closure plans

Having well defined objectives at the outset of an exercise is important, but it is equally important to compile lessons learned at its completion. In particular, the identification of possible areas for improvement while fresh in everyone’s minds is of value, whether developed from the written evaluator feedback or as part of section member debriefs (the “hot wash”) and After Action Reports (AAR), since the items are often the most relevant and most readily acted upon by the organizers and participants. The following section examines several representative topics that were identified during a number of post-exercise evaluations and offers some perspectives on the comments.

Value of After Action Reports and Recommendations for Future Exercise Enhancements

Since section debriefs are brainstorm-type indicators, they may not include the context applicable to specific items unless care is taken to document as much supporting information as possible during the ensuing discussion. Therefore caution should be exercised when reviewing these items since, without any specifically described context, they may lead to incorrect assumptions and conclusions. For those areas of improvement that have context and supporting documentation, they may be addressed, and possibly used, as a basis for refining objectives during future exercises.

With regard to lessons learned, feedback received immediately following an exercise during the well accepted “hot wash” process can be helpful, but are generally not able to go into depth because of potential time constraints and may not indicate any specific major gaps that exist with respect to exercise processes. Any significant items identified in the feedback sessions or by evaluators may be addressed or assigned by individual Section Chiefs as appropriate.

As a sampling of the kinds of comments that are received, the following examples represent some of the more common themes identified from debriefs and feedback that could warrant further action. They are not meant to be all inclusive by any stretch of the imagination, since the range of comments can go from how well the printers and internet connections work, e.g., “it was difficult to operate computers due to internet connectivity issues,” to the amount of room provided for each section, to how well Operations and Planning members were able to interact.

OBSERVATION: Unified Command - Situational Awareness / Media Availability

“Unified Command should make a concerted effort to provide frequent, timely and effective situation updates to the entire command center. A more formal and more comprehensive situation briefing on day 1 in the command post, during the early transition phase from local support to those coming in from elsewhere, should be conducted to make sure everyone is on the same page.”

Comment: It’s not unusual for responders and command center personnel who are on scene early in a response/exercise to get caught up in the emergency phase, i.e., the Initial Response of the “Planning P” (Figure2), and potentially fail to recognize that not all personnel in the room have the benefit of the incident history and current status. It’s important bring all personnel up to speed as effectively as possible, and inclusive briefing sessions have been found to be effective.

Figure 2:

The Planning P (FEMA, 2011)

Figure 2:

The Planning P (FEMA, 2011)

With respect to external stakeholders, Unified Command should consider making themselves available for longer media appearances to provide sufficient information and respond to media and other stakeholders (i.e., self-imposed 30 minute time limit during media events increased the level of frustration among those attending.

Comment: While this may be true, Unified Command has a number of responsibilities that demand their time. Other possible solutions that have been found to work include having representatives from the UC address general questions within a set timeframe and providing for others such as Technical Specialists to remain and address more specific, potentially operational questions.

OBSERVATION: External Positioning / Managing Public Information Demands

“The Joint Information Center (JIC) should make all efforts to stay ahead of the curve (i.e., warnings for land impact on potentially sensitive shorelines). Earlier development of strategies to manage expectations and meet commitments (i.e. to give hourly updates by leadership) should be a focus of the JIC.”

Comment 1: The JIC is a critical component of an emergency response since they are often the filter through which stakeholders will likely receive information. It’s not unusual that there will be a number of comments relating to how well the JIC functioned. This is often a thankless task that can be fraught with all sorts of criticism. However when it is done well, and it often is, comments such as the following may result:

  • The talking points developed for the press conference were very good and the press conference was well conducted.

Comment 2: The use of social media started well, and was expanded as the exercise continued – to the point where it was quite comprehensive. The JIC did a great job capitalizing on all available tools: tweets, PRs, talking points, website, press conferences, town hall meetings, blogged, reaching out to stakeholders by scheduling working lunches, etc. The goal was to keep stakeholders informed. This was a breakthrough effort that was extremely well planned and provided value.

OBSERVATION: Design Constraints Associated with an Emergency Response Exercise

“The exercise design should test aspects of the response beyond the first 2 days since there is an inherent limitation to only examining the first operational periods of a response. They are usually very focused on the emergency of the situation and don’t allow for a longer view of other focus areas.”

Comment: It is general the case that many exercises are designed to test response organizations during a simulation of the first two days of a response. Focus areas include Tier I response capabilities and the integration of initial responders, e.g., local Strike teams, with larger spill management teams, since these components can be viewed as mission critical during an emergency response. If this is not managed well, it may be expected that the ongoing, developing stages of a spill response could become derailed. Additionally, an important early-stage requirement of a spill response/exercise is the preparation of an “Incident Action Plan” since this defines the key plans for the upcoming response period. If this is not handled well and completed in a timely fashion, responsible party credibility and response sustainability may suffer a significant set-back. While it is probably the case that there is room for improvement in examining the later stages of a spill response, it is difficult to test these in a realistic sense when considering the constraints of time and personnel resource, i.e., exercises are of finite and relatively short duration.

A potential recommendation is to identify specific issues around the longer term aspects of a spill response and develop additional, focused exercise protocols that can be used to test or validate processes and procedures in a tailored exercise format for specific sections and section interfaces (i.e., Operations/Planning; Operations/Logistics, etc.) with specific timelines in mind.

OTHER CONSIDERATIONS

Real world incidents can also certainly be used as a basis for providing inputs to a robust AAR process, especially if exercise personnel have participated in both. In reality, inputs related to or derived from an actual incident can serve as a reality check during the exercise-related debriefs since they can serve to highlight potential best practices, improvement opportunities, or actual experiences with other potential gaps and how they were managed. This also serves to point out the value of identifying other types of information that may come out of an emergency response or exercise. These may include lessons learned, good practices, or other categorizations. A key aspect is how to document what is identified and/or learned so that others may benefit from knowing where to find them, since all too often, lessons are not learned and sub-optimal performance may be repeated. (Chopra, 2015) For example, the US Department of Homeland Security (DHS), and presumably other international governmental organizations, maintains a “Lessons Learned Information Sharing” system in order to disseminate information across the emergency response community. Specific categories to be documented included are shown in Table 2 . (DHS, 2007)

Table 2:

Examples of Lessons Learned Information Sharing Categories

Examples of Lessons Learned Information Sharing Categories
Examples of Lessons Learned Information Sharing Categories

DISCUSSION

Again, drawing from the view of the US Federal Emergency Management Administration (FEMA), “Planning makes it possible to manage the entire life cycle of a potential crisis. Strategic and operational planning establishes priorities, identifies expected levels of performance and capability requirements, provides the standard for assessing capabilities and helps stakeholders learn their roles.”1 The Evaluate/Improve component of the cycle (Figure 3) is a critical component of the continuous improvement process, a key feature of most safety and risk-related efforts.

Figure 3:

FEMA’s Example of the Preparedness Cycle

Figure 3:

FEMA’s Example of the Preparedness Cycle

As indicated by actual examples of AAR comments and suggestions for areas for improvement, a number of suggestions are actionable, but may not result in substantial enhancements of exercise outcomes, e.g., provide better printer capability. Other potential actions may provide significant improvements but may ultimately prove to be difficult to implement, e.g., expand the time period being tested from one operation period to multiple periods. The purpose of this discussion is to help focus efforts on those actionable suggestions that may result in the most significant enhancement during the preparedness cycle feedback process.

The response community is generally quite interested in providing as much value as possible so it can be reasonably considered that lessons learned and suggestions for improvements will be considered seriously. By continuing the process of AAR and looking for those highest value-added items from a broad suite of actual exercises, the evolution of exercises within the preparedness cycle will continue. Significant resources (time, people, expenses) are committed to large-scale exercises and there is a large effort to ensure that efforts are not sub-optimal. While the regulatory aspect of the evaluation process should not be minimized, since exercises are designed to test specific goals and objectives, additional value can certainly be gained if a robust process for exercise evaluation is established (Table 2 ). (IOGP-IPIECA, 2014)

Table 2:

Key components to be considered for effective programme evaluation

Key components to be considered for effective programme evaluation
Key components to be considered for effective programme evaluation

Some major themes that were identified during the review of AARs from a variety of exercises included:

  • Effective Integration of Response Teams During the Early Stages of a Response

  • Effective Communications between a Variety of Stakeholders

  • Proactive Engagement with External Stakeholders and Management of Expectations

  • Expansion of the Timescale of an Exercise beyond the Emergency Phase

Because of the potential impact that most of these would have if not handled well, it is not surprising that they often rise to the top during hot wash and AAR discussions. In most cases, the prospect of a quick hit, one time fix is not very likely and it is important to involve engaged individuals in the process of working to enhancements and solutions. It is an ongoing process that requires personnel who have a vested and potentially long term interest. This can be a challenge since many exercise participants have other “day jobs” that may make it difficult to commit significant amounts of time to the endeavor, but that does not mean that the attempt should not be made.

CONCLUSION

This paper has attempted to present an analytical view of recent OSRO-industry exercises with the goal of considering the types of comments and recommendations that result from After Action Reviews (AAR). The purpose was to find some level of thematic commonality of those reported by participants and evaluators with the intention that future exercises can benefit from continuous improvement. It is clear that there is a relatively small number of major themes that should be considered with an eye toward focusing on actionable suggestions. By considering the more significant findings, and their impact on the perceived success of an emergency response exercise by potential stakeholders, the value of the exercise can be extended beyond that of meeting the particular objectives of a specific event. By keeping an open mind and potentially embracing new approaches or the use of emerging technologies, a positive impact on preparedness, response and recovery may result.

It is hoped that by considering actionable recommendations that come out of the AAR process that the spill response community will be able to capture the benefits that result from going through the efforts associated with designing and running spill response exercises. As with many processes, it is the feedback process, and its incorporation into future planning for exercises that helps to enhance the overall value to its users.

REFERENCES

REFERENCES
API
,
2014
Guidelines for Oil Spill Response Training and Exercise Programs Guidance for Spill Management Teams and Oil Spill Responders
,
Chopra, 2015
Chopra
,
A.
,
Are Lessons Truly Learnt?
,
Interspill Conference
,
Amsterdam
,
2015
.
DHS
,
2007
Homeland Security Exercise and Evaluation Program (HSEEP) After Action Report/Improvement Plan
.
FEMA
,
2011
See for example training-related resources
IMO
,
2017
Sub-Committee on Pollution Prevention and Response (PPR), 4th session
,
16–20 January 2017
.
IPIECA-IOGP
,
2014
Oil spill exercises Good practice guidelines for the development of an effective exercise programme, IPIECA-IOGP, 2014
.
IPIECA-IOGP
2015
Oil spill preparedness and response: an introduction, IPIECA-IOGP, 2015
.