Outdoor Recreation Damages from Oil Spills: Why Current Assessments are Typically Wrong and What can be Done to Fix Them

The goals of a natural resource damage assessment (NRDA) under the Oil Pollution Act of 1990 (OPA) are to restore natural resources and compensate the public for injuries arising from oil spills.¹ The dollar damages recovered from the responsible party (RP) by the natural resource trustees (Trustees) must be devoted to implementing restoration projects and reimbursing Trustee assessment costs. Based on this, three key questions in an NRDA are:

• (i) How much restoration is enough to compensate for injuries to recreation?

• (ii) What methods should be used to answer the first question?

• (iii) How much information of what type should be collected and analyzed to efficiently implement the answer to the second question?

The process for answering the first question is called restoration scaling and quantitative economic methods for undertaking it have been much discussed. The processes for answering the second two are less formalized. The issue is that losses from injury and gains from restoration are inherently uncertain and we desire to neither over nor under estimate restoration scale. More information leads to better estimates and reduces the risk of errors in scaling, but comes at a cost. Striking an appropriate balance between reducing risk of errors in restoration decision-making on the one hand and increasing assessment costs on the other defines an efficient NRDA process. We call efforts to strike such a balance “restoration risk management” (RRM). The goal of this paper is to contribute to enhancing the efficiency of the NRDA process by advancing formal RMM methods applied to human use injuries. The specific issue we address is the method used for scaling restoration and the trade-offs between (i) a method with a known bias, but less information-intensive and (ii) alternatives that may be unbiased but cost more.

The implementing regulations for OPA NRDAs provide guidance on methods choice, stating that: “The additional cost of a more complex procedure must be reasonably related to the expected increase in the quantity and/or quality of relevant information provided by the more complex procedure.” (15 CFR § 990.27(a)(2)). It is not at all clear how one can reasonably relate costs to the quantity or quality of information, which are not elsewhere defined. Our analysis provides a way to operationalize this guidance.

The regulations for OPA NRDAs also specify (at § 990.53(d)(3)(ii)) three methods for restoration scaling (for both human use and ecological impacts) that we call Value-to-Value (VTV), Service-to-Service (STS), and Value to Cost (VTC). When applied to recreation, these scaling methods proceed as follows.

• STS: Measure the loss in the number of recreation trips to injured sites (where the number of trips is the measure of recreation services) and then find restoration that increases trips to restored sites by an equivalent amount.

• VTV: Measure the monetary value of the loss in trips caused by the spill and then find restoration projects that enhance recreation opportunities and create trips with a monetary value equivalent to the loss.

• VTC: Measure the monetary value of the loss in recreation opportunities caused by the spill and then spend that much money on restoration.

According to the OPA NRDA regulations, STS must be considered and is the preferred scaling method. STS assessment assumes that the trips restored are of approximately equal value (per trip) to the trips injured and thus requires the availability of restoration projects that provide such trips. If these projects are not available, the regulations recommend the use of VTV scaling, which adds information on the relative values of trips injured and restored. VTC is least preferred; it is recommended only if VTV assessment is deemed cost-prohibitive.

Despite these regulatory preferences, VTC scaling is by far the most commonly used scaling method for oil spills. This is an issue because VTC almost always provides biased estimates of restoration scale, justifying our titular claim that damage assessments “typically are wrong.” VTC is biased because it does not incorporate the benefits of restoration in generating trips and economic values. If those benefits are larger (respectively smaller) than project costs, then too much (too little) is spent on restoration. The rationale for using VTC despite this bias is that the benefits of restoration are uncertain and incurring the cost of studies to reduce this uncertainty is not warranted. Hence, the VTC estimate, while biased relative to VTV scaling, is thought “close enough.” This begs the RRM question: How should this rationale be evaluated?

In this paper we address this issue and argue that, while in some instances using VTC scaling might be justified by high costs of the VTV approach, in the majority it is not. VTV (or STS) scaling should predominate in NRDAs. The work reported here builds on the analysis in Byrd and Dunford (2017). The methods we propose can also be applied to the issue of using STS scaling versus VTC or VTV scaling, but our focus here is on VTC versus VTV methods.

This section presents a RMM model of the VTC vs. VTV decision, first under certainty and then using an approach that incorporates uncertainty.

The following elements of the restoration scaling models can (in principle) be estimated given appropriate data:

• The discounted present value of the economic value of lost trips over time due to the spill. We denote this by L. The cost of studies to estimate L is A^L.

• The discounted present value of the economic value of the trips gained from restoration, per unit of restoration scale. We denote this by G. If restoration of scale S is implemented, the value of restored trips is S × G. The assessment cost for estimating G is A^G.

• The discounted present value of the cost of restoration per unit of scale, which we denoted by C. The total spend on restoration of scale S equals C × S.

Henceforth, we denote the ratio of the recreation value gained from a restoration project to the cost of the project (the benefit-cost ratio) by BCR = G/C. As with (almost) every parameter in a NRDA, estimation involves choosing an estimation approach, sampling, statistical analysis, and other procedures that render the estimates uncertain. In this section we assume that point estimates of the above elements can be obtained, which will be random variables, but these are treated as known constants in scaling.

VTV scaling: Chose restoration scale to minimize the cost of restoration while simultaneously ensuring that the value of trips gained by restoration is at least as great as the value of the lost trips. The optimal scale is S^VTV = L/G. Damages are D^VTV = CS^VTV + A^L + A^G. We simplify this to D^VTV = (L/BCR) + A^VTV.

VTC scaling: Chose restoration scale with a cost equal to the value of lost trips. The computed scale is S^STS = L/C. Damages are D^VTC = L + A^G.

The degree of bias associated with VTC scaling hinges on the BCR. If restoration provides recreation opportunities worth more than their costs (BCR > 1), VTC scaling overestimates restoration spend. If restoration fails a benefit-cost test (BCR < 1), VTC scaling under-estimates restoration spend. VTC scaling is unbiased only if the BCR = 1; in this scenario it is preferred to VTV scaling since it is less costly (A^VTV − A^L = A^G > 0).

In most cases, there is a tradeoff between bias and cost. This can be evaluated via the loss to the public that may arise from using VTC scaling. The basic decision theory is shown in Figure 1.

The amount of restoration spend using VTV scaling is on the horizontal axis. On the vertical axis is the dollar loss ℒ from either under or over spending on restoration using VTC scaling. The loss is ℒ = C[S^VTV − S^VTC] if BCR ≥ 1 and ℒ = −C[S^VTV − S^VTC] if BCR ≤ 1.²VTC scaling involves no loss when the BCR = 1, but as the divergence between the BCR and 1 increases (in either direction), loss grows. At some point, the loss reaches the incremental cost of the information needed for VTV scaling, A^G. Beyond this degree of divergence of fact from assumption, it is better to invest in the additional information to estimate the BCR because the value of doing so (the loss avoided) is worth its cost. This analysis provides the “close enough” range for the BCR such that VTC, while biased, is preferred because of its lower costs.

The two critical unknown scaling parameters are the value of lost trips L and the BCR (henceforth B). To simplify the discussion, we assume that L (required for either scaling approach) is stipulated to by the parties and only the BCR is unknown. Initial beliefs about the BCR are given by a probability distribution over the possible values of the BCR. Decisions are based on minimizing the expected (in a statistical sense) loss from incorrect decisions. Let B^* be the true but unknown BCR. If this were known, restoration spend should beset to the unknown value S^* = L/B^*. If instead some other level of restoration spend S^d is implemented, the loss is ℒ(S^d|S^*). Initial (or prior) beliefs about the likely values of the BCR are represented by a probability distribution f⁰(S^*). This could come from judgement informed by existing studies or be uniform over some possible range of values. The best scaling decision minimizes the expected loss, E_f⁰ {ℒ(S^d|S^*)}, calculated using the prior beliefs. Let S⁰ solve this minimization problem, in which case E_f⁰ {ℒ(S⁰|S^*)} is the minimized expected loss.³ This also called the “risk” of the decision S⁰, which we will denote ℛ⁰.

If the Trustees have any defensible information about the BCR, then VTV scaling is preferred to VTC. This follows since, by definition, E_f⁰ {ℒ(S⁰|S^*)} ≤ E_f⁰ {ℒ(S^d|S^*)} for any feasible decision S^d, which includes the VTC decision S^d = L.

Therefore, VTC can only be rationalized only under two conditions: (i) the initial information does not provide a defensible basis for scaling and (ii) there is no study that could be done to obtain defensible information on the BCR where the benefits the information the study will provide exceed the cost of doing the study.

The benefit of obtaining information is called the expected value of information (EVOI). A study Y is a collection of possible results y that, before the study is done, will be obtained with probability h(y). Having the results of a study in hand leads to updated, more informed beliefs f^l(S^*|y) about the true value of the BCR. These are called posterior beliefs. Before the study is done, the expected posterior beliefs are f^l(S^*) = E_h{f^l(S^*|y)}. A central result of decision theory is that expected posterior beliefs have the same mean as the prior beliefs, but a smaller variance (Raiffa and Schlaifer 1961). This increased precision is valuable in decision-making about restoration scale. With the study result y in hand, the risk associated with the scaling decisions is E_h; this is the minimized expected loss calculated with the posterior beliefs. Before the study is done, the expected risk is E_h{ℛ^l(y)}. The benefit of doing a study is EVOI(Y) = ℛ⁰ − E_h{ℛ^l(y)}. Because the study reduces variance, it can be shown that the EVOI is positive.

If C(Y) is the cost of the study, condition (ii) for when VTC can be rationalized is that there is no study that could be done with EVOI(Y) − C(Y) > 0.

Use of VTC scaling indicates a belief that the true BCR is probably close enough to 1 that VTC scaling is preferred to VTV scaling because (i) there is no defensible basis for initial beliefs about the BCR and (ii) the time and money cost of getting any possible additional information is greater than that information's benefit. In our experience, the most common presumptive considerations in support of VTC scaling include:

• Estimation of restoration values needs to be conducted for actual human use projects (rather than generic projects) and so must follow detailed restoration planning with attendant delays in completing the NRDA;

• Estimation of restoration project gains should exhibit similar or less certainty than the estimated losses from injury, which will be costly to achieve;

• Satisfactory estimation of project benefits typically will require an expensive, high-quality primary study; and

• Monitoring and adaptive management needs to be performed after project implementation to assure that the estimated level of benefits are being realized.

We summarize our thoughts on these rationales in favor of VTC scaling.

• (a) Need for VTV Scaling to Use Actual Projects

  As with ecological services, restoration scaling for human use services can be performed for actual or representative potential restoration projects. While a degree of restoration planning is required, this is not disproportionate to what is undertaken for ecological projects, either in timing or cost. The Trustees should begin human use restoration planning early in the NRDA process in parallel with ecological project planning, perhaps inviting RPs to provide resources for initial identification and screening of possible projects. These can inform whether VTV or STS scaling may be appropriate and cost-effective.

• (b) Need for Certainty of Restoration Benefits

  It has been suggested that estimates of the restoration project should have a similar (or lower level) of certainty as estimates of injury. This ignores that there are many sources of uncertainty in restoration scaling that all have equal footing in the ultimate goal of restoration scaling. As is common for ecological restoration projects, NRD practitioners employ various (mostly ad hoc but nonetheless useful) methods for addressing this uncertainty.

  Furthermore, and perhaps more importantly, uncertainty associated with estimation of L in VTC scaling translate directly into damage estimates, while the same sources of uncertainty may appear in G in VTV scaling and hence “cancel out” in damage estimation. This may allow a more streamlined and efficient injury analysis. Many factors in estimating trip values do not have a consensus treatment in the economics literature and uncertainty bands for these can be quite high, leading to highly uncertain damage estimates using VTC scaling, even if there is low uncertainty in the quantity of lost trips. Examples include modeling approaches to properly account for spatial and temporal substitution,⁴ the value of leisure time,⁵ the appropriate monetary expenditures associated with travel distance,⁶ issues related to multi-day, multi-purpose, or multi-site trips,⁷ and potential changes in the value per trip as the composition of a group changes,⁸ among others. Using benefit transfer rather than a primary study introduces even more uncertainty. Combining the uncertainty in all these factors, the uncertainty in damages associated with VTC scaling for many spills may exceed an order of magnitude.

  VTV scaling can ameliorate some of this uncertainty. Many of the uncertain factors listed above would affect injury and restoration gains similarly and therefore “cancel out” at least to some degree. ⁹ For example, VTV would be expected to yield approximately similar damages regardless of the value of leisure time¹⁰, while damages in VTC scaling would be meaningfully affected by this modeling choice nearly as a proportional change in damages.

  The fact that many of these uncertain factors would have little influence on VTV damages means that the cost of assessing these factors can decrease in VTV scaling compared to VTC. Rather than negotiate the best value out of a range from the literature, practitioners can simply assume a middle-ground value and move on.

• (c) Need for a Primary Study of Restoration Benefits

  It may be thought that estimation of restoration benefits typically requires an expensive high-quality primary study in order to achieve a reasonable level of certainty. As Byrd and Dunford (2017) point out, such studies would often be warranted given the magnitude of damages at stake in most spills. In realistic examples we have constructed, the value of information is a significant fraction (e.g. 10–20%) of estimates of L. Thus, even at the lower fraction, an $8 million TVC result implies the EVOI is $800,000; a high-quality benefits study would cost less than half of this.

  However, we note that a site-specific restoration benefits study is not the only way to estimate restoration benefits.

  • (i) Estimation of the number of trips may be straightforward when restoration relaxes a capacity-constraint (e.g. more campsites at a campground, more boat launch lanes, or more parking spaces at an access point). If restored camping is similar to injured camping then values are the same and STS scaling may sued; if not, then benefits transfer (BT) might value the trips gained.

  • (ii) Counts at locations similar to restoration sites are often available, e.g. car counts at state parks, or parking or entrance fees. Relating use to publically available site descriptors (e.g. trail miles or other facilities) can then be statistically related to use. Again, BT could then value those trips.

  • (iii) BT studies may provide benefits for site attributes. For example, the Delaware beach study by Parsons et al. was used for the Buzzards Bay oil spill injury assessment; the value of beach width from the transfer study could have informed the benefits of such a restoration project for that spill.

  • (iv) If injury is assessed with a site-specific study, it may be straightforward to include site attributes that could be improved with a restoration action. One example is the LaVaca Bay assessment; another is the Deepwater Horizon spill, where site attributes such as beach width, existence of fishing piers, or others would naturally be included among the independent variables in the injury assessment equation.

  • (v) A restoration project may reduce injuries valued in a spill. For example, in an area with abandoned nearshore wells (like the Santa Barbara Channel), proper closure of an abandoned well could prevent oiling of beaches, the recreational value of which may have been directly estimated in an injury analysis.

• (d) Need for Post-Project Monitoring and Adaptive Management

  VTV scaling may include monitoring to ensure that the expected amount of recreational visitation is occurring. Actions to managing the risk that projects may not perform as planned is often embodied NRDA settlements including monitoring and adaptive management requirements. There is no reason that this framework cannot be applied to projects focusing on human use services. Using VTC scaling in order to accommodate such contingencies seems a poor and round-about substitute for addressing such matters head-on.

In summary, we believe that, particularly in light of the regulatory preference for VTV over VTC, Trustees should re-consider these rationales. The following section provides preliminary information to aid this re-consideration. The following Section summarizes preliminary results from a pilot study to estimate BCRs for human use projects that have been conducted for NRDAs.

There is surprisingly little information to inform a distribution of BCRs from the outdoor recreation or planning communities. Outdoor recreation professionals tend to focus on managing visitor experiences and maintaining appropriate natural and built aspects of outdoor recreation settings. Planners tend to emphasize maintaining appropriate recreational opportunities per capita or unit area. Economic analyses performed by these groups tend to be economic impact studies rather than economic welfare analyses, which is needed for BCR under NRDA.

As VTC scaling has been the dominant scaling method in NRDA, there is also little information from within NRDA itself. An exception is VTV scaling of early human use projects for the Deepwater Horizon spill (Deepwater Horizon Natural Resource Damage Assessment Trustees 2016). As part of negotiated settlement, the Trustees concluded that typical human use projects considered in early restoration that create or improve human use access and infrastructure have a BCR of 2.¹¹

We have developed preliminary BCR estimates for two of the Deepwater Horizon human use restoration projects that have been completed. We are monitoring several other projects scheduled to be completed in 2020 or 2021, including some significantly larger, higher cost projects. The BCR estimates for additional sites will help practitioners better understand the potential distribution of BCRs and their uncertainties for different types of projects.

The first project we studied is the Apalachicola River Wildlife and Environmental Area Fishing and Wildlife Viewing Access Improvements, located near Apalachicola, Florida.¹² There are two main components to the project: 1) construction of a fishing and wildlife observation platform and 5-space parking lot at Cash Bayou; and 2) construction of an elevated boardwalk on an existing, periodically wet interpretive trail. Monitoring is being conducted using vehicle counters. Based on the data collected to date, we estimate that approximately 150,000 visits would be made to the improved facilities over 30 years.¹³ Applying a value per trip of $20.74 and a three percent annual discount rate, the present-valued consumer surplus is about $2 million over 30 years.¹⁴ The construction cost was approximately $260,000 according to NOAA (2019). Including 29 years of annual O&M, estimated as five percent of the construction costs per year, the total present-valued project cost is about $500,000. Therefore, the BCR is approximately $2 million ÷ $0.5 million = 4. Once more data are available for the site, we will estimate a confidence interval for this estimate.

A second project we studied is the Island View Park project, one of four public shoreline enhancement projects that are bundled under the Florida Coastal Access Project.¹⁵ The Island View Park project is located along the “Forgotten Coast”, east of Eastpoint, Florida. The project developed a seven-acre waterfront park owned by Franklin County with picnic tables, interpretative signs, improved fishing piers, and a paved parking lot. The site formerly had informal access and dirt/sand parking. Based on the data collected to date, we estimate that the BCR could range from 0.5 to 3.0, with a best estimate of about 1.5. As with the previous project, this uses a conservative lower-bound of the value per trip.

These two relatively small projects are likely similar to the types of projects that might be implemented for smaller spills or as part of a package for moderate and large spills. The results above provide support that the BCR for typical NRDA projects is likely to be above 2, and that estimation of benefits can be conducted in a cost-effectively. At the least, these ranges can inform initial beliefs about the BCR.

This paper discussed why the frequently-employed VTC approach to estimating human use damages is generally biased. Preliminary results for two human use projects implemented for the Deepwater Horizon spill suggest that the BCRs for these projects are greater than one. Based on current information, one project has a BCR of about 4 and another project has a BCR of about 1.5.

The cost of obtaining information to estimate these BCRs was minimal and the findings are in line with visitation data from other coastal parks. These findings suggest that BCRs to support human use restoration scaling can be estimated using methods similar to those using on the injury side of the assessment, achieving reasonable levels of certainty and cost. These findings support our argument and those of Byrd and Dunford (2017) that VTV or STS scaling should be predominate scaling option, rather than VTC.