Accounting for Surveyor Effort in Large-Scale Monitoring Programs

Wildlife surveys performed over long temporal scales and covering large spatial extents have been used to inform conservation and management efforts with increasing frequency over the past two decades (Thomas 1996; Pollock et al. 2002; Cumming et al. 2010; Barlow et al. 2015). Many of these surveys provide estimates of some ‘true' local relative abundance at a given point in time and space (Link and Sauer 1998; Kéry and Schaub 2012). Reconciling estimates with reality is difficult because of inherent observation error and process variation (Kéry and Schaub 2012). Observation error can be decomposed into imperfect detection, including false positives and false negatives, availability bias (i.e., temporary absence from sites of known use during surveys), and counting errors such as double counts and other inconsistencies (Nichols et al. 2009). Identifying and quantifying these sources of bias are important components of any evaluation of long-term, wide-ranging wildlife survey data. We employ two approaches to account for and quantify error associated with survey duration, which has been shown to affect a large-scale monitoring program (Link and Sauer 1999). We use the Integrated Waterbird Management and Monitoring (IWMM) Program survey of North American waterbirds from the Mississippi River watershed to the coast of the Atlantic Ocean for demonstration.

An essential component of the design of large-scale wildlife surveys is ensuring that samples are taken at appropriate times and in appropriate places. For migratory species, such as waterbirds, this is especially important, requiring focus on critical periods and relevant habitat of the migratory cycle. The IWMM is a geographically broad survey effort to monitor North American waterbird habitats and populations during the nonbreeding period of their annual cycle, including migrations. The IWMM is a collaborative, multiagency effort designed to provide wildlife managers with decision support tools for managing wetland birds and their habitats at multiple scales. The primary efforts of IWMM are focused on gathering local-scale information on waterbird use and habitat conditions of wetlands across the United States. Bird counts and habitat data are voluntarily collected by agency staff and volunteers following standardized data collection protocols designed by IWMM scientists (Loges et al. 2014; Aagaard et al. 2015).

The data collection methods used by the IWMM are rigorous and standardized to minimize imperfect detection and counting errors. Nevertheless, errors in such a broad-scale collection of data may remain. In many cases, data collection errors (bias) may not affect management decisions in any meaningful way. A defined framework with clear and unambiguous criteria to determine whether a data set has sufficient bias to warrant adjustments of counts prior to decision-making is, therefore, highly useful. For example, consistent overcounting at a given location may lead to a disproportionate allocation of financial resources to an area with fewer than expected birds, and away from areas where those resources would be of greater utility (McDonald-Madden et al. 2008). Similarly, if a management decision-making framework includes threshold values with respect to wildlife resource levels (i.e., triggers for management action), unbiased estimates will make management more efficient. Further, biased counting of species abundance over space and time can have profound implications on the statistical power needed for adequate monitoring of status (Larsen et al. 2001; Thogmartin et al. 2007). Therefore, it is essential to identify and account for potential sources of bias and apply sufficient adjustments to the modelled effects of the data to reliably estimate relative abundance (Link and Sauer 1998; we note that our use of the term ‘adjustments' throughout refers to modelled effects in the analysis, not to processing of data prior to analysis).

Counting errors and other observation errors are not unique obstacles to quantitative analyses of ecological data (e.g., James et al. 1996), and rigorous, robust techniques for partitioning variance and adjusting wildlife counts to account for various types of errors have been well-vetted in the literature (e.g., Link and Sauer 1998, 1999; Link et al. 2006, 2008). Even in exemplary and well-standardized surveys such as the North American Breeding Bird Survey and the Audubon Christmas Bird Count many common sources of error have been identified to occur at rates substantial enough to warrant adjustments to estimates of relative abundance (Thomas 1996; Link et al. 2008). Although most participants in the IWMM are current or former professional wildlife biologists, errors may still exist and therefore must be evaluated and, where possible, adjusted accordingly (e.g., Johnston et al. 2018). These adjustments to modelled effects will make the data more useful for informing conservation and management actions at multiple spatial and temporal scales across North America—a core objective of the IWMM.

Germane to the issue of IWMM data evaluation and adjustment is the method provided by Link and Sauer (1999) and Link et al. (2006, 2008) accounting for variation in counts attributable to sampling effects, population effects, and random error. This method involves modeling count data via overdispersed Poisson log-linear regression, where each count is assumed to be a Poisson random variable conditional on its mean. Link et al. (2008) fit this model to data from the Breeding Bird Survey and Christmas Bird Count. Spatial-scale effects are included in the modeling framework for the Breeding Bird Survey and Christmas Bird Count (Link et al. 2008), and similar effects seem reasonable for models fit to the IWMM data, given the spatial extent of the survey locations.

We adopt the approach used by Link et al. (2008) to not only evaluate sources of variability and potential error in IWMM Program surveys, but also account for varying effort. This approach contains two key comparisons; the first is to compare observed and expected counts to evaluate the fit of the model to the original survey data, and the second is to compare estimates of relative abundance (the “adjusted counts”) using a simple effort adjustment (‘Butcher–McCulloch adjustment') and a saturating-effects effort adjustment (‘Link–Sauer adjustment'). This evaluation will serve as an important guideline for IWMM data-handling in future analyses, and we see great utility in detailing the process as it applies to the IWMM specifically as a guide for similar evaluations of data collected in other large-scale monitoring efforts.

The IWMM established 165 survey sites (see Tables S1 and S2, Supplemental Material) in U.S. Fish and Wildlife Service (USFWS) Regions 3 (Midwest: Minnesota, Wisconsin, Michigan, Iowa, Illinois, Indiana, Ohio, Missouri), 4 (Southeast: Kentucky, Arkansas, Tennessee, North Carolina, Louisiana, Mississippi, Alabama, Georgia, South Carolina, Florida), and 5 (Northeast: New York, Vermont, New Hampshire, Maine, Massachusetts, Connecticut, Rhode Island, Pennsylvania, New Jersey, West Virginia, Maryland, Delaware, Virginia), each under a distinct management authority (some federal, some state, and some private) and participating in frequent and recurring waterbird management actions (Figure 1). Sites varied in size and contained different numbers of wetlands. In total, these sites comprised 694 wetlands (hereafter, “units”; Tables S2 and S3, Supplemental Material) with a median area of 26 ha and a range of 0.26–5,015.71 ha.

A detailed description of the current data collection methods is available for IWMM in a separate manuscript (Loges et al. 2014). Over 166,000 observations (i.e., individual birds counted) were collected during the pilot phase of the IWMM program between 25 January 2010 and 11 July 2014 (Table S4, Supplemental Material). Land-based bird surveys were conducted weekly (or biweekly, though weekly surveys predominated) during peak migration, and biweekly (or monthly, with biweekly predominating) during subsequent migratory pulses, depending on logistical constraints and resources available at each unit. Primary migration is considered mid-October to late-December during autumn, and early January to mid-April during spring migration, depending on latitude. Bird surveys consisted of an intensive area-search method in which observers circumnavigated the wetland and recorded all birds detected. Observers did not collect data to estimate detection probability directly, but conducted counts from fixed locations on the wetland perimeter that were chosen to maximize visibility. Survey duration was noted, which allows for observer effort to be accounted for in the modeling framework as detailed below.

We restricted focus to the bird survey observations during the IWMM nonbreeding period (9 October one year to 22 April the next) in the pilot years 2010–2014. We evaluated only IWMM focal guilds (waterfowl, shorebirds, wading birds) in USFWS Regions 3 (Upper Midwest), 4 (Southeast), and 5 (Northeast). This resulted in a data set of 131,412 observations. In total, we used data from 21,309 bird surveys.

As an initial test for possible saturating effects of effort in the data, we evaluated the fit of two regression models with survey duration as the response variable regressed on count and other explanatory variables (survey duration as a function of count, area, unit, observer, and day of migration). One model included both linear and quadratic effects of observed counts with the explanatory variables (survey duration as a function of count, count², area, unit, observer, day of migration, and [day of migration]²); the other model included only linear effects of count and the other explanatory variables. We standardized each regressor, X, as X − X̄ / sd(X). This test served to illuminate the degree to which survey duration increases as more individuals are present (with count serving as a proxy for individuals present); or, if survey duration is relatively static, as the number individuals present increases and therefore effort is unrelated to count, obviating an effort-related adjustment.

In the former case, we would expect to see effort increase one of two ways. First, effort might increase linearly with count and thus the Butcher–McCulloch adjustment would be sufficient. The Butcher–McCulloch adjustment is defined as the quotient of the mean of count effort and the effort of each individual count, i, (Butcher and McCulloch 1990). Alternatively, effort might increase nonlinearly (approaching an asymptote) with count and the quadratic effect of count. This nonlinear increase would suggest evidence of diminishing returns in terms of birds counted as survey duration increases and thus the Link–Sauer adjustment would be necessary. The Link–Sauer adjustment includes a continuous covariate related to effort in terms of survey duration (see mathematical definition below).

The waterbird counts (Y_i) are analyzed using a log-linear model with an adjustment for survey duration of the following form:

where β₀ is the average count, β_a is the regression coefficient for area, and β₁ and β₂ are parameters describing seasonal change in expected counts. To describe seasonal changes in counts, we defined “day” relative to the onset of migration in each Region (i.e., “day” = 0 is the date of arrival of initial migrants in each region). We used an offset, ln(area), in the log-linear predictor to account for the variation in size of units. The random effects of year (η), Region (θ), unit (ω), and observer (ν) are normally distributed with mean 0 and constant variance (σ²). Year was defined by “nonbreeding seasons” from autumn to spring.

The function describing the effect of effort on the proportion of birds counted is

where S_i is survey duration resulting in count i and S̄ is average value of survey duration in each unit and in each year. B and p represent region-specific parameters determining the effect of survey duration. If p is negative and B positive, then there is an asymptotic relationship between effort and count (Link and Sauer 1999). Additionally, p <1 (and B positive), indicates a change in concavity over the range of measured effort (Link and Sauer 1999). Importantly, when B = 1, taking the limit as p → 0 of this formulation reduces to the Butcher–McCulloch adjustment (Link and Sauer 1999; Link et al. 2006, 2008); if B ≠ 1 and p ≠ 0, the Butcher–McCulloch adjustment will be inadequate to account for effects of effort. See Table 1 for definitions and consequences of the special cases of p and B outlined above.

Following the approach of Link et al. (2006), we tested the adequacy of the Butcher–McCulloch adjustment against the Link–Sauer adjustment. The Butcher–McCulloch adjustment is appropriate if p = 0 and B = 1 (Link et al. 2006). Link et al. (2006) suggest that a test of the adequacy of the Butcher–McCulloch adjustment could be made using distance measurements in the bivariate space defined by p and B. Specifically, a Bayesian P-value for the adequacy of the Butcher–McCulloch adjustment is calculated from the proportion of the joint posterior distribution of p and B that lies farther from the point (0,1) than from the mean of the distribution (p*, B*). To implement this test, we first calculated the distance between each pair of sampled values for p and B and the posterior mean (p*, B*) using

and then determined the proportion of these distances that were greater than the distance d(0, 1)

as suggested by Link et al. (2006). This is referred to as a Bayesian P-value and indicates the probability that the posterior distribution for the predicted data excludes the reference value (0,1; Gelman et al. 2004).

We fit this model using NUTS (No U-Turn Sampling) as implemented in the program Stan (Carpenter et al. 2016) in Program R (v. 3.3.1, R Core Team 2016) via the package ‘rstan' (Stan Development Team 2016). We used 2 chains, 2,000 iterations, and a burn-in of 1,000. We were able to achieve reliable posterior distribution approximation with relatively few iterations (relative to Markov Chain Monte Carlo sampling, that is) because NUTS is a more efficient sampler than Markov Chain Monte Carlo (Monnahan et al. 2017). Convergence was achieved for all cases (i.e., the potential scale reduction factor R̂ <1.1; Gelman and Rubin 1992; Brooks and Gelman 1997; Gelman and Hill 2007).

Results of two key comparisons indicated a high degree of concordance between the data and posterior predictions, and a clear difference between the Link–Sauer adjustment and the Butcher–McCulloch adjustment. First, observed counts and modeled estimates with the Link–Sauer adjustment demonstrated good agreement: the Link–Sauer adjustment imposed only small modifications to the observations (Figure 2). The average difference between counts and Link–Sauer effort-adjusted fitted-value model estimates was 32 (SD = 908 individuals), indicating (on average) greater counts than effort-adjusted model estimates. The median difference was one individual, indicating a greater occurrence of modeled estimates being near to counts (and, therefore, a few very large observed counts skewing the distribution). The average deviation between observations and Link–Sauer adjusted estimates was 0.13% (SD = 0.07%). Second, the parameter estimates (and 95% credible interval, CI) for p and B were equal to −0.17 (−0.30, −0.05) and 0.20 (0.16, 0.24), respectively; the Bayesian P-value for the adequacy of the Butcher–McCulloch adjustment was <0.001, suggesting that the Link–Sauer adjustment was more appropriate for these data (i.e., less than one tenth of one percent of the posterior for [p, B] was closer to [0,1] than the mean [p*, B*]). This condition of p < 0 and B > 0 further suggests that the Link–Sauer adjustment for IWMM data displays varying concavity over the range of survey durations reported in the data. Additionally, with p < 0, we can conclude that there are diminishing returns in terms of individuals counted as effort increases.

Our initial test for potential saturation provided some modest evidence that there may be asymptotic relationship between survey duration and numbers counted, because the model with quadratic effects fit the data better than the model with only linear effects (ΔAIC_c = 12.7). Our model yielded a posterior mean estimate for the offset for area (β_a) of 0.36 (0.12, 0.55). Counts (per unit effort) did not appreciably differ among Regions, and the posterior estimates for all Regions had 95% CIs that overlapped with 0 (Figure S1, Supplemental Material). Similarly, there was no difference among years (Figure S1, Supplemental Material). The distribution of the posterior means for each of the 224 observer effects ranged from −22.62 to 12.82 individuals, with a median of 0.76; the 95% CI for 21 observer effects did not include 0.76, indicating that these observers performed significantly different from the median observer: 11 negatively so, and 10 positively so. Similarly, 85 out of 695 units showed significant negative effects while 101 out of 695 units showed significant positive effects (Figure S2 and S3, Supplemental Material). The effect of day of migration was significant, with a 95% CI that did not include 0 (−0.5; −0.9996, −0.0004), but we did not find evidence of nonlinear within-season trends in the data (0.5; 0, 0.99).

The results of the hierarchical mixed-effects linear model applied to the IWMM Program's pilot data indicate that, as Link et al. (2008) found for Christmas Bird Count data, the Link–Sauer adjustment was more appropriate than the Butcher–McCulloch adjustment. This places an emphasis on the importance of auxiliary data collected as part of the surveys, such as survey length, to allow for adjustments such as those proposed by (Link and Sauer 1999). This is not to suggest that IWMM data or monitoring protocols are flawed, but rather, as with any survey effort applied across a wide spatial and temporal scale, corrective factors are often needed to standardize the data from the various participating locations and observers. Indeed, the IWMM pilot data showed good concordance with the modeled estimates, indicating that the protocol employed resulted in observed counts that are subject to only modest adjustments for effort.

The posterior distributions of p and B suggest saturating effects of effort on numbers counted, in which case the Butcher–McCulloch adjustment would be inappropriate. This is corroborated by the observation of a model with quadratic effects of effort providing a better fit to the data than one without. The effect of day of migration is potentially a conflation of environmentally imposed variation in species abundance and observation error. A more direct measure of the phenology of migration and environmental conditions, such as first leaf date as an index of spring (Allstadt et al. 2015; Kelly et al. 2016) or a winter severity index for describing the progress of autumn migration (Notaro et al. 2016), could be used to parse the day effect from meaningful environmental variation.

By incorporating observer effort (survey duration) and spatial effects on variation (unit and regional effects), we adjusted the estimates of relative abundance for expected confounding factors. Although we included a proxy for observer-related error (random observer effects), we did not explicitly include effects of imperfect detection and counting errors in the model. We found an asymptote in the relationship between survey effort and the total count from the IWMM area-search methods, yet effort adjustments to estimated relative abundance were small, suggesting that perhaps future research efforts should focus on other components of the observation process (e.g., imperfect detection). We also note that although the adjustments applied to IWMM data were marginal, they were helpful based on the criteria laid out by Link et al. (2006). We see this evaluation as a critical step in the management decision-making process: decision makers have a better understanding of the sources of variability and potential for bias present in IWMM data, and can make informed decisions based on this assessment. Given the modest adjustments to estimates of relative abundance, in some cases a small amount of bias may have negligible consequences for managers. Nevertheless, a reliable assessment framework is essential. In a specific management context, with well-articulated management objectives and information needs (e.g., management triggers and thresholds of relative abundance that would lead to specific management actions), our analytical framework provides a transparent means to assess bias caused by the observation process and environmental variation.

We applied a rigorous and robust hierarchical mixed-effects model accounting for variation in survey effort and random effects related to annual variation, geographic variation, and seasonality. In many similar survey contexts, such variation may not be pronounced enough to affect management decisions, and thus the effort of adjusting counts is unnecessary. However, without information about the magnitude and direction of the bias, decision makers cannot fully understand the benefits of model-based adjustments. We argue that identifying aspects of variation and bias are important to determine whether it is severe enough (and, if so, in what direction) to warrant adjustment or else risk spurious inference for management decision-making. In the context of the IWMM surveys, we found sufficient evidence to warrant adjustment of count data to account for temporal and spatial sources of variation. The adjustments we applied allow for the data to be used to inform relevant management and conservation efforts for waterbirds across their migratory and overwintering habitat and throughout the nonbreeding period of their annual cycle. Applying approaches like those of Link et al. (2006) and herein can inform the utility of model-based adjustments with specific data sets. We recommend evaluation of similar model adjustments for other long-term, broadly distributed survey efforts.

Please note: The Journal of Fish and Wildlife Management is not responsible for the content or functionality of any supplemental material. Queries should be directed to the corresponding author for the article.

Table S1. Spatial data associated with Integrated Waterbird Management and Monitoring (IWMM) sites during the pilot phase (2010–2014) of the program throughout North America. SiteCode is a unique identifier for each IWMM site, in the format [State Abbreviation] − [Site Number]. SiteName is the corresponding local name for the site. Desc provides a brief qualitative description of the site, when available.

Found at DOI: https://doi.org/10.3996/022018-JFWM-012.S1 (17 KB XLSX).

Table S2. Spatial data associated with Integrated Waterbird Management and Monitoring units during the pilot phase (2010–2014) of the program throughout North America. UnitCode is a unique identifier for each unit, in the format [State Abbreviation] − [Unit Number]. UnitName is the corresponding local name for the unit.

Found at DOI: https://doi.org/10.3996/022018-JFWM-012.S2 (34 KB XLSX).

Table S3. Complete table of information relating to Integrated Waterbird Management and Monitoring bird surveys conducted during the pilot phase (2010–2014) of the program throughout North America, including the following:

SiteCode, a unique identifier for each site, in the format of [State Abbreviation] − [Site Number];

SiteName, the corresponding local name for the site;

UnitCode, a unique identifier for each unit, in the format of [State Abbreviation] − [Site Number] − [Unite Number];

UnitName, the corresponding local name for the unit;

UnitLat, the latitude of the unit (centroid);

UnitLong, the longitude of the unit (centroid);

Observer, a unique identifier for the surveyor;

AOU code, the four-letter American Ornithologists Union species code for the observed species;

Common name, the common name of the observed species;

Guild, the guild of the observes species;

Count, the number of individuals of that species observed.

Found at DOI: https://doi.org/10.3996/022018-JFWM-012.S3 (7.27 MB XLSX).

Table S4. Area of each Integrated Waterbird Management and Monitoring unit during the pilot phase (2010–2014) throughout North America, including the following:

UnitCode, a unique identifier for each unit, in the format of [State Abbreviation] − [Site Number] − [Unite Number];

UnitLat, the latitude of the unit (centroid);

UnitLong, the longitude of the unit (centroid);

Hectares, the area (in hectares) of the unit.

Found at DOI: https://doi.org/10.3996/022018-JFWM-012.S4 (53 KB XLSX).

Figure S1. Posterior median values (circles) ± the 95% credible interval (triangles) for the effect of each Region and year on Integrated Waterbird Management and Monitoring counts collected during the pilot phase (2010–2014) of the program throughout North America. Covariates were not standardized and are not cross-comparable (i.e., regional effects cannot be directly compared with annual effects, hence the vertical dashed gray line). We have presented these figures separately in the appendices because of the number of posterior values for units, observers, and day.

Found at DOI: https://doi.org/10.3996/022018-JFWM-012.S5 (163 KB PDF).

Figure S2. Posterior median values (circles) for observer effects on Integrated Waterbird Management and Monitoring counts collected during the pilot phase (2010–2014) of the program throughout North America. Circles are darker for those with 95% credible intervals not overlapping 0.

Found at DOI: https://doi.org/10.3996/022018-JFWM-012.S6 (150 KB PDF).

Figure S3. Posterior median values (circles) for unit effects on Integrated Waterbird Management and Monitoring counts collected during the pilot phase (2010–2014) of the program throughout North America. Circles are darker for those with 95% credible intervals not overlapping 0.

Found at DOI: https://doi.org/10.3996/022018-JFWM-012.S7 (170 KB PDF).

Reference S1. Butcher GS, McCulloch CE. 1990. Influence of observer effort on the number of individual birds recorded on Christmas Bird Counts. Pages 120–129 in Sauer JR, Droege S, editors. Survey designs and statistical methods for the estimation of avian population trends. U.S. Department of the Interior Fish and Wildlife Service. Biological Report 90 (1).

Found at DOI: https://doi.org/10.3996/022018-JFWM-012.S8 (1.24 MB PDF); also found at: http://hdl.handle.net/1813/33091.

Reference S2. Loges BW, Tavernia BG, Wilson AM, Stanton JD, Herner-Thogmartin JH, Casey J, Coluccy JM, Coppen JL, Hanan M, Heglund PJ, Jacobi SK, Jones T, Knutson MG, Koch KE, Lonsdorf EV, Laskowski HP, Lor SK, Lyons JE, Seamans ME, Stanton W, Winn B, Ziemba LC. 2014. National protocol framework for the inventory and monitoring of nonbreeding waterbirds and their habitats, an Integrated Waterbird Management and Monitoring Initiative (IWMM) approach. Fort Collins, Colorado: U.S. Fish and Wildlife Service Natural Resources Program Center.

Found at DOI: https://doi.org/10.3996/022018-JFWM-012.S9 (7.02 MB PDF); also available at ServCat: http://ecos.fws.gov/ServCatFiles/Reference/Holding/40341.

We thank the numerous IWMM participants for volunteering their time to collect the data used in these analyses. We appreciate comments that greatly improved earlier versions of the manuscript from W. Link, C. McGowan, E. Weiser. We are also grateful for the time and attention of the reviewers and Associate Editor of this manuscript.

Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the U.S. Government.