Comparing Methods of Estimating Fecal-Pellet-Group Density in Woodlots of the Midwestern United States

White-tailed deer Odocoileus virginianus (hereafter, deer), are abundant throughout much of the United States, and can influence socioeconomic resources. Deer serve as a positive socioeconomic resource through hunting and recreational tourism. Economists estimate that hunting alone supports over 500,000 jobs, produces $16.9 billion annually from travel expenses and equipment, provides a major source of revenue for state wildlife management agencies via license sales, and results in millions of deer harvests each year (Hewitt 2015). Conversely, deer can also be economically damaging through deer–vehicle collisions, and by degrading forest ecosystems, agricultural crops, nurseries, and gardens (Knoche and Lupi 2011). Given that deer have a large economic impact, biologists, managers, and landowners often desire effective ways to monitor populations to inform management decisions. Thus, quick, easy to use, and cost-effective methods for estimating or indexing deer densities would improve the management of a highly sought-after game species.

Wildlife managers can estimate (Amos et al. 2014; Beaver et al. 2014; Belant and Seamans 2020; Macaulay et al. 2020) or index (Forsyth et al. 2007; Garel et al. 2010; Sollmann et al. 2013; Rouyer et al. 2018) deer population densities in multiple ways, including aerial and spotlight surveys, camera traps, and pellet-group counts. Although these methods vary in how they estimate or index density, studies suggest that they produce similar results (Anderson et al. 2012; Urbanek et al. 2012; Pfeffer et al. 2018). Previous studies have also found that these methods differ in terms of specialized equipment, ease of application, and total cost, with pellet-based surveys generally being the most cost effective and easiest to apply (Anderson et al. 2012; Urbanek et al. 2012; DeCalesta 2013; Pfeffer et al. 2018). Aerial surveys may take less time to collect data than pellet surveys, but the need for an aerial vehicle and other equipment increases the cost and required expertise (Urbanek et al. 2012). Similarly, camera traps require purchasing of cameras which increases cost, and the time to process a large number of images may result in pellet surveys being more efficient (Pfeffer et al. 2018). Spotlight surveys require similar equipment expenditures as pellet-group counts, although the cost required for the former is often higher (Anderson et al. 2012). A potential downside of pellet-based surveys is that they only directly provide an index of deer density, such as the density or the encounter rate of pellet groups. Conversely, other methods can produce information on abundances. Even so, researchers or managers can estimate deer density from pellet groups using defecation and decay rates (Marques et al. 2001).

Fecal-pellet surveys provide a quick, relatively inexpensive index of deer density and wildlife managers often use them to monitor populations. Therefore, several field methods exist for estimating the density of pellet groups (Camargo-Sanabria and Mandujano 2011; Alves et al. 2013), and different methodologies may produce varying results. Generally, three methods are used to estimate standing-crop densities of pellet groups: line-transect, strip-transect, and quadrat sampling (Figure 1). Field personnel often conduct line-transect sampling within a conventional distance sampling framework by sampling along transects of predetermined length and measuring the perpendicular distance to all pellet groups observed from the line (Marques et al. 2001). Strip-transect and quadrat sampling differ in the size and number of sampling units along each transect. Quadrat sampling involves surveying multiple small plots along each transect. Conversely, strip-transect sampling involves surveying one larger plot along the length of each transect. Smith (1968) concluded that the size and shape of the sampling area may affect results. For example, sampling in smaller plots resulted in fewer missed pellet groups than larger ones, leading to increased precision of density estimates. Although methods differ in the way they estimate pellet-group density, previous studies have found that the precision varies, but the density estimates do not (Camargo-Sanabria and Mandujano 2011; Alves et al. 2013). However, researchers conducted these studies in Mexico and Portugal where densities ranged between 4 and 6 deer/km², which is approximately half that of the midwestern United States (Rooney et al. 2002; Delisle et al. 2022) and the greater Central Hardwood Forest Region (Langdon 2001). Based upon a review of the available literature, we found no studies comparing these methods of estimating pellet-group densities in areas within the temperate forests of North America. Management decisions within regions with high deer populations, such as the midwestern United States, may require more precise density estimates. Thus, studies evaluating different methods of estimating densities and precisions are likely to advance deer management.

In this study, we compared three methods to estimate the pellet-group densities in Indiana: line transects, quadrat sampling, and strip transects. We evaluated the differences between the density estimates and corresponding measures of precision from each method in nine survey stands across three regions. Based on findings from similar studies (Camargo-Sanabria and Mandujano 2011; Alves et al. 2013), we hypothesized that methods would produce similar density estimates, but would have dissimilar precisions. Furthermore, we hypothesized that performance of each method would vary across regions due to differences in deer density and landscape composition. Given that we do not know the true density of pellet groups, we did not attempt to hypothesize which of these methods produced the most accurate estimates. Instead, we only evaluated the estimates and precision provided by each method.

We surveyed woodlots in the southern, central, and northern regions of Indiana (Figure 2) on sites owned and managed by Purdue University as part of concurrent research evaluating deer impacts on hardwood stump sprouts. Therefore, our study may have been limited in scope, given that we only sampled a small number of woodlots where deer and forest management were similar. Nonetheless, our study is the first that we are aware of to evaluate each of these fecal-pellet survey techniques within this region of North America. Woodlots within our sample sites were closed-canopy mixed hardwood forests, which are typical of these regions of Indiana, as well as the greater Midwest. Hunters were active in all sites each year.

In the southern region, we surveyed three sites (hereafter survey stands) within the 983-ha Southeastern Purdue Agriculture Center (SEPAC) in Jennings County (Figure 2). The property predominantly consists of actively managed forest (648 ha), with agricultural land as secondary component. The surrounding landscape is mainly a mixture of agriculture and forest, but high building and road densities exist to the north.

In the central region, two survey stands were within Meigs Woods, and one was at Cunningham Woods, both of which are in Tippecanoe County (Figure 2). Meigs Woods is located within the 336-ha Throckmorton Purdue Agriculture Center, where land use is mostly agricultural, but also contains grasslands, wetlands, and forests. Total forest area at Throckmorton Purdue Agriculture Center is ∼74 ha (including Meigs Woods), which is used for recreation and research, and the surrounding landscape is dominated by agriculture. Cunningham Woods is a 302-ha property, with 32 ha of forest, which is surrounded by agriculture. Property use at Cunningham Woods is primarily teaching and research.

In the northern region, we had two survey stands at Harrold Woodland and one at the Pinney Purdue Agriculture Center (Figure 2). Harrold Woodland is a 58-ha forested property in Whitley County. This site's primary use is for research and recreation and agriculture surrounds it. Lastly, Pinney Purdue Agriculture Center is 269 ha of primarily agricultural land located in Porter County; its forested portion (18 ha) is actively managed and the immediately surrounding area is primarily agricultural, but the greater landscape contains greater amounts of urbanization (i.e., roads, houses, and other buildings) than that of our other sites.

We used ArcGIS Pro (v 2.5.0, Esri, Redlands, CA) to delineate our survey stands. When there were multiple survey stands within one property, we separated them by at least 500 m, which is greater than the radius of a circle that is equivalent to the average size of a doe home range in the Midwest (Storm et al. 2007). We delineated each survey stand by creating a 5.6-ha minimum convex polygon. We used 5.6 ha because this was the smallest size that could contain 20 systematically placed 50-m transect lines. Transects were spaced 60 m east to west, and 15 m north to south, which was the maximum spacing that would fit within our survey stands. We placed all transects parallel to the southern edge of the woodlot running east to west.

We collected data at each survey stand from January to March 2021, using each of the three methods (line-transect, quadrat, and strip-transect sampling). We defined a pellet group as having at least six individual pellets with the same color and shine, and located in the same immediate vicinity as each other (Delisle et al. 2022). Because we were only interested in estimating the density of pellet groups, and not the defecation or accumulation rates, we did not clear the area of all pellet groups prior to sampling. A crew of three people performed all sampling, with one person collecting data for a single method in each survey stand. Observers simultaneously collected data for their method, and we avoided biasing the results by starting on different transects and staggering observers such that two people were never within eyesight. We used the same observers for the entire study and rotated methods after each survey stand. Therefore, one person conducted each method once within each region and three times overall. We also recorded how long it took to complete each method within a survey stand to evaluate temporal costs.

We collected line-transect data under a distance sampling framework, which accounts for decreased detection probability with increasing distance from the transect line (Buckland et al. 2001). We walked each line transect twice. First, we walked the 50-m line and only looked for pellet groups directly on the line. Then we walked the line a second time, looking for pellet groups on both sides as far as we could see. We recorded all pellet groups found, and recorded the perpendicular distance between the transect line and the centroid of the group.

For strip transects, we tallied all pellet groups found within 1 m on the right side of the 50-m transect line. We sampled 50 m² along each of our transects, for a total area of 1,000 m² per survey stand. These transect lines were the same ones surveyed during line transects. For quadrat sampling, we tallied all pellet groups found within ten 5-m² (2.25 × 2.25 m) quadrats that were placed every 5 m along the same transects that we sampled for our strip and line transects. Like strip transects, we sampled a total of 50 m² along each transect, for a total of 1,000 m² per survey stand.

We estimated the pellet-group density for line transects using conventional distance sampling (Buckland et al. 2001). We began by truncating our data to remove extremes in the distribution associated with low probabilities of detection, a common practice in distance sampling analysis to reduce the amount of variation and uncertainty (Buckland et al. 2001). To model the detection process, we fit the half-normal key function with two Hermite polynomial, two simple polynomial, two cosine, and no adjustment terms, and the hazard-rate key function with two cosine and no adjustment terms (Buckland et al. 2001). We did not detect enough pellet groups in each region to consider region-specific detection functions (roughly 60 pellet groups required to estimate a detection function; Buckland et al. 2001), so we additionally fit half-normal and hazard-rate key functions with the following combinations of factor covariates: 1) observer, 2) region (central, north, south), and 3) observer and region (Marques et al. 2007). We selected the most parsimonious model using Akaike Information Criterion (AIC) and checked the goodness of fit visually with Q–Q plots and a Cramer–von Mises test (Buckland et al. 2001). We used the Distance package (Miller et al. 2019) in the R programming language (version 4.1.1.; R Core Team 2021) to fit all detection functions.

Methods for estimating density using strip-transect and quadrat sampling are identical, as both assume perfect detection. We estimated density by the following formula: = n/a , where n is the number of detections across all plots and a is the total area of all the plots (Buckland et al. 2015). We computed the variance around with the R² method (Fewster et al. 2009) using the mrds package in the R programming language (Laake et al. 2021; R Core Team 2021).

We compared the distance-sampling estimates across regions using the two indices described in Delisle et al. (2022): consistency and magnitude of differences between bootstrapped distributions of density. We opted to use this method in lieu of a t-test because the estimates from each region are not independent due to sharing a detection function and a covariate in the detection function. Specifically, we bootstrapped (resampling transects with replacement) our data 999 times. From these bootstraps, we estimated 999 region-specific densities by fitting all the aforementioned detection functions and using an automated model selection process with AIC. As a measure of consistency, we used the fraction of instances across the 999 bootstraps in which the paired-bootstrapped difference in density was the same sign as the observed difference between the design-based estimates. As an index of the magnitude of the difference between the sampling distributions for densities from each region, we used , which is the area of overlap of the two distributions of bootstrapped densities (Pastore and Calcagnì 2019).

Secondly, we used a two-tailed t-test to compare the density estimates from each method within the same region. Lastly, we compared the precision of density estimates using the R package cvequality (Marwick and Krishnamoorthy 2019) to test for significant differences between the coefficient of variation (CV) of estimates both from different methods within a region and from the same method across regions using the modified signed-likelihood ratio test.

We evaluated differences in temporal costs using a linear model in program R (R Core Team 2021) with work time (minutes) as our dependent variable, and method, region, and observer as independent variables. Given the speculation that quadrats produce less precise estimates of pellet-group density compared to line transects due to a greater number of sampling units that capture zero pellet groups (Camargo-Sanabria and Mandujano 2011; Alves et al. 2013), we conducted a logistic regression in R (R Core Team 2021) to determine differences in the probability of observing pellet groups with each method. The presence of an observed pellet group along each transect was our dependent variable, and method, observer, and region were our independent variables. Assumptions of all tests were checked visually, and results were considered significant when P < 0.05.

In total, we surveyed 360 transects (1,800 quadrats) and detected 216 pellet groups across all three methods (Table 1; Data S1, Supplemental Material). After examination of the probability density function of perpendicular distances, we decided to truncate all detections at distances > 160 cm. Following this, we found the half-normal key function with observer as a covariate to be the best detection function (Table 2). Overall (all regions combined) and within regions, density estimates did not differ significantly between methods, although we did find a marginal (P = 0.08) difference between estimates from line and strip transects within the central region. Similarly, we did not detect a significant difference between densities from the southern and northern regions within a method (Table 3). However, we found density estimates from the central region to be higher than those from the northern and southern regions for all three methods (Figure 3; Table 3).

When comparing the CV associated with each method, we found significant differences overall, and within the southern and central regions (Table 4; Data S1, Supplemental Material). Examination showed that the CV of the quadrat methods was noticeably lower (≥ 5%) than the other methods in these regions (Table 5), indicating more precise estimates. However, the CV between line and strip transects were similar. When evaluating precision within individual methods, we again found significant differences in the CV across regions (Table 5). For all methods evaluated, we found a lower CV in the central region (Table 5), indicating more precise estimates.

We found no significant differences (F = 1.66, P = 0.22) in the time it took to complete a survey at each site across all methods, regions, and observers (Data S2, Supplemental Material). On average, only about ∼17 min separated the temporal cost for each method, suggesting that field personnel can complete each method in a similar amount of time. Although not significant, the quadrat method took the longest at each survey stand at an average (± SE) of 138 ± 8 min, followed by the strip- and line-transect methods, which took 121 ± 6 and 122 ± 9 min, respectively. Lastly, we found that the probability of observing a pellet group did not significantly differ across the different methods (χ² = 1.69, P = 0.43; Data S3, Supplemental Material).

Our results revealed that quadrat sampling was the most precise of the three methods we evaluated. Line transects have two sources of variation: the detection function and encounter rate, while only the encounter rate influences strip transects and quadrat sampling. Given that line transects have two sources of variation, it makes sense that estimates from this method would generally be less precise. Smith (1968) compared fixed-area sampling methods (i.e., strip transects and quadrat sampling) and found that methods with smaller plots generally produce more precise estimates because pellet groups are less clustered, which decreases variation from the encounter rate. Additionally, we had a greater number of sampling units per region in our quadrat sampling when compared to strip and line-transects (600 quadrats vs. 60 strip and line transects). Theory suggests that increasing the number of sampling units will increase the precision of the encounter rate (Buckland et al. 2001). Thus, this was possibly the cause of the increased precision we observed from quadrat sampling. Although we observed increased precision in our density estimates of pellet groups from the quadrat method, we do not infer any conclusions regarding accuracy, as true density of pellet groups was unknown. Thus, we only conclude that, for our study, the quadrat method resulted in more precise estimates than the strip- and line-transect methods.

Fixed-area sampling methods, such as strip-transect and quadrat sampling, assume a perfect detection within sampled plots (i.e., observers detect all pellet groups with certainty). However, this assumption may not always be true. Smith (1968) found that the estimated density of pellet groups declined with increasing plot size, partially due to observers failing to meet the assumption of perfect detection. Furthermore, the detection of pellet groups can potentially vary between observers (Neff 1968; Härkönen and Heikkilä 1999; Jenkins and Manly 2008). If field personnel do not meet the perfect detection function when sampling, then the precision of the density estimates from strip-transect, and quadrat sampling may be overestimated, as variation from imperfect detections are not accounted for. Although we did not employ double-observer methods, these methods can reveal if the perfect detection assumption was met (Jenkins and Manly 2008). Nonetheless, our pellet-group density estimates from strip-transect and quadrat sampling were not statistically different from line-transect estimates. Therefore, because line transects (i.e., distance sampling) account for imperfect detection, we believe that assuming perfect detection for strip-transect and quadrat sampling did not influence our results. In addition, our quadrat size of 5 m² was roughly equal to the smallest plot size (50 ft² [4.6 m²]) used by Smith (1968) to maximize pellet-group density and precision compared to larger plot sizes, hence we believe our quadrat size was likely small enough to ensure perfect detection.

Dissimilar to our results, other studies (Camargo-Sanabria and Mandujano 2011; Alves et al. 2013) found greater precision with line-transect sampling compared to quadrat sampling. However, both studies speculated that quadrats may perform better in areas with higher deer densities, as it may result in fewer empty quadrats. Average densities in these studies ranged from 4 to 6 deer/km², while estimates in woodlots in Indiana and the greater midwestern United States may range from 10 to 18 deer/km² (Rooney et al. 2002; Delisle et al. 2022). Logistic regression indicated that the probability of observing a pellet group did not differ across methods, suggesting that densities in our study areas were likely high enough to reduce the likelihood of encountering empty quadrats compared with other studies.

The encounter-rate variance is determined by the consistency with which pellet groups are detected between transect lines (Buckland et al. 2001). For instance, we would expect pellet groups that are distributed homogenously throughout an area (such as a woodlot or survey stand) to yield density estimates with a higher precision. For all methods, we found density estimates from the central region to be the most precise. Most of the variation (87–97%) from our line transects, and all the variation from our strip and quadrat estimates, came from the encounter rate, suggesting more homogenous dispersal of pellet groups within the central region. Pellet-group densities from the central region were also significantly greater than those from the other two regions. Assuming that the defecation and decay rates were similar between each region, the increased density of pellet groups in the central region would suggest a greater deer density. Greater deer densities can result in more intense intraspecific competition, and subsequently greater use of available resources (Kie and Bowyer 1999). During the winter (the season in which we surveyed), deer use woodlots heavily for forage and cover (Brown and Doucet 1991; Pauley et al. 1993). Hence, we hypothesize that the greater deer density in the central region increased intraspecific competition, forcing them to use the woodlots containing our survey stands more homogeneously in this region compared to the others. If deer defecate randomly, then this would result in more homogeneously distributed pellet groups, which would decrease the encounter-rate variance, and increase the precision of the pellet-group density estimates within this region. Researchers have previously suggested this positive relationship between density and precision for deer (Alves et al. 2013) and other mammals (Hodges and Mills 2008).

We reported our data as pellet-group densities instead of deer densities for three reasons. First, researchers and managers estimate deer density from pellet sampling using the density of pellet groups and the decay and defecation rates (Marques et al. 2001). Hence, deer density is a direct extension of pellet-group density, and one could expect that the two are directly correlated. Furthermore, because we did not measure the defecation or decay rates, we would have needed to use averaged values from past studies that may not truly represent our sites. This would add more variation and potential inaccuracies to our estimates. Secondly, our objective was to determine which method performed the best for estimating the density of pellet groups. Therefore, estimating deer density was outside the scope of our study. Lastly, multiple studies have found that pellet-group density may be an acceptable index for ungulate densities (Neff 1968; Rivero et al. 2004; Forsyth et al. 2007; Goda et al. 2008; Acevedo et al. 2010; Iijima et al. 2013). These findings suggest that pellet group surveys provide biologists and managers with a straightforward technique to monitor ungulate densities.

Our study found that each method produced similar density estimates of pellet groups, and indicated significant differences in densities across regions. We also determined that the quadrat method had the highest precision, suggesting that biologists or managers should use this method within similar woodlots of the Midwest and the greater Central Hardwood Forest Region. In addition, our results found that each of the methods required a comparable amount of work time, and had similar probabilities of capturing pellet groups. Quadrat sampling does require the use of a quadrat, typically built from small diameter chlorinated polyvinyl chloride piping, resulting in a slightly higher relative cost than the strip- and line-transect methods. Nonetheless, the quadrat method is still likely much cheaper than aerial and camera surveys. When employing pellet-group density as an index of local deer density, we suggest that biologists or managers follow a sampling scheme like ours by using numerous smaller quadrats instead of larger plots or transects. Pellet-based sampling is easy to conduct under a quadrat framework, and requires very little training, thus biologists, managers, or landowners who wish to gain insight about deer populations on their properties could easily employ it. We sampled a total area of 3 km² within each region, which was enough to produce reliable density estimates with adequate precision. We emphasize the caveat that the utility and precision of a technique likely depends upon the deer density within the region it is applied. Previous studies found that line transects produced more precise estimates in regions where densities ranged from 4 to 6 deer/km² (Camargo-Sanabria and Mandujano 2011; Alves et al. 2013). Such densities are much lower than the than those typically found within the midwestern United States (Rooney et al. 2002; Delisle et al. 2022). We believe that using the quadrat method will not only improve deer management decisions but that our study can act as a guideline to progress the conservation and management of other species as well.

Please note: The Journal of Fish and Wildlife Management is not responsible for the content or functionality of any supplemental material. Queries should be directed to the corresponding author for the article.

Data S1. Microsoft Excel file containing data used to estimate densities (and coefficients of variation) of white-tailed deer Odocoileus virginianus fecal-pellet groups across the northern, central, and southern regions of Indiana using strip transects, line transects, and quadrats during winter 2021. Data include region, site (Region.Label), observer, method, whether or not a pellet was found along the transect (Pellets Found Y/N), transect number (Sample.Label), the distance along transects where a pellet group was observed (in meters; Distance_along_transect), the quadrat number (Plot_num), the distance between the transect line and observed pellet groups for line-transects (distance).

Available: https://doi.org/10.3996/JFWM-21-098.S1 (215 KB XLSX)

Data S2. Microsoft Excel file containing data used to determine differences in the temporal cost (work time) for strip transects, line transects, and quadrats that we used to estimate densities of white-tailed deer Odocoileus virginianus fecal-pellet groups across the northern, central, and southern regions of Indiana during winter 2021. Data include region, site, method, the work time (in minutes), and observer.

Available: https://doi.org/10.3996/JFWM-21-098.S2 (20 KB XLSX)

Data S3. Microsoft Excel file containing data used to determine differences in the probability of observing a pellet group for strip transects, line transects, and quadrats that we used to estimate densities of white-tailed deer Odocoileus virginianus fecal-pellet groups across the northern, central, and southern regions of Indiana during winter 2021. Data include method, region, observer, site, transect number, and whether or not a pellet group was found along transects (data are a binary 0/1, where 0 represents no pellet group found and 1 represents a pellet group was found).

Available: https://doi.org/10.3996/JFWM-21-098.S3 (20 KB XLSX)

We would like to thank the Purdue University College of Agriculture and the Department of Forestry and Natural Resources for allowing us to survey on their properties. In particular we would like to thank Brian Beheler and Don Carlson for helping us identify suitable survey stands. We would also like to thank Patrick McGovern and Chris Orpurt for help with GIS work. We would also like to thank the Associate Editor and two anonymous reviewers who provided suggestions that improved this manuscript from our originally draft. This paper is a contribution of the Integrated Deer Management Project, a collaborative research effort between Purdue University and the Indiana Department of Natural Resources–Division of Fish and Wildlife. Funding was provided by the Indiana Department of Natural Resources Grant W-48-R-02.

R.D.S. aided in conceptualizing and designing the study, developing sampling protocols, analyzing and interpreting the results, and leading the writing and editing of the manuscript. R.D.T. collected field data and contributed to the writing and editing of the manuscript. Z.J.D. aided in designing the study, developing sampling protocols, estimating and comparing density estimates, interpreting the results, and writing and editing the manuscript. A.R.T. and P.J.H. collected field data and contributed to the writing of the manuscript. J.M.B. aided in conceptualizing and designing the study, and developing sampling protocols. M.A.J. aided in conceptualizing and designing the study, developing sampling protocols, and writing and editing the manuscript.

Any use of trade, product, website, or firm names in this publication is for descriptive purposes only and does not imply endorsement by the U.S. Government.