Manual Versus Semiautomated Bioacoustic Analysis Methods of Multiple Vocalizations in Tricolored Blackbird Colonies

Bioacoustic monitoring using autonomous recording units (ARUs) is widely used to collect presence–absence data on various vocal invertebrate and vertebrate taxa (Ospina et al. 2013; Aide et al. 2017; Mifsud and Vella 2019; Abrahams and Geary 2020; Vu and Doherty 2021). Many species vocalize in association with certain behaviors, so bioacoustic monitoring has the potential to remotely reveal patterns of animal behavior such as courtship and reproduction (Teixeira et al. 2019). Studying such animal behaviors through time can provide invaluable information on phenological shifts in behavior and population trends in response to climate change (Oliver et al. 2018). Thus, behavioral bioacoustic monitoring can aid in providing deeper understanding of wildlife populations for management and decision-making (Sordahl 1979; Herzing 1996).

Despite the utility of bioacoustics in wildlife research, bioacoustics remain infrequently used to monitor wildlife behavior because of two fundamental challenges (Teixeira et al. 2019). First, knowledge of vocalizations and how they relate to behavior is lacking for many species, rendering behavioral bioacoustic monitoring infeasible (Penar et al. 2020). Second, when behavioral bioacoustic monitoring is feasible, it often results in large acoustic data sets necessary to capture relevant behaviors (Teixeira et al. 2019; Schackwitz et al. 2020). Efficient analysis techniques of such large acoustic data sets are therefore essential, especially as studies across large geographic regions or using multiple vocalizations further increase the size of data sets. To fully utilize the power of bioacoustics, these challenges need to be overcome.

Vocalizations and their associated behaviors have been well-established for the tricolored blackbird Agelaius tricolor, a synchronous colonial breeding passerine that is nearly endemic to California (Collier 1968; Orians and Christman 1968). Schackwitz et al. (2020) demonstrated the utility of bioacoustics to monitor breeding success and related behaviors in tricolored blackbird colonies; however, the acoustic analysis methodology remains a challenge. Current methods of analysis for tricolored blackbird acoustic data require approximately 20 h/breeding attempt, or pulses, for a single colony and a trained expert to listen to each recording (Schackwitz et al. 2020). Conducting behavioral bioacoustic monitoring of colonies across the range of this species to provide a population-wide assessment of reproductive success is not practical given current analysis methodologies (Schackwitz et al. 2020). Therefore, a method to analyze the acoustic data more quickly without compromising the results and quality of the analysis is needed.

Following the work of Schackwitz et al. (2020), we have expanded behavioral bioacoustic monitoring to additional tricolored blackbird breeding colonies with the goal of developing a methodology to analyze acoustic data more efficiently from each colony. We utilized two approaches to analyze the bioacoustic data, calculated the time required for each method, and evaluated how well each method determined the timing and detection of tricolored blackbird breeding behaviors. Here, we report on the development of a streamlined methodology to analyze tricolored blackbird acoustic data that drastically reduced analysis effort while maintaining and expanding upon phenological information obtained through the original manual analysis technique.

We conducted bioacoustic monitoring at three tricolored blackbird colonies: Rush Ranch, Hay Landfill, and Denverton Creek. The Rush Ranch colony site is on the Rush Ranch Preserve, adjacent to the Suisun Marsh in Solano County, California (38.208, −122.014) at 9.0 m elevation. The colony nested in a 0.9-ha, stream-fed pond containing cattails Typha spp. and bulrushes Schoenoplectus spp. covering approximately 60% of the pond. Grasslands, grazed by cattle, surround the site. The historical size of this colony is estimated to be 500 to 2,000 adults. We placed the ARU approximately 3 m from the edge of the historical colony location. The Hay Landfill colony site is in a mitigation pond adjacent to the Hay Road Landfill in Vacaville, Solano County, California (38.306, −121.825) at 6.5 m elevation. The colony nested in a 7.0-ha, water-managed pond containing Typha spp. covering approximately 25% of the pond. The colony is surrounded by agricultural land and grassland on the eastern, western, and southern borders and a landfill on the northern border. A heavily traveled highway borders the eastern edge. The historical size of this colony is estimated at 5,000–10,000 adults. We placed the ARU within the colony on the southeastern border. The Denverton Creek colony site is in a cattle ranch field in Solano County, California, (38.241, −121.888) at 9.0 m elevation. The colony nested in 0.36 ha of Himalayan blackberry Rubus armeniacus in the cattle field and adjacent to Creed Road. Grasslands, grazed by cattle, surround the site. The historical size of the colony is estimated to be 500–2,000 adults. We placed the ARU approximately 125 m from the edge of the colony.

We deployed AudioMoth (Open Acoustical Devices, University of Southampton and Oxford University, UK; https://www.openacousticdevices.info) ARUs at three tricolored blackbird colonies for the duration of the breeding season, between February and July, from 2019 to 2021 (Schackwitz et al. 2020). We programmed the ARUs to record 1 min of audio every 20 min. In 2019, ARUs recorded between 0000 and 2400 hours and were active 6 February to 6 July at Rush Ranch, 17 March to 7 July at Denverton Creek, and 8 April to 7 July at Hay Landfill. In 2020, as a result of a programming error, ARUs recorded between 0000 and 1140 hours and were active 12 February to 10 July at all colonies. In 2021, ARUs recorded between 0000 and 2400 hours and were active 2 February until running out of battery on 8 May. After retrieving ARUs, we used Automated Remote Biodiversity Monitoring Network (Rainforest Connection, Katy, TX; hereafter referred to as Arbimon) for storage and acoustic analysis.

We analyzed the acoustic data from 2019 using a manual and semiautomated methodology (see “Manual analysis” and “Semiautomated analysis” sections below) and tracked the analysis effort in hours and minutes for each approach. After validating the performance of the semiautomated methodology on manually analyzed data using the 2019 data set, we applied the semiautomated analysis methodology to the full 2020 and 2021 acoustic data sets for each site. We used six focal tricolored blackbird vocalizations (Figure 1) for both analysis methods: male song, male chorus, female song, and begging calls from hatchlings, nestlings, and fledglings (collectively referred to as begging calls). Manual analysis involved listening to a subset of 60-s recordings and documenting the presence of the six focal vocalizations (Schackwitz et al. 2020). To reduce analysis time, we also developed a semiautomated methodology. Here, we used pattern matching algorithms in Arbimon that automatically locate potential vocalizations of interest in combination with an abbreviated manual analysis where automated detection of vocalizations frequently failed (see “Discussion”).

To evaluate the effectiveness of the semiautomated analysis methodology, we considered the results from the manual analysis to be accurate. We conducted pattern matching algorithms on the manually analyzed data to calculate the numbers of false positives (FP; vocalization determined as absent by manual analysis, but detected by semiautomated analysis), false negatives (FN; vocalization determined as present by manual analysis, but not detected by semiautomated analysis), true positives (TP; vocalization determined as present by manual analysis and semiautomated analysis), and true negatives (TN; vocalization determined as absent by manual analysis and not detected by semiautomated analysis). We then used these metrics to calculate recall rates [TP/(TP + FN)], false-positive rates [FP/(FP + TN)], false-negative rates [FN/(FN + TP)], and precision [TP/(TP + FP)]. We also evaluated the relative importance of call type, month, and colony site on the odds that Arbimon's pattern matching algorithm was correct (true positive or true negative). To this end, we fit a logistic regression model with the outcome of Arbimon's pattern matching algorithm as a binomial response variable (1 = correct result, 0 = incorrect result). We used call type (male song, female song, begging calls), month, and the interaction of months and sites as predictor variables. We added the interaction because we observed that the odds of a correct detection from pattern matching algorithms changed across months differently for each site. We used female song as the reference call type, Denverton Creek as the reference colony site, and April as the reference month. To determine which predictor variable had the largest effect, we used an analysis of deviance. To assess the goodness-of-fit of the model, we used a chi-square deviance test of the deviance explained.

Tricolored blackbirds have a rich repertoire of vocalizations specific to various breeding stages. Thus, determining the onset of each breeding stage is possible through documenting the timing of associated vocalizations. The recognized breeding stages of a colony are prospecting, settlement, incubation, brooding, fledging, and dispersal (Beedy et al. 2020). We can determine the timing of each breeding stage by documenting six types of vocalizations: male song, male chorus, female song, hatchling call, nestling call, and fledgling call (Figure 1). During male song, the songs of individual males can be distinguished in spectrograms (Figure 1A). This is in contrast to male chorus where individuals vocalize simultaneously, precluding clear spectrograms of individual male songs (Figure 1B). Full descriptions of each breeding stage and associated vocalizations are displayed in Table 1. Figure 2 visually displays the breeding stages and associated vocalizations of an idealized tricolored blackbird colony with a single, synchronous breeding attempt (hereafter referred to as breeding pulses) and successfully fledged young.

Tricolored blackbird colonies sometimes consist of multiple synchronous breeding pulses within a single season (Collier 1968; Hamilton 1998). These multiple breeding pulses usually indicate that additional groups of adults have joined an established colony and the entire breeding sequence (e.g., settlement, incubation, brooding, etc.) repeats (Collier 1968; Hamilton 1998). When this occurs, multiple breeding stages can occur simultaneously within a colony. For example, the initial arrivals might be in the incubation stage while later arrivals are in the settlement stage. This would be indicated as a complex bioacoustic pattern that must be carefully assessed to tease the timing of the overlapping stages (Figures S1C–S1E, Supplemental Material).

We considered manual analysis the most accurate methodology for determining approximate timing and transitions of breeding stages for colonies. We selected 13 hourly recordings per day from 0700 to 1900 hours for all three colonies. We listened to each 60-s recording and documented whether male song, male chorus, female song, and begging calls (hatchling, nestling, or fledgling) were present. We ended our analysis if there were six consecutive days with no tricolored blackbird vocalizations. For each day we calculated the percentage of manually reviewed recordings that contained tricolored blackbird vocalizations to infer breeding stages of colonies through time.

To reduce analysis effort compared to manual analysis, we developed a semiautomated methodology using a combination of manual analysis and pattern matching algorithms (as described in LeBien et al. 2020) from Arbimon (Figure 3). Using Arbimon's pattern matching utility, we generated templates for male song, male chorus, female song, hatchling call, nestling call, and fledgling call to attempt to locate the same vocalizations across yearly data sets for each colony and validated the template matching results (matches per recording = 1; matches per site = no limit; threshold = 0.2). Using the input template, Arbimon's pattern matching utility detects time-localized signals with a correlation (computed in the time-frequency domain) equal to or greater than the assigned threshold (LeBien et al. 2020). The pattern matching results for two of the vocalizations, male chorus and hatchling call, failed to return any TPs, and semiautomated analysis for these two vocalizations was abandoned. We suspect that detection of male chorus failed because this vocalization generates blurred spectrograms (Figure 1B) so there is no distinct pattern generated for the algorithm to find. We suspect that hatchling call failed because this vocalization is very faint and also lacks a distinctive pattern (Figure 1D). For the vocalizations that did return TPs using the pattern matching utility, we often used multiple templates to run several pattern matching algorithms for the same year and colony. For each vocalization type we selected the pattern matching job that, upon cursory inspection, returned the highest rate of TPs for more exhaustive validation. The rough rate of TPs was assessed by assessing a handful of the returned results and validating whether they were TPs or FPs. Our ARUs collected >5,000 recordings in a breeding season for a single colony, so Arbimon's pattern matching algorithm often returned several thousand putative matches. Thus, we decided to limit our validation time to 1 h or until we validated the presence of 1,000 TPs (LeBien et al. 2020). For nestling and fledgling calls, we divided the 1 h of analysis time or 1,000 validations between the two calls. We then used the pattern matching algorithm data to calculate the relative proportions of recordings containing tricolored blackbird vocalizations each day and to show the vocal activity and inferred breeding stages of colonies through time.

Occasionally, pattern matching algorithms returned few to no TPs for a particular vocalization. Examples of FP putative matches returned from the algorithm include other organisms, wind, or other background noises that precluded the detection of target vocalizations (see “Discussion”). We were uncertain if the lack of TPs was due to faint or distant vocalizations or true absence of the vocalizations in the recordings. To discern between these two interpretations, if there were <100 TPs for any of the vocalizations, we used a mini-manual analysis approach (defined below) to roughly evaluate the presence of vocalizations throughout the breeding season. We chose 100 TPs as a minimum threshold because this is approximately the number of total detections of female song (the most infrequent call) from either manual or semiautomated analysis for small colonies. For mini-manual analysis, we selected two morning recordings and one afternoon recording every third day throughout the breeding season and documented if each of the six vocalizations of interest (male song, male chorus, female song, hatchling, nestling, or fledgling calls) were present. We continued analyzing recordings until there were three consecutive days (seven calendar days) of evaluations with no tricolored blackbird vocalizations. We then used these data to calculate the proportions of recordings containing tricolored blackbird vocalizations, to show vocal activity, and to infer breeding stages of colonies through time.

Once we obtained rough timings of the vocalizations through pattern matching algorithms or mini-manual analysis, we refined the dates of various breeding stages by performing edge analyses for the onset of incubation, hatching, and dispersal. First, we used male chorus to determine the onset of a breeding pulse and incubation. Male chorus commonly exists outside of the onset of a breeding pulse in the mornings and evenings, but only exists midday when the colony is attempting to initiate breeding (Orians 1961b). Therefore, we evaluated recordings between 0900 and 1659 hours. Pattern matching algorithms were not successful at locating the presence of male chorus, so we had to manually locate recordings that contained male chorus. Recordings that contain male chorus can be quickly identified by visually inspecting the spectrograms. To roughly determine the dates containing male chorus we visually scanned a subset of the recordings (1100 to 1159 hours). After identifying consecutive blocks of days with midday male chorus, we evaluated hourly recordings from 0900 to 1659 hours for the presence of male chorus starting 3 d prior to and ending 3 d after the block of days we identified. Males at sites that fail to establish consistent colonies do not produce a consistent male chorus, as confirmed by visually examining recordings during the mini-manual analyses for these sites. We assigned the onset of settling behavior when there were at least three consecutive days with midday male chorus immediately followed by female song. We assigned the onset of incubation when midday male chorus ceased, but female song continued. To determine hatching dates, we started with the earliest date that we detected offspring calls and evaluated two recordings in the morning and one in the afternoon for the presence of hatchling calls. We continued moving back 1 d earlier until we had at least three consecutive days with hatchling calls absent. We then moved forward in time until we had at least three consecutive days with hatchling calls present and evaluated hourly recordings, as done in manual analysis, of these 6 d to approximate the first day of hatchling calls and when hatching calls become robust, which we assigned as the hatching date for a colony (>50% of recordings with hatchling calls present). We then reversed this technique to perform an edge analysis to determine when fledglings dispersed or that nestlings did not survive to an age consistent with fledging.

We analyzed acoustic data from the Denverton Creek, Hay Landfill, and Rush Ranch tricolored blackbird colonies in 2019, 2020, and 2021. In 2019, we applied the manual analysis and semiautomated analysis methodologies to the acoustic data. We then applied automated pattern matching algorithms to the manually analyzed 2019 data to assess the performance of the semiautomated analysis. To ensure that the semiautomated analysis methodology continued to be successful and minimize analysis time, we applied it to the 2020 and 2021 data from the colonies. Here, we report on the time expenditure for each method, detection performance of the semiautomated methodology, and a brief summary of the outcomes for each colony through time.

We found that semiautomated analysis took approximately one third of the time as manual analysis while yielding comparable estimates of timing of breeding stages and the numbers of breeding pulses. Manual analysis of all three sites took 104 h 30 min (Denverton Creek = 30 h 3 min, Hay Landfill = 29 h 40 min, Rush Ranch = 44 h 47 min). Semiautomated analysis of all three sites took 30 h 38 min (Denverton Creek = 12 h 11 min, Hay Landfill = 14 h 10 min, Rush Ranch = 4 h 17 min).

Overall, we determined that using a semiautomated analysis approach yielded accurate and detailed results. We chose the manually evaluated recordings from the 2019 data sets to evaluate the performance of the pattern matching algorithm. Of the 4,391 manually evaluated recordings, a total of 1,335, 271, 228, and 543 recordings contained male song, female song, nestling call, and fledgling call, respectively (Figure 4). Detection performance of pattern matching functions for each vocalization varied considerably between colonies (Table 2) likely because of differences in anthropogenic and natural background noise levels, proximity of the ARU to the colony, and number of vocalizing birds in each colony (see “Study Area”). Across all three sites, recall rates were generally highest for male song (0.79 ± 0.19) and fledgling call (0.77 ± 0.33), lowest for nestling call (0.32 ± 0.20), and varied for female song (range from 0.15 to 0.90). Rates of FPs were high (>0.50) for all vocalizations except for nestling call (0.19 ± 0.09). Similarly, FN rates were generally highest for nestling call (0.68 ± 0.20) followed by female song (0.54 ± 0.39) while FN rates for male song and fledgling call remained low (<0.50). Precision was highest for nestling calls (0.48 ± 0.33) and male song (0.46 ± 0.16) and lowest for female song (0.16 ± 0.03).

We fit the logistic regression model to 22,390 records of results from Arbimon pattern matching algorithms (Table 3). Female song, the reference call type, had the lowest odds of correct detection of all call types with a 38.5% lower chance of a correct detection than an incorrect detection by the pattern matching algorithm (P < 0.0001). Similarly, the odds of an incorrect detection for male song were higher than for a correct detection (P < 0.05). Nestling calls had the highest odds of being correctly detected (P < 0.0001), with 2.75 times the chance of a correct detection than not. Fledgling calls also had high odds of being correctly detected and were 2.26 times more likely to be correctly detected than not (P < 0.0001). The effect of site, after accounting for the interaction of month and site, was statistically significant such that the odds of a correct result were 0.352 times lower at Hay Landfill (P < 0.0001) and 11.7 times higher at Rush Ranch (P < 0.0001) than at Denverton Creek. The effect on correct detections at Rush Ranch was greatest in April, the reference month, (P < 0.0001) and lower in other months, although still statistically significant in May, June, and July (Table 3). At Rush Ranch, the odds of a correct result were highest in April and decreased monotonically from May to July and were significantly lower than the reference month (April; P < 0.0001 in all cases). The odds of a correct result at Denverton Creek were relatively constant across all months but varied between 25 and 35% at Hay Landfill (significantly lower in June and higher in May, with respect to April, P < 0.0001 in both cases). The effect of site is the largest among all variables (Table S1, Supplemental Material), accounting for nearly four times the deviance explained by any other effect.

In 2019, tricolored blackbirds consistently occupied Denverton Creek throughout the breeding season. There was evidence for at least one breeding pulse and one successful round of fledglings (Figures 5A–5D and 6A; Table S2, Supplemental Material). In 2020 and 2021, tricolored blackbirds were sporadically present at this site with no permanent breeding colony and did not attempt breeding (Figures 6D and 6G; Figures S1A and S1B and S2A and S2B, Supplemental Material).

In 2019, tricolored blackbirds occupied Hay Landfill with evidence for two breeding pulses and two successful rounds of fledglings (Figures 5E–5G and 6B; Table S2). The second round of nestlings did not disperse before the ARU stopped operating, so we continuously detected nestlings to an age consistent with high survival rates in fledglings as documented previously (Beedy et al. 2020). In 2020, tricolored blackbirds occupied Hay Landfill with evidence for two main breeding pulses, each being joined by additional birds as evidenced by multiple consecutive rounds of male chorus, short gaps between hatching of nestlings, and continuous female song (Collier 1968; Figure 6E; Figures S1C–S1E). Based on the continuous nestling calls detected from the hatching of the very first nestlings in April until mid-July (Figure 6E; Table S1), we conclude that both pulses and subpulses of nestlings successfully fledged. In 2021, tricolored blackbirds occupied Hay Landfill with evidence of one breeding pulse (Figure 6H; Table S1; Figures S2C–S2E). The ARU ran out of power before nestlings were of fledging age, but we detected nestlings the last day the ARU operated.

In 2019, tricolored blackbirds occupied Rush Ranch (Figures 5H–5K and 6C; Table S2). Males chorused followed by female song, indicating an attempt at breeding. However, females stopped singing and we detected no nestlings, indicating an unsuccessful breeding attempt. In 2020 and 2021, tricolored blackbirds were sporadically present at this site with no permanent breeding colony (Figures 6F and 6I; Figures S1F, S1G, S2F, and S2G).

In this study, we evaluated manual and semiautomated acoustic analysis approaches to reveal aspects of breeding success and activity of tricolored blackbird colonies using multiple vocalizations. We found that using a semiautomated approach to analyze acoustic data to determine the timing of breeding stages and the number of breeding pulses from colonies was more efficient than a manual analysis approach while utilizing comparable amounts of recordings (Figure 4). These findings add to a suite of methodologies proposed to handle acoustic data, especially in cases where phenology and aspects of behavior are main research questions. Furthermore, our proposed framework provides examples and applications of acoustic analysis to colonial animals where overlapping calls are common and known to cause issues in automated detection (Larsen et al. 2021; Linares et al. 2022).

Manual analysis of tricolored blackbird acoustic data remains useful because it is accurate, and recordings are analyzed chronologically throughout the breeding season. The human ear can detect a wider range of faintness and variation in vocalizations that is not always identified by automated detection software (Goyette et al. 2011). Additionally, listening to recordings chronologically throughout the breeding season allows for accurate determination of the onset and cessation of hatching and male chorus without additional effort. This method is also useful to determine the number of breeding pulses, especially when nestlings from different breeding pulses are simultaneously present. However, this method only uses a subset of all collected recordings and is time intensive.

Semiautomated analysis provides a more efficient way to analyze tricolored blackbird acoustic data. However, pattern matching algorithms can fail to detect recordings with tricolored blackbird vocalizations and performed poorly at detecting faint vocalizations and male chorus, resulting in inaccurate timing of incubation and hatching. In these cases, we relied on a mini-manual analysis to minimize analysis time while focusing analysis effort on the timing and determination of breeding behaviors, if any (Figure 3). The use of mini-manual analysis provided similar advantages to those of manual analysis and was less time intensive. However, this methodology used fewer recordings per day, only provided rough timing of breeding stages, and did not provide the same fine-scale frequency information of each vocalization over time as did analysis with pattern matching algorithms. Our goal was to determine the timing of hatching, fledging, and incubation; therefore, the incorporation of edge analyses (Figures 5C, 5F, and 5J) was necessary but time intensive to conduct. Overall, semiautomated analysis is advantageous because it can be amended to work for both large, loud colonies and small, faint colonies without becoming as time consuming as the manual analysis methodology.

The use of manual and automated or semiautomated acoustic analysis methods are common across taxa including amphibians, birds, and primates (Goyette et al. 2011; Digby et al. 2013; Heinicke et al. 2015; Williams et al. 2018; Larsen et al. 2021; Symes et al. 2022). The relative success of automated or manual analyses will likely depend on the research question and species of interest. Our results add to a suite of examples where manual acoustic analysis is the most time-consuming methodology but accurate in locating vocalizations of interest and of various sound qualities (Swinston and Mennill 2009; Goyette et al. 2011; Digby et al. 2013; Mellinger and Heimlich 2013; Symes et al. 2022). Manual analysis is suggested to be best for animals with low vocalization rates and when accurate accounts of temporal patterns in call frequency are needed (Swinston and Mennill 2009). For example, in three woodpecker (order Piciformes) species, a semiautomated template matching analysis method required less time but located fewer target sounds than manual analysis, likely in part as a result of infrequent calling by woodpeckers (Swinston and Mennill 2009). Overall, studies relying only on manual analysis are limited in locality and scope because this method is not feasible on large scales (Swinston and Mennill 2009; Digby et al. 2013; Williams et al. 2018; Larsen et al. 2021; Symes et al. 2022).

For studies on large scales, acoustic analysis incorporating an automated aspect to detect vocalizations of interest is generally more efficient (Bardeli et al. 2010; Goyette et al. 2011; Digby et al. 2013; Frommolt and Tauchert 2014; Stowell 2018). However, automated analysis usually results in high rates of false positives and misses recordings containing vocalizations of interest, which must be tolerated if using a similar methodology (Bardeli et al. 2010; Goyette et al. 2011; Digby et al. 2013; Frommolt and Tauchert 2014; Stowell 2018). A component of automated analysis is likely to work best for studies generating many recordings and for species with frequent and readily detectable vocalizations (Swinston and Mennill 2009; Digby et al. 2013). Tricolored blackbirds nest in synchronous colonies; therefore, vocalization rates of males, females, and nestlings are frequent enough to rely on automated detection where occasionally vocalizations might be missed. This is especially evident in the use of semiautomated detection for female song, which had the lowest odds of being correctly detected and resulted in generally high recall compared with low precision to obtain sufficient detections of the vocalization across the breeding season (Table 1; Figure 2). Automated and semiautomated analysis can suffer from missed recordings and high error rates, but studies with large data sets and scalability will benefit from the efficiency of such analysis methodologies (Stowell 2018).

The results from our study are consistent with those where automated or semiautomated detection performed poorly at detecting target sounds during times of high wind, high background noise, overlapping calls, and faint vocalizations (Bardeli et al. 2010; Goyette et al. 2011; Ehnes and Foote 2015; Gibb et al. 2018; Linares et al. 2022; Symes et al. 2022). All colonies were affected by high wind, but this is especially apparent in the difference of odds of correct detections (Table 3) and recall rates at each site (Table 2). The Hay Landfill colony was located adjacent to a busy road with other organisms using the wetland and had the lowest odds of correct detections and low recall rates (Table 2). The Denverton Creek colony had higher recall rates, but the odds of correct detections at this site (Table 3) were still low compared with the Rush Ranch colony because the ARU at the Denverton Creek site was located farthest from the colony (see “Study Area”), resulting in faint vocalizations. At the Rush Ranch colony, there was little traffic and the ARU was placed near the colony, resulting in the highest odds of a correct detection among the three sites. Furthermore, faint vocalizations in our study primarily resulted from female song and nestling calls. Many recordings containing female song were returned through the pattern matching algorithm as evidenced by high recall rates (Table 2), but female songs also had the lowest odds of a correct detection out of all call types. Contrastingly, nestling calls proved to be difficult to initially detect through low recall rates, but had the highest odds of being correctly detected when pattern matching algorithms located recordings containing this vocalization. For male song where individuals often call frequently but in rapid succession to one another, semiautomated detection resulted in high recall rates but very high rates of FPs (Table 2) and low odds of correct detection (Table 3). Additionally, pattern matching algorithms completely failed to detect male chorus where individuals call simultaneously. Such confounding factors with automated detection are therefore important to consider when implementing different analysis techniques and interpreting results from acoustic data sets.

Our semiautomated methodology was successful in extracting key behaviors and breeding attempts from tricolored blackbird colonies, but this method was hampered by high error rates and had to be adapted to work for colonies with atypical breeding behavior. One limitation of this study is the generally high FP rates using the pattern matching function in the semiautomated analysis to detect tricolored blackbird calls. However, to calculate these statistics we used the recordings that have been manually analyzed, which were only a subset of all recordings. Over the course of a breeding season, the ARUs collected >5,000 recordings. Thus, while the high FP rates are concerning, we still obtained large numbers of true detections in less time than manual analysis, so we tolerated these rates. Additionally, in instances when pattern matching functions completely failed, we relied on mini-manual analysis to evaluate colonies and tolerated a low number of recordings detected with vocalizations. Using few recordings may lead to unreliable results, but this is typically the case for the inconsistent presence of birds or when birds fail to breed. To quickly verify what happened over the course of a breeding season, we do not need to utilize many recordings. Similarly, for small colonies or poorly placed ARUs we also expect to use mini-manual analysis instead of pattern matching algorithms. Most of the recordings obtained from such colonies are likely to be of low quality (Digby et al. 2013), so roughly analyzing fewer low-quality recordings is likely to be a better use of time than manually analyzing the entire data set. If a colony of tricolored blackbirds is present, breeding birds typically call frequently enough that using fewer recordings and focusing efforts on edge analysis is feasible. Overall, semiautomated analysis remains useful because it can be adapted to work for loud and quiet colonies with and without successful breeding attempts and provides an efficient way to analyze data from multiple vocalizations.

Our study provides further evidence of the utility of bioacoustic monitoring to detect breeding behaviors in tricolored blackbird breeding colonies and presents a novel and more efficient semiautomated analysis approach compared with previous methodologies (Schackwitz et al. 2020). In the future, we hope to further reduce analysis time and the need for edge analysis by incorporating prior noise reduction (Bardeli et al. 2010; Gibb et al. 2018; Stowell 2018) or the use of machine learning and neural networks (Deecke et al. 1999; Dugan et al. 2013; Banga et al. 2020; Kiskin et al. 2020; Zhong et al. 2020; Clink and Klinck 2021). While technological improvements are developing, results obtained by the automated detection of vocalizations continue to be influenced by harsh weather or other background noises, faint vocalizations, and overlapping or masked calls (Marin Cudraz et al. 2019). Thus, for our work, it remains essential to employ an analysis methodology incorporating manual and semiautomated analysis components. We also demonstrated that by acoustically monitoring the same colonies through time, we can more thoroughly understand the conditions under which breeding colonies succeed or fail and contribute to efforts to document breeding season changes in response to environmental conditions and climate change. Our results will also assist efforts to better understand tricolored blackbird population dynamics and may contribute to efforts to manage and protect the species. We believe that our analytical methods may be applicable to a wide range of vertebrate species where bioacoustic monitoring may be used to detect presence and document phenological changes under climate change and other ecosystem stressors.

Please note: The Journal of Fish and Wildlife Management is not responsible for the content or functionality of any supplemental material. Queries should be directed to the corresponding author for the article.

Table S1. Results of the analysis of deviance to determine what factor accounts for the most deviance in semiautomated detection of tricolored blackbird Agelaius tricolor vocalizations in California in 2019. Acoustic data from three tricolored blackbird breeding colonies were analyzed using both manual and semiautomated methodologies in 2019. The semiautomated analysis was then applied to the manually analyzed data to validate the performance of the semiautomated method before applying it to data from other years.

Available: https://doi.org/10.3996/JFWM-22-065.S1 (20 KB DOCX)

Table S2. Dates of breeding transitions and stages obtained from analysis of acoustic data for three tricolored blackbird Agelaius tricolor colonies in California from 2019 to 2021.

Available: https://doi.org/10.3996/JFWM-22-065.S2 (20 KB DOCX)

Figure S1. Results from semiautomated analysis for Denverton Creek (A–B), Hay Landfill (C–E), and Rush Ranch (F–G) tricolored blackbird Agelaius tricolor colonies in California in 2020. Analysis of acoustic data from tricolored blackbirds was compared using a manual and semiautomated approach with the goal of obtaining relevant data on breeding stages while minimizing analysis time using the semiautomated approach. Green represents male song, orange represents male chorus, purple represents female song, and blue represents begging calls from hatchlings, nestlings, and fledglings. Darker colors correspond to higher frequencies of each vocalization per day. At Denverton Creek in 2020, there was no evidence for successful breeding although males were present at the site. (A) Pattern matching algorithm frequencies. (B) Mini-manual analysis frequencies to confirm the lack of female song and nestling and fledgling call results from the pattern matching algorithms. At Hay Landfill, there were two asynchronous breeding pulses as evidenced by significant overlap in male chorus, female song, and begging calls. Hatching and fledgling dates were therefore approximated based on what we could detect in recordings. (C) Pattern matching algorithm frequencies. (D) Edge analysis results for the onset of male chorus, hatchling calls, and fledgling dispersal. (E) Graphical summary of breeding stages for the colony. At Rush Ranch in 2020, males sporadically were detected at the site. (F) Pattern matching algorithm frequencies. (G) Mini-manual analysis frequencies to confirm the lack of vocalization resulting from the pattern matching algorithm.

Available: https://doi.org/10.3996/JFWM-22-065.S3 (256 KB DOCX)

Figure S2. Results from semiautomated analysis for Denverton Creek (A–B), Hay Landfill (C–E), and Rush Ranch (F–G) tricolored blackbird Agelaius tricolor colonies in California in 2021. Analysis of acoustic data from tricolored blackbirds was compared using a manual and semiautomated approach with the goal of obtaining relevant data on breeding stages while minimizing analysis time using the semiautomated approach. Green represents male song, orange represents male chorus, purple represents female song, and blue represents begging calls from hatchlings, nestlings, and fledglings. Darker colors correspond to higher frequencies of each vocalization per day. At Denverton Creek in 2021, there was no evidence for successful breeding although males were present at the site. (A) Pattern matching algorithm frequencies. (B) Mini-manual analysis frequencies to confirm the lack of female song and nestling and fledgling call results from the pattern matching algorithms. At Hay Landfill in 2021, there was at least one asynchronous breeding pulses as evidenced by significant overlap in male chorus, female song, and begging calls. (C) Pattern matching algorithm frequencies. (D) Edge analysis results for the onset of male chorus and hatchling calls. The automated recording unit (ARU) ran out of battery before nestlings were old enough to disperse. Nestlings were detected the last day the ARU was active (E). Graphical summary of breeding stages for the colony. At Rush Ranch in 2021, males sporadically were detected at the site. (F) Pattern matching algorithm frequencies. (G) Mini-manual analysis frequencies to confirm the lack of vocalization resulting from the pattern matching algorithm.

Available: https://doi.org/10.3996/JFWM-22-065.S4 (240 KB DOCX)

We thank Robert Meese for comments on early drafts of this manuscript and providing expert information on tricolored blackbird behaviors. We also thank Leo Salas for assistance with the statistical analysis for this research and Mike Schackwitz for the software necessary to graph and visualize the analyzed acoustic data. Further, we are grateful to Arbimon for the free, open-source acoustic analysis software. Finally, we thank the Associate Editor and the reviewers for their comments in refining earlier versions of this manuscript. Funding for this research was provided by the Solano County Propagation Fund administered by the Napa–Solano Audubon Society to M.H.

Any use of trade, product, website, or firm names in this publication is for descriptive purposes only and does not imply endorsement by the U.S. Government.