Effects of Training on Side-Scan Sonar Use as a Fish Survey Tool: A Case Study of Alligator Gar

Consumer-grade side-scan sonar (SSS) is an increasingly popular tool in fisheries management. First applied to assess aquatic habitat features in rivers (Kaeser and Litts 2008, 2010), its use has expanded to serving as a fisheries survey instrument. Bollinger and Kline (2017) used SSS to successfully derive biomass estimates of fishes using artificial reefs in the Gulf of Mexico, and Fleming et al. (2018) and Vine et al. (2019) used SSS to estimate the abundance and distribution of adult (i.e., > 1 m total length) Alligator Gar Atractosteus spatula and Atlantic Sturgeon Acipenser oxyrinchus, respectively. The authors of these studies noted significant increases in the efficiency, versatility, and affordability of SSS when compared with more traditional fisheries sampling methods, indicating a likelihood for continued growth in use of the technology by management agencies.

An important consideration when adopting novel sampling methodologies into agency management programs is the time investment required for staff to become proficient. Because human interpretation is an integral part of SSS image analysis, skill and experience of individual users can affect the accuracy of surveys (Gonzalez-Socoloske and Olivera-Gomez 2012; Flowers and Hightower 2013). Thus, a need exists to understand the abilities of novice users and assess the potential for training to improve their proficiency.

Alligator Gar stocks have experienced a significant increase in recreational fishing pressure across much of their current distribution in recent years. In response, agencies have increased efforts to monitor these fisheries (Smith et al. 2020). Population assessment using direct sampling rarely yields numbers sufficient for robust statistical analyses or data trends because catch rates are typically low and variable, even in healthy populations (e.g., 5 fish/h of gill-net effort; Bodine et al. 2015). This prompted the Texas Parks and Wildlife Department to consider the incorporation of SSS into standardized monitoring efforts. Although Fleming et al. (2018) validated the use of SSS for surveying Alligator Gar, the authors did not assess whether the technique could be broadly applied by inexperienced users. Therefore, the objective of our study was to characterize the ability of novice users to identify Alligator Gar in SSS imagery and assess the effect of a 2-h training exercise on user performance. Our results will be useful in guiding the use of SSS in management and conservation efforts for Alligator Gar and other large-bodied fishes.

We solicited staff from the Texas Parks and Wildlife Department - Inland Fisheries Division to participate in the study on a voluntary basis. Criteria for participation consisted of individuals having familiarity with the species and SSS technologies, but no formal experience or training in identifying Alligator Gar in SSS imagery. In 2015, we provided each participant the same set of 20 SSS images, randomly selected from a set of 40 images collected to validate the use of SSS to survey Alligator Gar (Fleming et al. 2018), and asked them identify all Alligator Gar present in each image. Upon completion, we surveyed each participant to rate their confidence in identifying Alligator Gar in the imagery, based on a five-point, Likert-type scale.

Participants then attended a 2-h training session, which detailed the characteristics employed to discriminate Alligator Gar from other features in SSS imagery in the Fleming et al. (2018) validation study. These included relative size, shape, and reflectivity of Alligator Gar (and associated acoustic shadows) in contrast to other fishes and habitat features (e.g., large woody and anthropogenic debris), as well as their behavioral characteristics of shoaling and suspension in the water column. Following training, we then asked each participant to identify all Alligator Gar in the remaining 20 images in the data set, followed again by responding to the same survey question characterizing their confidence in sonar image interpretation.

We assessed each participant's ability to distinguish Alligator Gar both pretraining and posttraining by comparing their observed counts with expected counts determined during the Fleming et al. (2018) validation study (Table S1, Supplemental Material). Sixteen of the SSS images included in the pretraining set contained between one and eight Alligator Gar, whereas the remaining four images contained no fish. In the posttraining image set, 17 contained between 1 and 13 fish, with the remainder containing no fish. We calculated mean accuracy, defined as the mean difference between observed and expected counts, as well as precision, measured as the standard deviation [σ] about the mean accuracy estimate for each participant. We also used images in the pretraining and posttraining sets that contained no Alligator Gar to assess the rate of false positives (i.e., Alligator Gar were identified in the image when absent).

We assessed statistical differences in pretraining and posttraining confidence, the frequency of false positives, and accuracy. For survey question responses and false positives, we calculated the difference between pretraining and posttraining data sets and compared them with a null distribution with Wilcoxon Rank Sum tests. We assessed pretraining and posttraining accuracy for each participant with t-tests as well as across all participants using a paired t-test (SAS Enterprise Guide 7.1). We considered differences significant at an α = 0.05 level of confidence.

Eight individuals participated in the study. Prior to training, seven of the eight participants lacked confidence in their ability to identify Alligator Gar in the sonar imagery (Figure 1). Mean accuracy among participants ranged from −2.6 to 1.3 fish; precision (σ) among participants ranged from ±1.2 to ±2.4 fish (Table 1). Most participants generally underestimated the number of Alligator Gar present in the pretraining imagery (i.e., negative mean accuracy values). False positives were common among participants prior to training, with individuals identifying between one and three fish in 50 to 75% of the images containing no Alligator Gar (Table 1).

Posttraining, participants reported significant increases in confidence during image interpretation (S = 92; df = 7; P = 0.007; Figure 1). Mean accuracy significantly differed between pretraining and posttraining image sets for three individual participants (Table 1), but not across all participants combined (T = 1.29; df = 7; P = 0.24). All participants underestimated Alligator Gar abundance in the imagery, with the exception of one participant who showed no bias. Precision following training ranged from ±1.6 to ±3.2 fish. Posttraining, false positives were significantly reduced (S = 100, df = 7, P = 0.0002), with half of the participants recording no false positives (Table 1).

Given increasing use of SSS in fisheries management and growing efforts to actively manage Alligator Gar populations, understanding the abilities of novice readers and the potential effect of training on image interpretation is important. Quantifying the magnitude and directionality of biases, as well as the potential for training to increase the accuracy and precision of abundance estimates, can be used to further the development of standard sampling procedures for SSS. To our knowledge, this study represents a small but important first step in assessing these factors.

Our results demonstrate that participants were able to consistently determine the number of Alligator Gar in the imagery to within a few fish, even without prior experience or training. However, the training exercise did meaningfully improve several of the metrics we examined. The most notable effect of training was the reduction in false positives, which indicated that the training exercise was effective at reducing misidentification of objects in the imagery as Alligator Gar. Reducing the occurrence of false positives is important because even low rates of overestimation can substantially bias estimated occupancy, colonization, and extinction rates (Royle and Link 2006; McClintock et al. 2010a, 2010b; Miller et al. 2011, 2013). Minimizing or eliminating sources of overestimation is particularly important for rare species (Miller et al. 2011), including remnant and recovering stocks of Alligator Gar.

The significant differences we observed between pretraining and posttraining accuracy for three of the study participants were associated with a change from overestimation to underestimation. This change also improved consistency among participants—mean accuracy following training underestimated abundance by one or two fish among all individuals. Thus, the effect of training on accuracy was consistent with the reduction in false positives, further reducing the occurrence of overestimation.

Training also improved participant's confidence surrounding the identification of Alligator Gar in the imagery. Quality assurance–quality control programs that provide opportunities for biologists to gain interpretive experience have been advocated in the literature pertaining to fish age estimation for decades (e.g., Campana 2001; Buckmeier 2002; Morison et al. 2005; Dembkowski et al. 2019). Increased confidence improves efficiency and reduces uncertainty associated with estimates derived from the image interpretation data.

We conclude that, irrespective of training, SSS can be easily incorporated into large-scale fishery monitoring efforts for the Alligator Gar, and by extension, other large-bodied fishes, such as sturgeons. However, a minimal investment in training can improve consistency and reduce uncertainty among novice users. Given the potential negative consequences of overestimation, particularly where remnant populations are concerned, we recommend that agencies implement quality assurance–quality control programs for the use of SSS in population monitoring and assessment, including image interpretation training, such as described here. Experienced readers of aging structures have also been shown to decrease in accuracy over time (Campana et al. 1995; Campana 2001; Dembkowski et al. 2019). Thus, periodic retraining of experienced image interpreters may also be necessary for long-term monitoring efforts.

Please note: The Journal of Fish and Wildlife Management is not responsible for the content or functionality of any supplemental material. Queries should be directed to the corresponding author for the article.

Table S1. Pretraining and posttraining study data for participants asked to identify Alligator Gar Atractosteus spatula in a series of 20 side-scan images before and after a practical training session in 2015. Column headings A through H refer to individual participants.

Available: https://doi.org/10.3996/JFWM-21-026.S1 (13 KB XLSX)

We thank T. Bister, M. Baird, L. Wright, A. Barkoh, N. Smith, K. Bodine, D. Wilson, and A. Stevens for their participation in this study. Funding for completion of this project was provided through Federal Aid in Sportfish Restoration Grant F-231-R provided to the Texas Parks and Wildlife Department. We thank the editorial staff, three anonymous reviewers, and the Associate Editor for their assistance in improving our manuscript. The funding source did not have a role in conducting this research or the production of this manuscript.

Any use of trade, product, website, or firm names in this publication is for descriptive purposes only and does not imply endorsement by the U.S. Government.