LABORATORY CONCORDANCE STUDY FOR THE MOLECULAR DETECTION OF MYCOPLASMA OVIPNEUMONIAE

Multiple species of the bacterial genus Mycoplasma are known respiratory-associated pathogens of humans and animals. One example is Mycoplasma ovipneumoniae, a pathogen associated with multifactorial, polymicrobial respiratory disease in members of the subfamily Caprinae. This bacterium has been reported in association with high mortality in bighorn sheep (Ovis canadensis; Besser et al. 2012), with muskoxen (Ovibos moschatus) pneumonia epizootics (Handeland et al. 2014), and in association with broncho-pneumonia in seven mountain goat (Oreamnos americanus) kids that died between 2011 and 2015 (Wolff et al. 2019). Lack of detection of this bacterium, and absence of mass mortality events in Dall's sheep (Ovis dalli dalli) led to the misconception that M. ovipneumoniae was not present in Alaska, US. While investigating Mycoplasma spp. presence in Alaska wildlife, we detected M. ovipneumoniae in Dall's sheep and non-Caprinae species previously considered to be outside of this pathogen's host range, including moose (Alces alces gigas) and caribou (Rangifer tarandus granti; Highland et al. 2018). Although no evidence exists linking mass respiratory disease mortalities to M. ovipneumoniae in Alaska wildlife, M. ovipneumoniae was detected in nasal swab and lung tissue specimens collected from a yearling caribou that died of polymicrobial broncho-pneumonia (Rovani et al. 2019). Additionally, M. ovipneumoniae was detected in tissues from several (<5) cases of Dall's sheep diagnosed with polymicrobial bacterial pneumonia opportunistically sampled 2004–19 (K.B.B. pers. comm.).

Reliable and accurate identification of individuals and populations carrying a pathogen of concern is critical for understanding potential health impacts; assessing transmission risk; and developing further management and research efforts. Molecular techniques, such as PCR assays, are widely used to detect the presence of pathogens, including M. ovipneumoniae. Interlaboratory discrepancies may occur due to differences in DNA extraction methods and efficiencies, including the volume or amount of the sample used and PCR inhibitor coextraction. Interassay discrepancies may occur due to PCR parameters including primer binding efficiency; primer and template concentrations in the final reaction volume; cycling conditions (i.e., time and temperature); and number of cycles. All of these can influence the sensitivity, specificity, and detection limit of an assay. The effect of these potential differences has been previously assessed by a ring study (Walsh et al. 2016) looking at agreement among six laboratories for detection of M. ovipneumoniae using various extraction and detection methods. That study, focused on a small number of homogenates of previously tested samples to facilitate the analysis, found overall good agreement (median agreement 89–95%) among participating laboratories.

Our study analyzed three PCR amplicons within the 16S rRNA and intergenic regions (Fig. 1) that have been shown to be effective in detecting and/or differentiating Mycoplasma species and describing strain diversity. The Universal Mycoplasma PCR (UM-PCR; van Kuppeveld et al. 1992) was developed to detect the presence of mycoplasma species. McAuliffe et al. (2003) developed primers to specifically detect M. ovipneumoniae (LM-PCR), but found these primers also detect other mycoplasmas, including Mycoplasma bovoculi. The intergenic spacer region (IGS-PCR; Besser et al. 2012) is more variable and is used to help differentiate strains of M. ovipneumoniae but is not generally used as primary method of M. ovipneumoniae detection. Because published assays have proven to be nonspecific for M. ovipneumoniae detection, sequencing PCR-generated amplicons is critical for confirming the presence of a specific organism. Although a M. ovipneumoniae–specific real-time PCR assay exists (Manlove et al. 2019), it does not produce a sequenceable product, so direct confirmation of a detection being M. ovipneumoniae is not possible. Culture and subsequent identification methods are often used for identifying the presence of a bacterium within biological samples (Laupland and Valinquette 2013); this method is often time consuming and insensitive for M. ovipneumoniae, due to the fastidious nature of this bacterium (Weiser et al. 2012; Ackerman et al. 2019).

For health surveys of respiratory microbial pathogens, the Alaska Department of Fish and Game (ADF&G) frequently has samples analyzed for mycoplasmas other than M. ovipneumoniae. Thus, understanding how the more generalized mycoplasma tests compare is important. While surveying nasal swabs from Alaska ungulates using the LM40 protocol (Highland et al. 2018) to detect Mycoplasma spp., including an uncharacterized mycoplasma found in Alaska moose (Herndon et al. 2021), M. ovipneumoniae was detected in nasal swabs from caribou that were not previously thought to be susceptible (Highland et al. 2018). Additionally, the LM40 protocol was used to detect M. ovipneumoniae in Dall's sheep populations that had been previously extensively surveyed using available methods and found to be negative (ADF&G data). Understanding the sources of differing detection frequencies is crucial for management of these wildlife populations.

When the true-positive or true-negative status of an individual host is unknown, it can be difficult to interpret laboratory result discrepancies. Concordance analysis, while not a means to determine which testing method is more “correct,” can be used to determine the degree of agreement between two assays. Understanding how results might compare between tests can guide researchers and managers to decide which tests are most appropriate for their study goals, and to help interpret whether differences in results might be test-related or due to other factors (such as presence or absence of other microorganisms in a population). Often, concordance analysis examines overall agreement (the percentage of tests with the same result whether positive or negative). Interpreting this statistic can be limited, especially if an event is rare. Thus, calculating both positive and negative agreement separately can provide information on where tests differ (Center for Devices and Radiological Health 2007). However, when an event is rare, even a single chance difference between test results can result in a large difference in the percentage of samples with that result. The Cohen kappa has been proposed to assess the quality of agreement, by determining how much the observed agreement (both positive and negative) differs from what would be expected by chance alone (Kwiecien et al. 2011). Although this assessment method also has limitations when events are rare, it can provide additional information when interpreting results (Viera and Garrett 2005).

During previous routine respiratory pathogen surveillance in Alaska wildlife using PCR to detect Mycoplasma spp., significant interlaboratory differences in M. ovipneumoniae detection rates had become apparent. To help understand if the differences in detection rates were due to differences in organism prevalence in a population; field sample collection; or sample processing and testing methods at different laboratories; we performed concordance analysis of results from three different PCR assays conducted at two laboratories. Because all samples sent to the laboratories were from the same set of individuals, the effect of organism prevalence within a population on detection probability would be the same for all tests. We controlled for sampling differences by testing split biological samples. Our study aimed to estimate the differences in agreement among multiple PCR techniques used to detect M. ovipneumoniae in Alaska wild ungulates, to facilitate future research and management efforts.

The ADF&G personnel collected nasal swabs during July–September 2018 from both live-captured and hunter-harvested wild ungulates using a dual swab collection system (BBL CultureSwab EZ-II, Becton, Dickinson and Company, Sparks, Maryland, USA). They placed the two swabs collected simultaneously from each animal into a vial containing universal transport media (UTM; HealthLink, Diagnostic Hybrids Inc., Athens, Ohio, USA) and stored at 3–5 C until shipped. Staff of ADF&G shipped the chilled samples within an average of 3 d (range 0–9 d) to the Fairbanks ADF&G office where, within 24 hr of receipt, we split them into three aliquots in a biosafety cabinet. We placed one of the two swabs, and 1 mL of UTM into a sterile 2.0 mL cryovial, left the second swab and 1 mL of UTM in the original sample vial, and archived the remaining UTM in a 1.2 mL cryovial at –70 C. We shipped both aliquots with swabs separated into plastic bags by species and vial type chilled and overnight within 3 d of splitting (mean 1 d) to the Washington Animal Disease Diagnostic Laboratory (WADDL) in Pullman, Washington, US. The US Department of Agriculture (USDA), Agricultural Research Service, Animal Disease Research Unit, also in Pullman, picked up the original sample vial aliquots from WADDL the day of arrival. Of the 354 samples tested by both laboratories, we omitted eight samples from the concordance analysis; four (two mountain goats and two Dall's sheep) because of poor or mixed sequence data, and four (one Dall's sheep and three caribou) for sequence failures. The concordance analysis included a total of 346 samples from four species (Dall's sheep, mountain goats, caribou, and moose), and two families (Bovidae, subfamily Caprinae; and Cervidae) (Table 1). Most samples were from recently harvested, unskinned, and not previously frozen hunter-harvested animal heads presented to ADF&G offices for management and research data collection purposes. Additionally, there were 28 mountain goat samples from live-captured individuals.

We analyzed samples for detection of M. ovipneumoniae using three different PCR assays (Table 2), followed by Sanger sequencing (Sanger et al. 1980) of amplicon products of expected size, as visualized by gel electrophoresis. Extraction of DNA and the LM40-PCR assay performed in the USDA laboratory have been previously described (Highland et al. 2018). The USDA laboratory's IGS-PCR protocol used published PCR primers (Besser et al. 2012) under optimized protocol conditions as follows: primers were each used at 0.4 µM in 25 µL reactions containing 1 µL of primer (0.4 µM final), 2 µL of sample DNA, 12.5 µL of Multiplex PCR Master Mix (Qiagen, Germantown, Maryland, USA), and 8.5 µL water. Cycler conditions included a 15 min denaturation cycle followed by 35 cycles of 95 C denaturation (30 s), 55 C annealing (30 s), and 72 C extension (1 min), with a final 2 min extension and 4 C hold. Because UM-PCR was performed at a fee-for-service diagnostic laboratory, details of DNA extraction and PCR assay procedures are proprietary and not provided with the test results. Each laboratory analyzed sequence data via BLASTN (NCBI 2020) to determine the highest identity match (UM-PCR) or highest identity to type strains (LM40 and IGS). For the LM40 amplified 16S rRNA region, we considered sequences with ≥97% identity to M. ovipneumoniae type strain 16SrRNA (GenBank no. NR_025989.1) a detection for M. ovipneumoniae. For the IGS sequence data, if the highest identity match was M. ovipneumoniae, we considered the sample a detection, and then compared it with the Y98 type strain genome sequence (GenBank GCA_001715085.1), percent identity to this sequence ranged from 96% to 97%. Detections for the UM-primer amplified 16S rRNA sequences were reported by WADDL as being identical to M. ovipneumoniae GenBank MH045512 and others. We considered any samples that did not produce an amplicon of appropriate size, or that produced an amplicon of sequence inconsistent with being M. ovipneumoniae, as being a nondetection for M. ovipneumoniae.

For concordance calculations, we compared detection and nondetection results for the three PCR tests (Fig. 2). Contingency tables for each laboratory test pairing were created. We evaluated concordance using overall concordance, as well as positive and negative agreement. The quality of the agreement was assessed using the Cohen kappa. We interpreted results using standard values (Landis and Koch 1977; Fig. 3).

For LM40 and IGS-PCR amplicons, we trimmed forward and reverse primer generated sequences to remove low-quality sequence regions, merged, and manually inspected for errors. We aligned LM40 consensus sequences using the Clustal plugin for Sequencher (GeneCodes, Ann Arbor, Michigan, USA), and calculated percent identity matrix for the LM40 sequences using CLC Genomics Workbench (Qiagen, Redwood City, California, USA). We omitted one sequence from a caribou from the identity matrix because it was incomplete.

In addition, we had each sample with a M. ovipneumoniae detection by UM-PCR sent to Dr. Thomas Besser's laboratory at Washington State University (WSU-TBL) in Pullman, Washington for multilocus sequence typing (MLST; Cassirer et al. 2017). This is a method of differentiating strains of M. ovipneumoniae by concatenating sequences from multiple PCR amplicons amplifying different parts of the genome by different primers. This four-locus MLST included sequences using the LM (McAuliffe et al. 2003) and IGS (Besser et al. 2012) primers. Although MLST is traditionally applied to pure isolated cultures, for this study amplification and sequencing was performed directly on DNA extractions as has been described previously (Cassirer et al. 2017; Highland et al. 2018; Kamath et al. 2019). The results of the four-locus MLST analysis are not part of this study; however, we compared the LM and IGS-PCR amplicon sequences that were generated from WSU-TBL to the sequences generated in the USDA laboratory using BLASTN. Because WSU-TBL performed MLST on samples that WADDL had detected M. ovipneumoniae, this provided a means to genetically compare sequence results of M. ovipneumoniae detected by both the USDA and WADDL laboratories.

The WADDL tested a subset of 291 samples using a M. ovipneumoniae–specific real-time PCR (Manlove et al. 2019). We used the results as an intralaboratory check. Based on their internal validation studies, WADDL states that their M. ovipneumoniae–specific PCR assay is sensitive and specific for M. ovipneumoniae, and requires fewer colony forming units per milliliter to detect M. ovipneumoniae than the UM-PCR (WADDL 2020). However, sequencing cannot be performed on an amplicon generated by this M. ovipneumoniae–specific PCR test.

We determined in vitro method detection limits (MDL) for the LM40 and IGS-PCR assays by culturing the Y98 type strain of M. ovipneumoniae (American Type Culture Collection, Manassas, Virginia, USA) in SP4 glucose broth (Thermo-Fisher, Waltham, Massachusetts, USA) at 37 C and shaking at 200 × G. We monitored cultures for color change, harvested bacteria by centrifugation (15,000 × G for 15 min at 15 C), and extracted DNA as previously described (Highland et al. 2018). We quantified DNA (Qubit, Life Technologies Corporation, Eugene, Oregon, USA) and made 10-fold serial dilutions (10⁶ to 0 copies). We performed LM40 and IGS-PCRs on both pure culture-extracted DNA and culture-extracted DNA spiked into three M. ovipneumoniae nondetection DNA samples. Spiked nondetection samples included DNA extracted from UTM samples from a caribou and a Dall's sheep, and from a dry swab collected from a mountain goat. We assessed the LM40 and IGS-PCR limits of detection by amplicon visualization using standard gel electrophoresis.

We detected M. ovipneumoniae by more than one PCR assay only in caribou samples (Table 1). Of the 18 caribou samples with M. ovipneumoniae detection by at least one PCR assay, nine had detection by at least two assays, and seven had detection by all three assays (Table 3). In addition to caribou, we detected M. ovipneumoniae in Dall's sheep and mountain goats using LM40.

We found that UM and IGS had excellent quality of concordance (Table 4). Positive agreement was not high between LM40 and IGS nor between LM40 and UM (ranged from 37% to 41%); however, overall concordance was still 93%. Because of the low positive agreement, we considered the quality of concordance to be only fair between those tests (Table 4).

Sequences from the LM40 detections fell into two groups, one with ≥99% identity to the Y98 M. ovipneumoniae reference strain and the other with 97–98% identity to Y98 (Table 5). All detections from the IGS and UM-PCR assays were in caribou that had LM40 results that fell in the second group (97–98% identity to Y98).

Of the nine caribou samples that were detected via the UM-PCR, WSU-TBL was able to produce both LM and IGS sequences from seven, and all had identical results. For the other two caribou samples, WSU-TBL analysis produced sequences only for the LM locus for one sample, and for the IGS locus for the other. All eight WSU-TBL LM and WSU-TBL IGS amplicon sequences were identical at these loci. These nine caribou samples were also detections by either LM40 or IGS-PCR assays performed in the USDA laboratory. The IGS amplicon sequences generated by WSU-TBL and by USDA were identical for these samples. Six of the eight WSU-TBL LM sequences were part of the group of eleven caribou (CAR4-14 in Table 5) that were detected and sequenced by USDA LM40-PCR that had 100% identity to each other. The LM40 generated sequences for the other two caribou (CAR 15 and CAR17 in Table 5) each differed from the WSU-TBL generated LM sequences at a single site, an insertion in CAR15 and a deletion in CAR17.

All nine detections using the UM-PCR were also detections using the M. ovipneumoniae–specific PCR, and WADDL did not find any additional detections using the M. ovipneumoniae–specific PCR in the subset tested.

In vitro MDL testing found that the LM40-PCR and the IGS-PCR have similar detection limits (≥10 copies) when DNA extracted from Y98 pure culture media is used as the PCR template. However, for each of the three nondetection samples that were spiked with serial dilutions of the pure Y98 culture DNA extract, amplicons were visualized at ≥10 copies using LM40-PCR and ≥10² copies using IGS-PCR (data not shown).

We found that concordance was fair to excellent depending on the tests being compared. Negative concordance was excellent across all assays; however, this should be interpreted conservatively, considering the relatively low frequency of detections. When a specific result is a rare event (as detections are in this study), high agreement is expected for the more common result. The highest overall concordance was between UM and IGS-PCR assays. Concordance between LM40 and UM, and between LM40 and IGS-PCR, assays were similar. The LM40-PCR assay detected M. ovipneumoniae in more individuals than the other tests. Test differences were more important for detection agreement than interlaboratory differences, because one of the assays used by the research laboratory had excellent quality of agreement with the fee-for-service laboratory.

The increased frequency of detection with the LM40 assay introduces the concern for carry-over contamination. This is a concern for PCR detection assays, especially when maximizing the number of cycles to improve MDL (Persing 1991; Aslanzadeh 2004). Within our study the higher number of LM40-PCR detections relative to the other PCR methods were unlikely to be caused by carry-over contamination. Testing at the USDA laboratory included negative control samples extracted alongside test samples (one negative control per 6–10 test sample DNA extractions) that were dispersed among test samples throughout the 96-well PCR plate; these negative control samples had no visualized amplicon. Additionally, detection samples were interspersed among nondetection samples, and groups of identical sequences from samples in proximity during DNA extractions or within the same PCR run were not observed.

Some differences in detections appear to be associated with genotype. All UM-PCR and IGS-PCR detections fell within the group of LM40 detections that had 7–8 base pair changes from Y98 (97–98% identity). The group detected only by the LM40-PCR, that had 1–4 base pair changes from Y98 (≥99% identity to Y98), might represent evidence of a strain/genotype with low organism numbers or possibly with poor viability in the host. Further investigations to better understand strain/genotype differences, virulence factors, potential fitness, and how they relate to detection differences are warranted.

Although the nasal microbiota of ungulates in Alaska has not been described in detail, we would expect it to contain a variety of different organisms, potentially including multiple mycoplasma species. Each of the PCR assays used in our study (LM40, IGS, UM) can detect multiple Mycoplasma spp.; thus, sequence analysis not only is important for comparing genotype of samples, but is also necessary to confirm specific detection of M. ovipneumoniae. The presence of multiple mycoplasma species within a sample has the potential to impact detection of M. ovipneumoniae. Of those samples with LM40 detections that were negative for M. ovipneumoniae on UM-PCR (n=23) or IGS-PCR (n=22), UM and IGS-PCR assays detected other mycoplasmas in 2/23 and 3/22, respectively, as determined by sequencing of samples that had visible bands on gel electrophoresis. Additionally, within the subset of samples tested by the M. ovipneumoniae–specific real-time PCR at WADDL (n=291) were two samples in which other mycoplasmas were detected by UM-PCR and that were negative by the M. ovipneumoniae–specific real-time PCR. Although multiple mycoplasmas could still potentially be contributing to differences among the different assays, this does not appear to be the primary reason for differences in detections within samples analyzed here.

Other factors that might influence inter-assay detection differences include sample handling and processing, DNA extraction procedures, and PCR primers and cycling parameters. The DNA extraction technique does not explain the difference in detections between LM40 and IGS, because these two assays were performed using the same DNA extracts. Although extraction procedures at WADDL have not been reported, the high level of concordance between IGS and UM-PCRs suggests that it is unlikely that extraction method alone explains the differences in detection levels among assays; however, the effect of method and sample volume used for DNA extraction in addition to the effect of differences in other PCR factors on detection is worth investigating. It is worth noting that amplification of an organism using PCR is not always successful, even when the organism has previously been detected on other assays. For example, intralaboratory differences were seen with primer amplification success with the LM40 and IGS assays at the USDA, as well as the PCR assays performed for the MLST in the WSU-TBL.

Results from the in vitro MDL assessment using spiked nondetection samples reveal that the LM40-PCR, which runs 40 cycles, had 10-fold better MDL compared with the IGS-PCR. This feasibly accounts for the detection difference seen between these two assays. We were unable to perform similar MDL comparisons between the UM-PCR and LM40-PCR because the details of the UM-PCR methods (DNA extraction and UM-PCR) were not provided with the laboratory results. As per WADDL website (WADDL 2020), the MDL for the PCR assays performed by WADDL is based on cfu/mL, with a reported MDL of 6 cfu/mL for the M. ovipneumoniae–specific PCR method and 600 cfu/mL for the UM-PCR method. However, calculating the MDL by cfu does not account for all the bacteria in the original test sample, because only viable bacteria are accounted for using the cfu method to determine MDL. Therefore, we cannot directly compare the MDL of the USDA's standard PCR with those of the WADDL PCR methods used for M. ovipneumoniae detection without further detail of the extraction (i.e., concentration of DNA extracted from a pure culture M. ovipneumoniae), and the PCR reaction volume and volume of template used (i.e., volume of extracted DNA used per reaction). We recommend further studies to elucidate the reasons for M. ovipneumoniae detection differences among PCR assays.

Wildlife management strategies require efficacious diagnostic tools. It is critical to remember that PCR assays are not infallible. Occasionally primers might not be able to amplify DNA sequences despite success using other primers, even when assays are performed within the same laboratory. Differences in assays used to detect M. ovipneumoniae can influence the result (e.g., specificity, sensitivity, and MDL). Circumstances or situations dictate the tolerance for these differences. For example, increased sensitivity with lower risk of false negatives could be critical for translocations to prevent introduction of a pathogen to a naive herd, whereas increased specificity and a lower risk of false positives might be more important when ensuring that a diagnosis for a specific organism is accurate. The known presence or absence of M. ovipneumoniae has major implications for wildlife health management, especially informing decisions about translocation and reintroduction programs of wild sheep. Managers need to understand the present distribution of this pathogen to identify and select M. ovipneumoniae–free source populations and to mitigate the risks of exposure to M. ovipneumoniae posttranslocation. Local or regional regulations on animal movements (such as the recent amendments; Department of Environmental Conservation 2021) to the animal health regulations in Alaska (18 Alaska Administrative Code [AAC] 36.125 for domestic goat; 18 AAC 36.135 for sheep; 18 AAC 36.215 for disease reporting) can include requirements for the documentation of M. ovipneumoniae–free status. Additionally, gaining an understanding of the historical impacts of M. ovipneumoniae on wildlife populations as part of retrospective studies, and comparing longitudinal studies that use different testing methods, are only effective with a thorough knowledge of how the results of the testing methods compare with each other.

The authors thank the ADF&G staff and hunters as well as D. Sinnett, for providing the samples for this study. We also thank T. Besser, D. Bradway, D. Bruning, A. Crane, L. Dreese, P. Grossman, S. Guritz, T. Kavalok, C. Krenz, T. Lohuis, G. Pendleton, K. White, and K. Wiggles-worth for technical assistance and advice. Financial support for this study was provided though Federal Wildlife Restoration Grant AKW-23, Project 18.74 and by the US Department of Agriculture, Agricultural Research Service, Current Research Information System Project funds 2090-32000-036-00D.