Evaluation of alternative detection methods for foodborne pathogens typically involves comparisons against a “gold standard” culture method, which may produce false-negative (FN) results, particularly under worst-case scenarios such as low contamination levels, difficult-to-detect strains, and challenging food matrices (e.g., matrices with a water activity of <0.6). We used extended enrichment times (up to 72 h for both primary and secondary enrichments) to evaluate a gold standard method for Salmonella detection (the U.S. Food and Drug Administration Bacteriological Analytical Manual [BAM] method) in two low-water-activity foods (dry pet food and chocolate) inoculated at low contamination levels (most probable number ca. 1/25 g) with five Salmonella strains. Strains were selected to include those with a poor ability to grow in enrichment media. Among the 100 pet food and 100 chocolate samples tested, 53 and 50, respectively, were positive with the standard BAM method, and 57 and 59, respectively, were positive with the extended BAM method. Thus, the FN probabilities for the standard BAM method were 7% for pet food and 15% for chocolate. An alternative enzyme immunoassay method for detection of Salmonella in chocolate produced FN probabilities of 6 and 20% when compared against the standard and extended BAM methods, respectively. Detection of Salmonella Mississippi was significantly reduced with the alternative method (P = 0.023) compared with the extended BAM method. We calculated a composite reference standard to further define FN probabilities based on variable results from multiple assays (the standard BAM, extended BAM, and alternative methods). Based on this standard, the enzyme immunoassay for Salmonella detection in chocolate had a 28% FN probability and the standard and extended BAM methods had 23 and 9% FN probabilities, respectively. These results provide a framework for how inclusion of extended enrichment times can facilitate evaluation of alternative detection methods.
Worst-case conditions provide for stringent evaluation of detection methods.
Extended enrichment can assess FN probabilities of culture methods.
BAM methods can have up to a 15% FN probability.
The genus Salmonella comprises two species and >2,600 serotypes. Low-water-activity foods are now recognized as important sources of human and animal Salmonella infections (29). While many different dry foods (e.g., cereal, spices, dairy powders) have been linked to salmonellosis cases and outbreaks (28), for this study, we used pet food and chocolate as matrices for evaluation of Salmonella detection methods. Pet food has been linked to Salmonella infection outbreaks in both humans and pets, such as the 2006 to 2008 outbreak of Salmonella Schwarzengrund infection that caused 79 clinically confirmed human cases (2). Contamination of pet food is particularly concerning because pets can become Salmonella carriers while remaining asymptomatic (10), potentially serving as sources of human infections (17). Chocolate is another example of a low-water-activity food that has been linked to multiple salmonellosis outbreaks. In the mid-1970s, 119 salmonellosis cases were reported in the United States and Canada associated with chocolate contaminated with Salmonella Eastbourne (5, 6). Ten years later, another large outbreak linked to chocolate resulted in 245 reported salmonellosis cases in England and Wales (12), and another chocolate-linked outbreak occurred in Germany from 2001 to 2002, with >439 registered cases (34).
Detection of Salmonella in low-water-activity human and pet foods can be challenging, particularly with the need to detect low-level contamination (e.g., one cell per 25 g) (18). The ability of Salmonella to adapt to extreme conditions allows it to remain viable in dry environments for extended periods of time (28) and makes Salmonella cells more difficult to culture, leading to challenges in detection particularly with rapid methods that have shortened enrichment times relative to standard culture methods. Enrichments times longer than 24 h are more likely to result in overgrowth of pathogens by commensal microbiota but can facilitate resuscitation and subsequent growth of stressed or damaged cells in matrices with low microbial background levels, and enrichments with multiple subculturing steps can enhance the selective enrichment of the target pathogen (18, 24, 25).
The specific aims of this study were to characterize the sensitivity of the U.S. Food and Drug Administration (FDA) Bacteriological Analytical Manual (BAM) culture method for detection of difficult-to-detect Salmonella strains in low-water-activity (<0.6) foods (30) and in parallel to evaluate an alternative method (an enzyme immunoassay) for its ability to detect Salmonella in dark chocolate. The BAM method was evaluated by performing the standard BAM method with additional extended enrichment times for both primary and secondary enrichments. We predicted that this approach would increase the sensitivity for detection of Salmonella. In previous research, extended periods of enrichment improved sensitivity, and a delayed secondary enrichment (due to a prolonged primary enrichment) substantially increased test sensitivity (32, 33). Assay evaluations were designed to simulate worst-case scenario conditions and to identify the failure modes of each method (i.e., conditions under which assays are likely to fail or underperform). These evaluations were conducted by using (i) low inoculation levels (a fractional positive approach), (ii) challenging matrices, and (iii) Salmonella strains that in preliminary screenings had either reduced growth in primary or secondary enrichment media or were inconsistently detected with the alternative detection method.
MATERIALS AND METHODS
Bacterial strains and initial inclusivity screens
A previously published set of 68 Salmonella strains representing 66 unique serotypes (29) (Supplemental Table S1) was used for two initial inclusivity screens. Two strains included in this collection had incomplete serotype information and were further characterized using whole genome sequencing and in silico serotyping with SeqSero (35). This approach allowed us to identify isolates FSL R8-9020 and FSL R8-7977 as serotypes Agona and Oranienburg, respectively (Table S1). The two screens conducted with the overall strain set assessed (i) the ability of the strains to grow in primary enrichment media (lactose broth [LB] for dry pet food and nonfat dry milk [NFDM] for dark chocolate) and (ii) the ability of the enzyme immunoassay to detect these 68 strains. The goal of these assessments was to identify strains that are likely to lead to reduced detection sensitivity with the BAM method and the enzyme immunoassay evaluated here. Ability of strains to grow in enrichment media was evaluated by inoculating strains onto approximately 1 g of matrix at approximately 103 CFU/g, followed by a primary enrichment in 9 mL of LB (for dry pet food) or NFDM (for dark chocolate) and subculturing into secondary enrichment broths tetrathionate (TT) and Rappaport-Vassiliadis (RV). Pathogens were enumerated after 24 h of primary enrichment and after 24 h of secondary enrichment in TT and RV. The ability of the enzyme immunoassay to detect strains was assessed essentially as previously described (29) by running the enzyme immunoassay with pure cultures, using Salmonella levels at the assay detection limit plus 1 log CFU/mL (ca. 106 CFU/mL).
Based on this initial screen and the relevance of these strains to their respective food matrices and public health, five strains were selected for inoculation onto each of two food matrices: dry pet food and dark chocolate. For the dry pet food, the five selected Salmonella strains were from serotypes Typhimurium, Enteritidis, Saintpaul, Dublin, and Minnesota. Salmonella Minnesota detection sensitivity was not reduced in any of the assays; therefore, this strain was included as a control. Salmonella serotypes Typhimurium, Enteritidis, Typhi, 13,22:b:−, and Mississippi were inoculated onto the chocolate.
Matrix inoculation at fractional positive levels
For each strain, at least three sets of 30 samples were inoculated as described previously (29). Each set represented a different inoculation level based on die-off rates, which were estimated as previously described to target a most probable number (MPN) of ca. 1/25 g (29). Inoculating multiple sets helped to ensure that one set would fall within a target fractional positive range of 50% ± 25% positive samples after 14 days of inoculum stabilization on the food matrix. This approach is consistent with the 14-day inoculum stabilization period detailed by the AOAC International for dry foods (9). To determine which inoculation level (i.e., which set) to test after the 14-day inoculum stabilization period, 10 samples from each group of 30 were used for a prescreen on day 10 after inoculation. The 10 samples were tested by enrichment in LB for pet food or NFDM for dark chocolate for 24 h then transferred to secondary enrichments in TT and RV, incubated for 24 h, and plated onto xylose lysine deoxycholate agar (XLD). For each strain, the inoculation level of the sample set that yielded 25 to 75% positive samples was selected for the full analysis of the remaining 20 samples 14 days after inoculation.
For day 14 samples, a modified AOAC International MPN assay was used for enumeration of Salmonella to determine the number of Salmonella cells per sample. Inoculated chocolate and pet food samples were used to prepare a three- by five-tube MPN series (with 2, 1, and 0.5 g of sample per tube representing the three levels). Tubes were prepared using aliquots from the same sample bags that were tested during the experiment with the standard BAM method. Final MPN values were calculated using the macros-enabled Excel sheet provided in the BAM (4).
BAM methods and modifications for the extended BAM method
The standard BAM method was used as a “gold standard” and was performed according to the Salmonella chapter in the BAM (1). The primary enrichments (PEs) were set up to include 25 g of dry pet food or dark chocolate and 225 mL of LB or NFDM, respectively. The PEs were incubated at 35°C for 24 ± 0.5 h. Then aliquots were used to inoculate the secondary enrichments (SEs) in TT and RV, which were incubated at 35 and 42°C, respectively, for 24 ± 0.5 h. Each SE was then plated onto XLD and Hektoen enteric agar (HE), incubated for 24 ± 1 h, and observed for presence of Salmonella. For each sample, at least one colony was selected for colony PCR assays with primers targeting invA (21) to confirm the presence of Salmonella.
In addition to the standard BAM method, all samples were also tested with an extended BAM protocol that included (i) extended PE times (up to 72 h) and (ii) extended SE times (up to 72 h). This approach was used to characterize the sensitivity of the standard BAM method. In addition to the 24-h PE with subsequent 24-h SE (standard BAM protocol), four other conveniently selected time combinations were tested: (i) 48-h PE with 24-h SE, (ii) 72-h PE with 24-h SE, (iii) 24-h PE with 48-h SE, and (iv) 24-h PE with 72-h SE. For each strain, all combinations originated from the same PE culture, which was returned to the incubator after the appropriate aliquots were removed for testing at each time point (Fig. 1).
Evaluation of a rapid enzyme immunoassay for detection of
Salmonella in dark chocolate
To assess the value of including extended BAM enrichment procedures in an alternative method evaluation, we also evaluated an AOAC International–validated enzyme-linked fluorescent immunoassay for its ability to detect Salmonella in dark chocolate with the same 100 samples (20 samples for each of the five Salmonella strains) that were also tested in the standard and extended BAM methods. The PE was performed in NFDM as specified by the instructions for the enzyme immunoassay, which referred to the BAM for selection of PE procedures. The PEs for the enzyme immunoassay were incubated for 19.5 h followed by subculture into TT and RV (1.0 and 0.1 mL, respectively). After 6 h of incubation, aliquots of the SEs for each given sample were combined into a proprietary tertiary enrichment, which was incubated for 19.5 h and was then used for the enzyme immunoassay (Fig. 2) according to the manufacturer's recommendations. The manufacturer is not specified here because the goal of this work was not to evaluate a specific assay but to assess the value of including extended BAM enrichment methods in an evaluation. This approach was also chosen to ensure that other researchers would not use this study in lieu of true validation of this assay (i.e., with samples representing actual product from the supply chain). Paired culture confirmation for this immunoassay included plating onto XLD and HE of (i) TT and RV (after incubation for 24 h) and (ii) tertiary enrichments (after incubation for 19.5 h) (see Fig. 2).
False-negative probabilities were calculated to facilitate comparisons between the standard BAM, extended BAM, and enzyme immunoassay methods. Although false-negative rates are often used for analyses, we used false-negative probabilities because they represent measurements, not rates. A composite reference standard (CRS) was calculated and represents the number of samples that were positive with at least one of the three methodologies (standard BAM method, extended BAM method, or enzyme immunoassay). A false-negative probability was calculated using the CRS as the gold standard as 1 − sensitivity = 1 − (test positives)/(CRS positives). As detailed in the AOAC International guidelines (9), McNemar's exact test was used to test for significant differences in results for chocolate between the standard BAM, extended BAM, and enzyme immunoassay methods with R statistical software (version 1.0.136, R Foundation for Statistical Computing, Vienna, Austria) and the exact2x2 package.
Standard BAM method for detection of
Salmonella in dry pet food and chocolate produced false-negative probabilities of 7 and 15%, respectively, compared with the extended BAM method
For pet food samples, the standard BAM method produced positive results for 53 of 100 samples (Table 1). An additional four samples tested positive when plated after extended enrichment times, resulting in four false negatives for the standard BAM method (a false-negative probability of 7%). Samples that tested positive with only the extended BAM method were obtained with different extended enrichment times. For example, one sample was positive after 48 h of PE, and another was positive after 24 h of PE plus 48 h of SE (see Table 2 for a complete list). Among the five strains tested on pet food, three strains were responsible for these four false-negative result: serotypes Typhimurium (one false negative), Minnesota (one false negative), and Saintpaul (two false negatives) (Table 2).
For the 100 chocolate samples, 50 were positive with the standard BAM method and 59 were positive samples with the extended BAM method, resulting in a false-negative probability of 15% for the standard BAM method (Table 1). Among the five strains tested on chocolate, only strains of serotypes Typhimurium and Mississippi yielded false-negative results (two and seven false negatives, respectively) (Table 2).
Enzyme immunoassay alternative detection method produced 6 and 20% false-negative probabilities compared with standard and extended BAM methods, respectively
The enzyme immunoassay evaluated yielded Salmonella-positive results for 47 of the 100 dark chocolate samples. The same 47 samples tested positive with the two BAM culture methods (Table 3), yielding a false-negative probability of 0% when culture confirmation was used as a reference. Using the paired standard BAM and extended BAM methods as a reference (50 and 59 positive results, respectively), the resulting false-negative probabilities were 6 and 20%, respectively.
The data obtained for the paired culture confirmation BAM and extended BAM methods allowed us to define a CRS value (23), which represents the number of samples that are positive with at least one of these three approaches (65 of 100 samples). Using the CRS as the gold standard (Table 3), the false-negative probability for the enzyme immunoassay was 28%. Against the same CRS, the standard and extended BAM methods had false-negative probabilities of 23 and 9%, respectively. Although the culture confirmation protocol for the enzyme immunoassay is similar in principle to that of the standard BAM method, the culture confirmation protocol has a shorter (by ∼4.5 h) incubation time for the PE (compared with the standard BAM method), which may be responsible for the detection of additional positive samples with the paired BAM methods. Comparisons of the dark chocolate results obtained with the enzyme immunoassay and the standard BAM culture method revealed some differences between these two methods at the strain level (see Table 3). Statistical analysis revealed that (i) the Salmonella Enteritidis strain was significantly more commonly detected with the standard and extended BAM methods (13 positive results; P = 0.013 for each) than with the enzyme immunoassay (5 positive results), and (ii) the Salmonella Mississippi strain was significantly more commonly detected with the extended BAM method (14 positive results) than with the standard BAM method and the enzyme immunoassay (7 positive results; P = 0.023 for each). These results suggest that Salmonella Mississippi may have reduced growth in the BAM enrichment media (detection was significantly enhanced with prolonged enrichment; see Table 2). Because prescreens in pet food indicated that the Salmonella Enteritidis strain used here may have growth deficiencies in TT and RV (see Table S2), the short enrichment in TT and RV used for the enzyme immunoassay (6 h) may lead to reduced detection with this assay. Salmonella 13,22:b:− was numerically (P = 0.248) more frequently detected with the standard and extended BAM methods (12 positive results each) than with the enzyme immunoassay (9 positive results). Salmonella Typhimurium was numerically more frequently detected with the enzyme immunoassay (13 positive results) than with the standard and extended BAM methods (8 and 10 positive results; P = 0.074 and 0.248, respectively). Similarly, Salmonella Typhi was numerically (P = 0.248) more frequently detected with the enzyme immunoassay (13 positive results) than with the standard and extended BAM methods (10 positive results each).
As the need for rapid and reliable pathogen detection methods continues to increase, the need also increases for rational approaches to selection and validation of rapid methods that are more appropriate than classic culture-based gold standard methods. Validation of rapid (alternative) methods is typically performed by comparing results obtained with a given rapid method with those obtained with a traditional method designated as the gold standard. However, this approach can be challenging because typically, and per definition, alternative methods cannot be easily shown to have improved performance over gold standard methods, which tend to be thought of as not yielding false-negative results. To allow for more accurate relative evaluation of alternative methods, we assessed a gold standard method (the FDA BAM method for Salmonella) under challenging conditions (including strains with reduced growth in enrichment media) using an extended enrichment protocol. We also evaluated an alternative commercial enzyme immunoassay against both standard and extended enrichment BAM methods using dark chocolate as a food matrix. Our results indicate that culture-based gold standard methods can have substantial false-negative probabilities under challenging conditions and that certain weaknesses of alternative methods may be detected only when extended enrichment procedures are included in the evaluation.
Classic culture-based gold standard methods may have substantial false-negative probabilities under challenging conditions
Gold standard detection methods do not have 100% sensitivity and may yield false-negative results, particularly when challenged with certain strains or matrices (23). However, few publications have included quantitative data on false-negative probabilities for the standard culture-based methods for foodborne pathogen detection such as the BAM methods, which are often used as the formal gold standard for evaluation of alternative rapid methods. To provide data on false-negative probabilities of gold standard methods, we designed an evaluation approach to supplement the standard FDA BAM method for detection of Salmonella with extended PE and SE times to increase the sensitivity of this assay. For comparison of the BAM method and the extended BAM method, this approach yielded false-negative probabilities of 7 and 15% for dry pet food and dark chocolate, respectively. The Salmonella Mississippi strain had substantially reduced growth during BAM enrichment of chocolate samples; extended enrichment yielded 7 more positive samples (of 20 tested samples) than did the regular enrichment protocol. Because our evaluations specifically included Salmonella strains with reduced growth rates in PE or SE media, the false-negative probabilities found here are higher than what would be expected during routine use of a method; many of the Salmonella strains in food grow well in enrichment media.
Consistent with our findings, other researchers have acknowledged the value of including a 48-h SE time with Salmonella detection methods to reduce false-negative results that may occur with shorter enrichment times (7, 14, 27). However, disagreement exists regarding the value of an enrichment time beyond 48 h. In one study from 1975, enrichment for 96 h was helpful for identifying additional Salmonella-positive samples of sewage and receiving waters (13). In another study from 1984, incubation times beyond 48 h yielded reduced numbers of Salmonella-positive samples compared with shorter enrichment times (11) for naturally contaminated sludge samples. Therefore, shorter enrichment times may still be necessary because prolonged enrichment times may reduce the likelihood of Salmonella detection when the natural microflora in a sample outcompetes the Salmonella during the enrichment step. However, for samples with low-level contamination, such as those used in the present study, enrichment times beyond 48 h may be more likely to improve detection.
The results of these studies support stringent evaluation of a given enrichment scheme or assay that should include testing of enrichments at multiple time points because some positive samples may be detected at only one enrichment time. The importance of using multiple enrichment times is also supported by results of standard BAM detection methods for other pathogens. The standard BAM method for Listeria monocytogenes detection requires plating on selective media after both 24 and 48 h of enrichment (15).
Overall, our results indicate that the standard BAM method does not have perfect sensitivity and provide estimates of false-negative probabilities that may occur with this BAM method. The false-negative probability assessment framework outlined here can help end users assess whether a given rapid method may be an appropriate replacement for classic culture-based methods, such as the standard BAM method, currently used by a company. For example, a company may decide that a rapid Salmonella pet food testing method with a 4% false-negative probability (such as method B evaluated in a previous study (29)) may not be an appropriate method for routine use. However, the data reported here indicate that this false-negative probability is not substantially different from that of the standard BAM method, suggesting that rapid method B could be used instead of the more time-consuming BAM method. Hence, a CRS as described here and previously (23) can be a useful for combining results from multiple methods (e.g., standard and extended BAM) when formally evaluating an alternative method, including calculation of false- negative probabilities that use the CRS as the gold standard.
The evaluations reported here did not include all scenarios that can lead to false-negative results in culture-based Salmonella detection methods, such as specific phenotypic characteristics of a particular Salmonella strain. For example, H2S-negative or lactose-fermenting Salmonella strains may often be missed with standard culture-based methods because standard plating media (e.g., XLD and HE) utilize H2S production and the inability to ferment lactose as differential characteristics (1, 20, 22).
We also did not test exclusivity because the tested alternative enzyme immunoassay had already received AOAC International Official Methods of Analysis certification. Rapid tests such as immunoassays tend to have more false-positive results. For example, Citrobacter can produce a positive Salmonella result, likely because of the similarity in the O-antigens of these two organisms (3, 16, 19, 26). However, a complete assessment of an alternative method should include both false-positive and false-negative probabilities when a decision is being made on implementation and use of a new model.
Enzyme immunoassays may be effective for detecting
Salmonella in at least some matrices
The initial results indicated that the enzyme immunoassay method evaluated here had near perfect sensitivity when the paired manufacturer-recommended culture confirmation method was used as a reference and had low false-negative probabilities when the standard BAM method was used as a reference. This finding is in contrast to a different surface molecule–targeted assay previously tested on dark chocolate (29), which had false-negative probabilities of ∼15% when the paired manufacturer-recommended culture confirmation method was used as a reference. The previously evaluated test yielded the highest false-negative probabilities for Salmonella 13,22:b:− and Salmonella Typhimurium strains (20 and 44%, respectively), which had been identified in the initial screen as difficult to detect with the evaluated method (29). However, the previously evaluated method and that evaluated here differ in technique and enrichment schemes; the assay evaluated here requiring a multistep enrichment time of at least 42 h, whereas the previously evaluated assay included a single step with an enrichment time of ca. 24 h.
Inclusion of the extended BAM method increased the overall false-negative probability of the enzyme immunoassay (compared with the standard BAM method) and identified possible specific shortcomings of this assay. Some strains (specifically a strain of Salmonella Mississippi) needed longer enrichment times, whereas other strains (most obvious with Salmonella Enteritidis) the BAM methods (with standard and extended) had more sensitive detection than the enzyme immunoassay. This difference may be due to differences in enrichment schemes for the manufacturer-specified culture confirmation test and the paired BAM methods (see Fig. 2), including (i) the longer enrichment time in NFDM for the BAM method (24 versus 19.5 h for the culture confirmation procedure) and (ii) the longer enrichment time in TT and RV for the BAM method (24 versus 6 h for the culture confirmation procedure).
These findings suggest that culture confirmation methods may sometimes require extended enrichment times and further support the challenges associated with reduced enrichment times. Although the enzyme immunoassay method produced a numerically higher proportion of positive samples for some serotypes (e.g., Salmonella Typhimurium), these observations likely reflect a chance event because the only plausible biological explanation for this observation would be presence of a high level of competitive microflora (possibly spores in chocolate (8)) that may outcompete some but not all serotypes, particularly with extended enrichment times. Overall, these results further support the importance of validating alternative and rapid methods with both the strains and the matrices relevant to a given user, with subsequent use of these data to identify the most appropriate assay to be implemented. Final decisions on assay selection typically should be risk based, which would mean false-negative results for rarely encountered strains and serotypes may be considered less problematic than false-negative results for frequently encountered strains and serotypes.
This project was supported by a contract from Mars, Inc. (McLean, VA) to M. Wiedmann. The work of J. Kovac at The Pennsylvania State University was supported by the U.S. Department of Agriculture National Institute of Food and Agriculture, Hatch Appropriations under project PEN04646 and accession 1015787.
Supplemental material associated with this article can be found online at: https://doi.org/10.4315/0362-028X.JFP-19-422.s1