Cross-contamination, improper holding temperatures, and insufficient sanitary practices are known retail practices that may lead to product contamination and growth of Listeria monocytogenes. However, the relative importance of control options to mitigate the risk of invasive listeriosis from ready-to-eat (RTE) products sliced or prepared at retail is not well understood. This study illustrates the utility of a quantitative risk assessment model described in a first article of this series (Pouillot, R., D. Gallagher, J. Tang, K. Hoelzer, J. Kause, and S. B. Dennis, J. Food Prot. 78:134–145, 2015) to evaluate the public health impact associated with changes in retail deli practices and interventions. Twenty-two mitigation scenarios were modeled and evaluated under six different baseline conditions. These scenarios were related to sanitation, worker behavior, use of growth inhibitors, cross-contamination, storage temperature control, and reduction of the level of L. monocytogenes on incoming RTE food products. The mean risk per serving of RTE products obtained under these scenarios was then compared with the risk estimated in the baseline condition. Some risk mitigations had a consistent impact on the predicted listeriosis risk in all baseline conditions (e.g. presence or absence of growth inhibitor), whereas others were greatly dependent on the initial baseline conditions or practices in the deli (e.g. preslicing of products). Overall, the control of the bacterial growth and the control of contamination at its source were major factors of listeriosis risk in these settings. Although control of cross-contamination and continued sanitation were also important, the decrease in the predicted risk was not amenable to a simple solution. Findings from these predictive scenario analyses are intended to encourage improvements to retail food safety practices and mitigation strategies to control L. monocytogenes in RTE foods more effectively and to demonstrate the utility of quantitative risk assessment models to inform risk management decisions.

Sampling surveys of ready-to-eat (RTE) foods suggest that products sliced or prepared at retail delicatessens are more frequently and more heavily contaminated with Listeria monocytogenes and present a substantially higher risk than products sliced and packaged at the manufacturer level (1, 4, 7, 11, 21). L. monocytogenes strains are found to be widely distributed and to persist in retail facilities (13, 23, 26). Common retail practices may lead to product contamination from contaminated environmental sites, cross-contamination from one RTE product to another, or both (10, 16, 22). Additional risk factors include the impact of improper holding temperatures on the growth of L. monocytogenes and insufficient sanitary practices that may contribute to L. monocytogenes contamination of RTE foods at retail (32, 33). The relative importance of these different factors on the potential risk of invasive listeriosis from the consumption of products sliced or prepared at retail is nevertheless not well understood. Therefore, a quantitative risk assessment (QRA) was conducted collaboratively by the U.S. Department of Health and Human Services, U.S. Food and Drug Administration (FDA), Center for Food Safety and Applied Nutrition and the U.S. Department of Agriculture (USDA), Food Safety and Inspection Service (FSIS), in consultation with U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. A primary objective of this QRA was to assess the risk of foodborne listeriosis associated with current practices in retail delis. However, the interest of this QRA relied less in the prediction of the risk of invasive listeriosis than on the extent to which this risk may increase or decrease significantly when some changes in the retail deli practices are applied. Indeed, the evaluation of putative risk management actions is often based on comparison of the baseline risk estimate with a forecast risk that could result from pursuing various alternative strategies (sometimes called “what-if” scenarios) (9). In this second article of the series, we conduct a scenario analysis using the model described previously (18) to examine how the risk may be mitigated by changes in practices that limit L. monocytogenes growth or prevent cross-contamination of RTE foods prepared in retail delis.

Description of model.

The discrete-event model developed for this study (i.e. the “virtual deli” model) has been described extensively in the first article of this series (18). In brief, the model mimics an operating retail deli department and tracks the L. monocytogenes that may potentially be present on various environmental sites (e.g. slicer, utensils) and on selected food products (i.e. deli meat, deli cheese, or deli salad). The model simulates worker behavior, the contamination and cross-contamination of RTE foods, and the growth or decline of L. monocytogenes in the deli-prepared RTE foods, and then it predicts the subsequent risk of invasive listeriosis among consumers.

Description of baseline retail conditions.

Two sources of L. monocytogenes contamination were considered in the model: (i) contamination on incoming RTE foods entering the deli (e.g. chubs of meat, blocks of cheese, or bulk salad) and (ii) contamination on retail environmental surfaces or niches. The approach used was to evaluate and compare the public health impact of various food safety mitigations under six different baselines that may characterize different operational conditions in retail delis. In brief, these six baseline conditions were as follows: (i) a retail deli with multiple niches or regular environmental contamination of nonfood contact surfaces (NFCS) (baseline denoted as “multiple niche 100W”); (ii) a retail deli with no niches or environmental L. monocytogenes transfer (“no niche”); (iii) a retail deli with no niche or environmental L. monocytogenes transfer, with one incoming RTE product that supports growth and is contaminated at L. monocytogenes levels higher than those observed in USDA-FSIS monitoring (30) (“highly contaminated incoming chub growth”); (iv) a scenario similar to the scenario iii where the product does not support growth (“highly contaminated incoming chub – no growth”); (v) a retail deli with multiple niches, compliant with temperature control (“multiple niche and temperature control”); and (vi) a retail deli without niches, compliant with temperature control (“no niche and temperature control”). A more complete description of these retail baseline conditions is provided in Table 1 and in the companion article (18). Within each of the six retail baseline conditions, we used the model to evaluate the public health impact of a variety of what-if scenarios related to changes in sanitary practices, worker behaviors, product formulations, cross-contamination, and impact of product storage temperature and duration.

TABLE 1.

Risk assessment model retail baseline conditions (18)

Risk assessment model retail baseline conditions (18)
Risk assessment model retail baseline conditions (18)

Description of tested food safety mitigation options.

Fourteen specific risk management questions were developed (Table 2). Some of the risk management questions were generated by the Interagency Retail L. monocytogenes Risk Assessment Workgroup, whereas other questions were the outcome of discussions with stakeholders. These risk management questions were then translated into a list of 22 what-if scenarios to be evaluated within the risk assessment model. Some of the what-if scenarios directly addressed a specific risk management question. For example, the scenario “Retail deli workers clean deli NFCS as if they were food contact surfaces (FCS), i.e., every 4 h in accordance with the 2013 FDA Food Code” directly addresses the question “What if scale touch pads, refrigerator and deli case handles, and other frequently touched NFCS were considered FCS and were therefore cleaned and sanitized at a minimum frequency as per FDA Food Code (34) requirements?” In other instances, additional scenario analysis was conducted to evaluate the public health impact of specific risk management questions more effectively. For example, the what-if scenario “Retail deli workers change gloves for every sale of RTE products” was evaluated to address the question “What is the potential public health impact of increasing the use of single service gloves in the retail delis?” An additional scenario, “Retail deli workers do not use gloves when serving customers,” also was evaluated to better characterize the impact of this worker behavior practice. Finally, some what-if scenarios were evaluated to characterize and describe the model behavior. For example, a scenario without any cross-contamination in the deli was tested to explore the impact of cross-contamination in this setting, even though no risk management option could achieve this goal consistently. Similarly, a scenario without any sanitation was tested to evaluate the overall impact of sanitation, which does not correspond to any risk management option. Tables 3 through 8 provide a description of each of the tested scenarios as well as how they were translated in the model.

TABLE 2.

Expanded risk management questions and corresponding risk assessment scenarios

Expanded risk management questions and corresponding risk assessment scenarios
Expanded risk management questions and corresponding risk assessment scenarios
TABLE 2.

Continued

Continued
Continued
TABLE 3.

Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for sanitation-related scenarios

Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for sanitation-related scenarios
Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for sanitation-related scenarios
TABLE 4.

Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for worker behavior–related scenarios

Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for worker behavior–related scenarios
Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for worker behavior–related scenarios
TABLE 5.

Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for growth inhibitor-related scenarios

Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for growth inhibitor-related scenarios
Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for growth inhibitor-related scenarios
TABLE 6.

Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for cross-contamination–related scenarios

Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for cross-contamination–related scenarios
Percentage of change in the mean risk of listeriosis in the susceptible population relative to each baseline for cross-contamination–related scenarios
TABLE 7.

Percentage of change in the mean risk of listeriosis for the susceptible population relative to each baseline for control of storage temperature and duration scenarios

Percentage of change in the mean risk of listeriosis for the susceptible population relative to each baseline for control of storage temperature and duration scenarios
Percentage of change in the mean risk of listeriosis for the susceptible population relative to each baseline for control of storage temperature and duration scenarios
TABLE 8.

Percentage of change in the mean risk of listeriosis relative to each baseline for reduced incoming product concentrations

Percentage of change in the mean risk of listeriosis relative to each baseline for reduced incoming product concentrations
Percentage of change in the mean risk of listeriosis relative to each baseline for reduced incoming product concentrations

Evaluation of food safety mitigation option impact.

Each what-if scenario was evaluated for each of the six retail baseline conditions. Some additional scenarios were explored in more detail to understand the impact of an increased L. monocytogenes contamination in incoming products and the impact of home storage on the predicted risk.

Each what-if scenario was simulated by changing one or a few parameters from the baseline model. As an example, the “no-glove” scenario could be evaluated from the baseline condition by setting the probability for a food worker to use gloves to 0. A brief description of the changes made from the baseline models is provided (in parentheses) in Tables 3 through 8. For each retail baseline condition, the mean risk of listeriosis per serving was evaluated in 30 simulations, with each simulation modeling 100 stores that each provided 1,000,000 servings (total of 108 servings per simulation). Each of the 30 simulations was performed using a different random seed. Each what-if scenario was evaluated with one simulation of 100 stores, with each providing 1,000,000 servings for every baseline condition. The percentage of change in the mean risk per serving for the susceptible population in a given scenario relative to the median from the 30 estimations of this mean risk in the corresponding baseline condition was evaluated and used as an estimate of the impact of the given scenario in the current baseline. If the mean risk per serving for the susceptible population in the what-if scenario was lower than the 2.5th percentile or greater than the 97.5th percentile of the 30 estimations of this mean risk for the baseline, the what-if scenario was deemed significantly different from the baseline. This procedure allowed us to identify those changes that are significant and not due to random uncertainty in the model.

The risk assessment model was specifically designed as a virtual deli to evaluate the impact of changes in retail practices and potential mitigation options. As has been discussed previously (9), such evaluations are best studied through what-if scenarios that simulate the impact of specific changes in handling practices and interventions on the predicted change in public health risk and that allow for comparisons between baseline risk estimates and the forecasted risk that could result from the implementation of various alternative mitigation strategies. Notably, some of the evaluated scenarios are hypothetical in the sense that they do not represent actual risk management options, but are instead pursued to understand the risk assessment model and the intricate impacts of selected changes in the deli system. These scenarios should be considered as sensitivity analyses rather than evaluations of the actual risk management option (9).

In Table 3, each column represents one of the six different baseline conditions that were tested, and each row shows the impact of a selected mitigation option. Tables 4 through 8 are organized similarly. The impact of each scenario was evaluated for the susceptible population only. Although the predicted absolute risks differed between the susceptible populations and the general population, the predicted relative change in risk was similar for both populations. This observation is linked to the linearity of the Food and Agriculture Organization of the United Nations, World Health Organization dose–response model for L. monocytogenes (8). We report here only the results for the susceptible population. The impact of selected mitigations (in each row) is shown as the percentage of change in the risk per serving relative to the respective baseline estimate (hereafter relative risk). The scenarios are organized by topic in the tables as follows: sanitation (Table 3), worker behavior (Table 4), use of growth inhibitors (Table 5), cross-contamination (Table 6), storage temperature control (Table 7), and reduction in the level of L. monocytogenes on incoming RTE products (Table 8). In these tables, positive values represent an increase in the predicted absolute risk for the susceptible population, compared with the corresponding baseline. Negative values represent a decrease compared with the corresponding baseline in the predicted relative risk per serving. The relative effectiveness of the scenarios applied to a specific baseline condition can be assessed by reading down the columns of Tables 3 through 8.

The relative impact on the predicted risk of listeriosis from a change in retail practice across different operating conditions in different types of retail delis can be assessed by comparing estimates across different cells in each row in Tables 3 through 8. For correct interpretation of the tables, it is important to consider the order of magnitude of the mean risk per serving for the susceptible population predicted for the applicable baseline conditions, when no specific scenario is considered (i.e. 1.7 × 10−7 for the multiple niche 100W baseline, 1.4 × 10−7 for the no niche baseline, 16.6 × 10−7 for the highly contaminated incoming chub – growth baseline, 2.8 × 10−7 for the highly contaminated incoming chub – no growth baseline, 1.5 × 10−7 for the multiple niche and temperature control baseline, and 1.2 × 10−7 for the no niche and temperature control baseline). Importantly, analysis of the baseline results indicates that the risk of listeriosis was strongly correlated with the contamination of RTE products that support L. monocytogenes growth, regardless of the origin of the contamination (e.g. contaminated as an incoming product, contaminated through cross-contamination from the retail environment, contaminated via other RTE products) or whether these food vehicles support L. monocytogenes growth (18). This finding is relevant in the interpretation of mitigation options.

Sanitation-related scenarios.

The results shown in Table 3 indicate that retail sanitation practices are key drivers in reducing the predicted risk of listeriosis from consumption of RTE foods prepared or handled in delis. When sanitation activities were modeled as absent (“no sanitation”), the predicted risk increased by 50.2% (i.e. multiple niche and temperature control column). A study of the pathways by which L. monocytogenes was introduced into the deli shows that the elimination of bacteria through sanitation practices was a significant factor. Indeed, the smallest predicted increase in risk from omitting sanitation was 2.9% (highly contaminated incoming chub – growth column). In this scenario, the impact of sanitation was outweighed by the additional bacteria stemming from highly contaminated incoming product and growth of L. monocytogenes in the product while in the retail deli. However, on the basis of the results provided here, it seems that once a reasonable level of sanitation has been reached, factors such as modifying any of the single sanitation-related practices individually, such as cleaning more effectively (i.e. “wash and sanitize” scenario, significant in only one baseline condition), increasing the number of sporadic cleaning sites (i.e. “clean 8 sporadic” scenario, significant in only one baseline condition), or not conducting any sporadic cleaning (“no sporadic cleaning” scenario, not significant), has limited impact by itself on the predicted relative risk in each retail deli condition studied (Table 3).

It was observed that NFCS, such as scale touch pads and refrigerator and deli case handles, usually are not cleaned and sanitized as frequently as FCS (15). Treating NFCS as FCS for cleaning and sanitizing purposes had little impact on the predicted risk (i.e. “NFCS as FCS” scenario). The only retail condition where the risk reduction was statistically significant for this scenario was for a retail deli without any niches that implemented temperature control. In this situation where L. monocytogenes levels are low and growth is limited, the additional cleaning was predicted to be beneficial in controlling this already low baseline risk of 1.2 × 10−7.

Worker behavior–related scenarios.

The impact of simulated changes to worker behavior on the predicted risk varied, depending on the baseline retail deli condition, as shown in Table 4. For example, using separate slicers in a retail deli that had multiple niches (i.e. multiple niche 100W column and “separate slicer” scenario) reduced the predicted relative risk by 6.3%. If workers did not use gloves that could be routinely discarded (i.e. no-glove scenario), the predicted relative risk increased from 5.1 to 8.5%. In other cases, the public health benefits of some interventions seem to have been cancelled out by other factors.

The lack of glove use (i.e. no-glove scenario) consistently increased the predicted risk across all retail deli types, often at statistically significant levels (+1.2 to +7.0%). Therefore, retail glove use probably contributes to L. monocytogenes control and is even more critical for other foodborne pathogens, such as norovirus or Shigella that are associated with transmission by food workers via the fecal-oral route (27). Changing gloves after every serving (i.e. “gloves every serving” scenario) did not result in a significant change in the predicted risk, but gloves are currently already changed frequently, for ~65% of servings in baselines (15, 18).

Preslicing generally increased the predicted risk of listeriosis, often substantially, and especially if a contaminated growth-supporting RTE product was being sliced (i.e. “preslice” scenario). This trend was due to contamination of the slicer with a contaminated chub during the preslicing process. A high number of RTE product servings are then cross-contaminated, leading to a higher predicted risk. A retail deli with an incoming contaminated chub that does not support the growth of L. monocytogenes was the only situation in which preslicing led to a significantly lower predicted risk compared with the baseline. In this situation, preslicing leads to a distribution of the bacteria to the same category of (nongrowth) RTE products, rather than to various (growth and nongrowth) RTE product that is sliced throughout the day and limits potential cross-contamination of the RTE product that supports the growth, a key driver of the predicted risk.

The risk assessment model generally predicted that having separate slicers for products that support L. monocytogenes growth and those that do not had minimal impact in mitigating the risk of listeriosis (i.e. separate slicer scenario). The only exception was observed in the baseline where one incoming product that does not support growth of L. monocytogenes was contaminated at levels higher than those observed in USDA-FSIS monitoring (30). Additional studies of the outputs of the model suggest that, in the tested situation including two slicers in the deli, the slicer for incoming nongrowth product was eventually used for more products than in the baseline scenarios where slicers were not separated; as a consequence, more RTE product was incidentally cross-contaminated with L. monocytogenes. The resulting predicted risk was therefore higher than when slicers were not separated. However, this counterintuitive result would not necessarily be generalizable to larger stores with a large number of slicers. Importantly, such model behavior points out the complexity of the interaction among the various elements in the model and emphasizes the need for in-depth analysis of model outputs and extensive sensitivity analysis.

Preventing contact between the case handle and hands or gloves resulted in no significant change in the predicted risk in the baselines studied (i.e. “no-contact glove case” scenario). Similarly, slicing RTE product onto deli paper rather than onto gloves (i.e. “do not slice on gloves” scenario) had little impact on the relative risk predicted by the model, with the exception of the multiple niche and temperature control baseline condition, where the risk was significantly increased. Slicing onto gloves was a behavior observed during the observational study (15). Usually, no reduction in the predicted risk was associated with eliminating this practice, although risk was reduced with some of the other mitigation options that are directed at preventing cross-contamination. Together, these results suggest that, within a given baseline scenario, no single cross-contamination event had a major impact on the predicted risk with the exception of the cross-contamination within the slicer (see “Cross-contamination–related scenarios”). These results suggest that a combination of multiple cross-contamination events results in an increased risk from RTE food prepared at retail. It may be difficult, but necessary, to prevent L. monocytogenes cross-contamination through a combination of sanitary controls (see “Cross-contamination–related scenarios”).

Growth inhibitor–related scenarios.

Of all the mitigation options evaluated using this QRA, the formulation of RTE products with growth inhibitors had the greatest impact on the predicted relative risk (Table 5). The use of growth inhibitors in all products almost eliminated the predicted risk of listeriosis, reflected by reductions in relative risk that ranged from −94.4 to −97.5%. Although it is probably not feasible to formulate all RTE products with growth inhibitors because of adverse impacts on the quality of the products (e.g. taste), this level of predicted relative risk reduction (i.e. ~95%) is a significant finding and emphasized that this mitigation option may nearly eliminate the risk. Growth inhibitors continue to prevent growth even after the RTE product leaves the retail deli and therefore directly limit bacterial growth during home storage, a key driver of the listeriosis risk (6, 8, 19, 28). The baseline scenarios consider that products in the retail deli are a mixture of products that include growth inhibitors and products that do not. In a comparison scenario in which no products in the retail deli contained growth inhibitors (“no GI”), the predicted risk nearly doubled to 184.1 and 191.5% compared with the baselines. The only apparent exception is in the highly contaminated incoming chub – growth baseline, where the estimated relative increase in predicted risk was only 35.1%. This relatively low value is somewhat misleading, because the predicted absolute risk for this baseline was already almost 10 times higher than for other baseline results. These results confirm the overwhelming importance of limiting the contamination of RTE products that support the growth of L. monocytogenes to mitigate the risk of listeriosis, as has been consistently observed in other studies (8, 28, 29).

Cross-contamination–related scenarios.

Table 6 results show that controlling cross-contamination in retail delis has a significant impact on the predicted risk of listeriosis. In the what-if scenarios in which cross-contamination did not occur in the retail deli (i.e. “transfers and slicer to 0” row), the predicted relative risk reduction was significant for all baseline conditions (i.e. from −9.5 to −60.8%). However, when L. monocytogenes transfers from all sites were eliminated, but transfers from the slicer were not eliminated (transfers to 0 scenario), the predicted relative risk was not reduced. This result further highlights the importance of the slicer in retail delis as a primary source of cross-contamination at retail. The importance of the slicer as a point of cross-contamination has been demonstrated experimentally (3, 12, 16, 17, 24, 25, 35). Our results predict a significant risk reduction if all cross-contamination events are eliminated in a deli department setting—up to 60% decrease in the relative risk. This suggests that management practices that prevent cross-contamination in retail delis would substantially mitigate the risk of listeriosis from retail-prepared RTE foods.

Control of storage temperature control and duration.

Improving temperature controls in display cases to limit growth resulted in a lower predicted relative risk across all tested retail deli conditions (Table 7). Note that the “temperature = 5°C” (41°F) alternative represents an increase in the temperature for some of the RTE products (i.e. those stored at a temperature of <5°C) compared with the baseline. By setting the temperature of display cases to 5°C, the overall predicted risk was still reduced. The hypothetical “no growth” scenario results in up to a 20% reduction in predicted risk.

For the scenario in which the RTE foods were held at the recommended temperature (i.e. “temperature ≤ 5°C”), the predicted reduction in risk was roughly the same as the reduction associated with holding RTE foods at temperatures that completely prevents growth of L. monocytogenes (i.e. no growth). The model studied the effect of full compliance with the 2013 FDA Food Code (34) temperature requirements on the predicted risk, compared with typical temperature conditions observed in a study by EcoSure (5). The model predicts that maintaining products in the deli display at a temperature recommended by the FDA Food Code (34) can prevent almost all of the additional risk linked to bacterial growth at retail. The effect was greatest in a retail deli without any niches. The model predictions suggest that a reduction of the risk of listeriosis could be achieved by better compliance with recommended RTE product storage temperatures in the deli environment. The FDA's 2008 Retail Risk Factor Study revealed that the failure in delis to control RTE product holding temperatures and times was the risk factor with the highest “out of compliance” percentage (32). For example, in 60% of the 98 retail delis studied by the FDA in 2008, at least one observation was made in which food requiring temperature control was not held at 5°C or below, as specified in the FDA Food Code (32). This finding further demonstrates the importance of controlling the growth of L. monocytogenes in RTE products, as suggested in previous risk assessments (8, 28, 29).

Shortening the maximum time a refrigerated RTE product that supports growth of L. monocytogenes is allowed to remain on hand in the retail deli after opening or preparation from 7 to 4 days was predicted to have little effect on the risk. Under the current model, the time from when the chub is opened until it is completely sold is generally shorter than 7 days, as required by regulations modeled after the FDA Food Code. Importantly, the model does not currently simulate refrigerated storage of product during transport or in reach-in and walk-in refrigerators before the chub or deli salad bulk is opened.

Concentrations of L. monocytogenes in incoming product.

Reducing the concentration of L. monocytogenes on incoming RTE products significantly reduced the resulting risk of listeriosis from retail prepared foods (i.e., “reduce level” scenario, Table 8). A decrease in L. monocytogenes concentration on incoming RTE products by a factor of 2 (from a lognormal distribution with a mean of −9.5 log CFU/g to a lognormal distribution with a mean of −9.2 log CFU/g, with the standard deviation being maintained at 2.9 log CFU/g) was predicted to reduce the risk of listeriosis from retail prepared RTE foods by −10 to −24%. In this scenario, incoming RTE products contaminated with L. monocytogenes represent one of the major sources of contamination of retail FCS and subsequent cross-contamination of other RTE foods. Even in retail deli settings in which L. monocytogenes is already present in niches, a reduction in the levels of L. monocytogenes on incoming RTE products was predicted to have a substantive impact in reducing the risk from deli-prepared RTE foods. The model predicts that controlling even low levels of L. monocytogenes on incoming RTE products is important to mitigating the risk of listeriosis from deli-prepared RTE foods. What-if scenarios were also conducted to explore the public health impact of higher levels of L. monocytogenes on incoming RTE products. In response to public comments on the QRA, additional scenarios were conducted to evaluate the impact of high contamination rates (incoming RTE products that support growth or that do not support growth with 100 CFU/g of L. monocytogenes) on the predicted risk (14, 31). The concentration of L. monocytogenes on incoming RTE products was set from a lognormal distribution with mean −9.2 log CFU/g of L. monocytogenes (standard deviation of 2.9 log CFU/g of L. monocytogenes [baseline]) to exactly 100 CFU/g of L. monocytogenes (Table 9). Increased contamination of RTE products that support growth increases the average predicted risk from 1.4 × 10−7 to 300 × 10−7 per serving. This increase in the risk is primarily the result of high levels of L. monocytogenes on incoming RTE product, because the predicted risk is unchanged if there is no retail cross-contamination (300 × 10−7, Table 9). In contrast, when the incoming RTE products do not support L. monocytogenes growth, retail cross-contamination contributes to a substantial increase in the risk of listeriosis, from 1.2 × 10−7 to 66 × 10−7. Without retail cross-contamination, increasing the level of L. monocytogenes in RTE products that do not support L. monocytogenes growth does not contribute much to the predicted risk of listeriosis (1.2 × 10−7) compared with the baseline (1.1 × 10−7). Note that these additional scenarios assumed that the proportion of incoming RTE products with ≥100 CFU/g would shift from 0.01 to 100%. This scenario might be considered somewhat extreme in that it is unlikely that all incoming product would be contaminated at 100 CFU/g. Overall, these scenarios demonstrate that controlling the level of L. monocytogenes on incoming RTE products, both those that support L. monocytogenes growth and those that do not, is critical to mitigating the risk of listeriosis. The results also show that even RTE products that do not support L. monocytogenes growth could substantially contribute to the risk of listeriosis as a result of cross-contamination in the retail deli.

TABLE 9.

Predicted absolute risk of invasive listeriosis for the susceptible population per serving of RTE food sliced or prepared and sold at retail delis according to various scenarios under the no niche baseline

Predicted absolute risk of invasive listeriosis for the susceptible population per serving of RTE food sliced or prepared and sold at retail delis according to various scenarios under the no niche baseline
Predicted absolute risk of invasive listeriosis for the susceptible population per serving of RTE food sliced or prepared and sold at retail delis according to various scenarios under the no niche baseline

Consumer storage practices.

An additional evaluation of the impact of consumer storage practices on the predicted risk was conducted. In a first scenario, products were kept at home during a time similar to that in the baseline, but all home refrigerators were consistently held at a temperature below 5°C. In a second scenario, the home refrigerator temperatures were set back to the temperature used in the baseline (5, 20), but all meat products and salads were consumed within 3 days after purchase and all cheese products were consumed within 4 days after purchase. All other parameters were set as in the baseline conditions. These scenarios were run for two baseline retail conditions: multiple niche 100W and no niche. The predicted risk of listeriosis was almost eliminated if the products were stored at home less than 3 to 4 days or if home-refrigerator temperatures were lower than 5°C (Table 10). Importantly, these scenarios assumed a complete implementation of the recommendations by all consumers (i.e. home-refrigerator temperature in the first scenario and use-by days in the second scenario), which is unlikely to be met. The reductions in the risk were therefore likely to be overestimated, compared with what would be expected based on imperfect implementation of the recommendations in practice. Nonetheless, these scenarios confirm the dramatic impact of home-storage duration and temperature on the risk of listeriosis. Similar findings have been reported in other national and international risk assessments on listeriosis (8, 29), namely, that the risk of listeriosis was driven by storage for a longer period and at higher-than-recommended temperatures for contaminated RTE products that supported growth. In addition, this present risk assessment shows that the risk can be reduced by avoiding contamination and cross-contamination of these products in advance.

TABLE 10.

Predicted absolute risk of invasive listeriosis for the susceptible population per serving of RTE food sliced or prepared and sold at retail delis according to the baseline conditions and the scenario

Predicted absolute risk of invasive listeriosis for the susceptible population per serving of RTE food sliced or prepared and sold at retail delis according to the baseline conditions and the scenario
Predicted absolute risk of invasive listeriosis for the susceptible population per serving of RTE food sliced or prepared and sold at retail delis according to the baseline conditions and the scenario

In conclusion, some of the essential duties of risk managers are to determine which hazards or practices present more than an acceptable level of risk to society and to consider what control options may be available (2). The ultimately chosen mitigation options need to be effective and efficient in mitigating risk and must be feasible to implement. In a retail setting, numerous parameters impact the potential risk of contamination and growth of L. monocytogenes in RTE products. Experimental studies in delis or mock delis provide useful specific information for QRA model calibration and validation. When integrated into QRAs, such as this virtual deli model, we can evaluate the public health impact of current practices and predict the public health impact of changes in practices. The development of this QRA, mimicking existing delis operations and practices, represents a complement to experimental studies and allows for the testing and comparison of the impact of various and interrelated risk factors.

This QRA represents the first large-scale effort to model L. monocytogenes cross-contamination at retail in its entirety. The use of the QRA, through the conduct of what-if scenario analyses, provides the first in-depth insight into the extent to which changes in practices and mitigation options impact the risk of listeriosis from deli-prepared RTE foods. The risk assessment model contributes to our understanding and to the comparison of the relative impact of L. monocytogenes transmission, survival, and growth in the retail environment and was used to evaluate how retail practices may impact the predicted risk of listeriosis. The approach used evaluated the public health effect of various changes in practices under six different baseline conditions that may characterize a retail deli and the RTE food it serves. Although some risk mitigations had a consistent impact on the predicted listeriosis risk in all baseline conditions (e.g. presence or absence of growth inhibitor), others were greatly dependent on the initial baseline conditions or practices in the deli (e.g. preslicing of products, sanitation practices). Overall, the key findings from this scenario analysis are as follows:

  • Control growth: Using practices that prevent bacterial growth reduced the predicted risk of listeriosis, as observed in other L. monocytogenes risk assessments. The use of growth inhibitors for suitable products prevents growth of L. monocytogenes in RTE foods both at retail and during consumer home storage. In this scenario analysis, use of growth inhibitors led to a large reduction in the predicted risk of listeriosis (~95%) for all baseline conditions. The strict control of temperature during refrigerated storage in retail delis also reduced the predicted risk. The impact of temperature is nevertheless lower, as it reduces growth only during this specific storage time (5 to 20% reduction according to the baseline and the scenario). Temperature control and storage time at the consumers' homes led to a far greater risk reduction (approximately −99%).

  • Control cross-contamination: Eliminating cross-contamination of L. monocytogenes in the retail environment significantly reduced the predicted risk of listeriosis. Although cross-contamination during the routine operation of the retail deli is not amenable to a simple solution, effective cleaning and sanitizing practices can reduce the likelihood that contamination in the retail environment will be a source for cross-contamination.

  • Control contamination at its source: Decreasing the levels and transfers of L. monocytogenes from incoming products, the environment, or niches directly decreases the predicted risk of illness in all baseline conditions. Decreasing L. monocytogenes concentration in incoming product decreased the predicted risk of listeriosis whether or not the contaminated RTE product itself supported growth. The decrease in predicted risk was greater when the equivalent contamination occurred on product that supported the growth of L. monocytogenes.

  • Identify key routes of contamination: The slicer is a primary source of L. monocytogenes cross-contamination to deli meats and cheeses. Control of L. monocytogenes cross-contamination at this point during retail preparation reduced the predicted risk of listeriosis in all baseline conditions.

  • Continue sanitation: Sanitation practices that eliminate L. monocytogenes from deli FCS result in a reduction in the predicted risk of illness. Cleaning and sanitizing FCS reduced the predicted L. monocytogenes levels in the deli area. Wearing gloves while serving customers reduced the estimated risk of listeriosis.

In summary, these scenario analyses and more generally this QRA, improve our understanding of the extent and prioritize the various factors that contribute to L. monocytogenes contamination in the retail deli. This work is intended to encourage improvements to retail food safety practices and mitigation strategies to reduce L. monocytogenes contamination of RTE foods.

Support for this work was provided by Virginia Tech in collaboration with the USDA-FSIS (contract AG-3A94-P-08-0166). This work was also supported by appointments to the Research Participation Program at the Center for Food Safety and Applied Nutrition administered by the Oak Ridge Institute for Science and Education through an interagency agreement between the U.S. Department of Energy and the USDA.

1.
Ahmed
,
O. M.
,
P.
Pangloli
,
C. A.
Hwang
,
S.
Zivanovic
,
T.
Wu
,
D.
D'Souza
, and
F. A.
Draughon
.
2015
.
The occurrence of Listeria monocytogenes in retail ready-to-eat meat and poultry products related to the levels of acetate and lactate in the products
.
Food Control
52
:
43
48
.
2.
American Chemical Society
.
1998
.
Understanding risk analysis: a short guide for health, safety, and environmental policy making
.
American Chemical Society
,
Washington, DC
.
3.
Chaitiemwong
,
N.
,
W. C.
Hazeleger
,
R. R.
Beumer
, and
M. H.
Zwietering
.
2014
.
Quantification of transfer of Listeria monocytogenes between cooked ham and slicing machine surfaces
.
Food Control
44
:
177
184
.
4.
Draughon
,
A. F.
2006
.
A collaborative analysis/risk assessment of Listeria monocytogenes in ready-to-eat processed meat and poultry collected in four FoodNet states
.
Presented at the International Association for Food Protection 93rd Annual Meeting, Calgary, Alberta, Canada, 13 to 16 August
.
5.
EcoSure
.
2008
.
EcoSure 2007 cold temperature database
.
Available at: http://foodrisk.org/exclusives/EcoSure/. Accessed 1 April 2016
.
6.
Ellouze
,
M.
,
J. P.
Gauchi
, and
J. C.
Augustin
.
2010
.
Global sensitivity analysis applied to a contamination assessment model of Listeria monocytogenes in cold smoked salmon at consumption
.
Risk Anal
.
30
:
841
852
.
7.
Endrikat
,
S.
,
D.
Gallagher
,
R.
Pouillot
,
H.
Hicks Quesenberry
,
D.
Labarre
,
C. M.
Schroeder
, and
J.
Kause
.
2010
.
A comparative risk assessment for Listeria monocytogenes in prepackaged versus retail-sliced deli meat
.
J. Food Prot
.
73
:
612
619
.
8.
Food and Agriculture Organization of the United Nations, World Health Organization
.
2004
.
Risk assessment of Listeria monocytogenes in ready to eat foods. Technical report
.
Food and Agriculture Organization of the United Nations and World Health Organization
,
Rome
.
Available at: http://www.fao.org/docrep/010/y5394e/y5394e00.htm. Accessed 1 April 2016
.
9.
Food and Agriculture Organization of the United Nations, World Health Organization
.
2009
.
Risk characterization of microbiological hazards in food
.
Food and Agriculture Organization of the United Nations and World Health Organization
,
Rome
.
Available at: ftp://ftp.fao.org/ag/agn/jemra/MRA17_05.10.09_f.pdf. Accessed 1 April 2016
.
10.
Gibson
,
K. E.
,
O. K.
Koo
,
C. A.
O'Bryan
,
J. A.
Neal
,
S. C.
Ricke
, and
P. G.
Crandall
.
2013
.
Observation and relative quantification of cross-contamination within a mock retail delicatessen environment
.
Food Control
31
:
116
124
.
11.
Gombas
,
D. E.
,
Y.
Chen
,
R. S.
Clavero
, and
V. N.
Scott
.
2003
.
Survey of Listeria monocytogenes in ready-to-eat foods
.
J. Food Prot
.
66
:
559
569
.
12.
Hoelzer
,
K.
,
R.
Pouillot
,
D.
Gallagher
,
M. B.
Silverman
,
J.
Kause
, and
S.
Dennis
.
2012
.
Estimation of Listeria monocytogenes transfer coefficients and efficacy of bacterial removal through cleaning and sanitation
.
Int. J. Food Microbiol
.
157
:
267
277
.
13.
Hoelzer
,
K.
,
B. D.
Sauders
,
M. D.
Sanchez
,
P. T.
Olsen
,
M. M.
Pickett
,
K. J.
Mangione
,
D. H.
Rice
,
J.
Corby
,
S.
Stich
,
E. D.
Fortes
,
S. E.
Roof
,
Y. T.
Grohn
,
M.
Wiedmann
, and
H. F.
Oliver
.
2011
.
Prevalence, distribution, and diversity of Listeria monocytogenes in retail environments, focusing on small establishments and establishments with a history of failed inspections
.
J. Food Prot
.
74
:
1083
1095
.
14.
The Interagency Retail Listeria monocytogenes Risk Assessment Workgroup
.
2013
.
Interagency risk assessment: Listeria monocytogenes in retail delicatessens – response to public comments
.
U.S. Department of Agriculture, Food Safety and Inspection Service
. .
15.
Lubran
,
M. B.
,
R.
Pouillot
,
S.
Bohm
,
E. M.
Calvey
,
J.
Meng
, and
S.
Dennis
.
2010
.
Observational study of food safety practices in retail deli departments
.
J Food Prot
.
73
:
1849
1857
.
16.
Maitland
,
J.
,
R.
Boyer
,
D.
Gallagher
,
S.
Duncan
,
N.
Bauer
,
J.
Kause
, and
J.
Eifert
.
2013
.
Tracking cross-contamination transfer dynamics at a mock retail deli market using GloGerm
.
J. Food Prot
.
76
:
272
282
.
17.
Perez-Rodriguez
,
F.
,
R.
Castro
,
G. D.
Posada-Izquierdo
,
A.
Valero
,
E.
Carrasco
,
R. M.
Garcia-Gimeno
, and
G.
Zurera
.
2010
.
Evaluation of hygiene practices and microbiological quality of cooked meat products during slicing and handling at retail
.
Meat Sci
.
86
:
479
485
.
18.
Pouillot
,
R.
,
D.
Gallagher
,
J.
Tang
,
K.
Hoelzer
,
J.
Kause
, and
S. B.
Dennis
.
2015
.
Listeria monocytogenes in retail delicatessens: an interagency risk assessment-model and baseline results
.
J. Food Prot
.
78
:
134
145
.
19.
Pouillot
,
R.
,
V.
Goulet
,
M. L.
Delignette-Muller
,
A.
Mahe
, and
M.
Cornu
.
2009
.
Quantitative risk assessment of Listeria monocytogenes in French cold-smoked salmon: II. Risk characterization
.
Risk Anal
.
29
:
806
819
.
20.
Pouillot
,
R.
,
M. B.
Lubran
,
S. C.
Cates
, and
S.
Dennis
.
2010
.
Estimating parametric distributions of storage time and temperature of ready-to-eat foods for U.S. households
.
J. Food Prot
.
73
:
312
321
.
21.
Pradhan
,
A. K.
,
R.
Ivanek
,
Y. T.
Grohn
,
R.
Bukowski
,
I.
Geornaras
,
J. N.
Sofos
, and
M.
Wiedmann
.
2010
.
Quantitative risk assessment of listeriosis-associated deaths due to Listeria monocytogenes contamination of deli meats originating from manufacture and retail
.
J. Food Prot
.
73
:
620
630
.
22.
Pradhan
,
A. K.
,
R.
Ivanek
,
Y. T.
Grohn
,
R.
Bukowski
, and
M.
Wiedmann
.
2011
.
Comparison of public health impact of Listeria monocytogenes product-to-product and environment-to-product contamination of deli meats at retail
.
J Food Prot
.
74
:
1860
1868
.
23.
Sauders
,
B. D.
,
M. D.
Sanchez
,
D. H.
Rice
,
J.
Corby
,
S.
Stich
,
E. D.
Fortes
,
S. E.
Roof
, and
M.
Wiedmann
.
2009
.
Prevalence and molecular diversity of Listeria monocytogenes in retail establishments
.
J. Food Prot
.
72
:
2337
2349
.
24.
Sheen
,
S.
2008
.
Modeling surface transfer of Listeria monocytogenes on salami during slicing
.
J. Food Sci
.
73
:
E304
E311
.
25.
Sheen
,
S.
, and
C. A.
Hwang
.
2008
.
Modeling transfer of Listeria monocytogenes from slicer to deli meat during mechanical slicing
.
Foodborne Pathog. Dis
.
5
:
135
146
.
26.
Simmons
,
C.
,
M. J.
Stasiewicz
,
E.
Wright
,
S.
Warchocki
,
S.
Roof
,
J. R.
Kause
,
N.
Bauer
,
S.
Ibrahim
,
M.
Wiedmann
, and
H. F.
Oliver
.
2014
.
Listeria monocytogenes and Listeria spp. contamination patterns in retail delicatessen establishments in three U.S. states
.
J. Food Prot
.
77
:
1929
1939
.
27.
Todd
,
E. C.
,
B. S.
Michaels
,
J. D.
Greig
,
D.
Smith
, and
C. A.
Bartleson
.
2010
.
Outbreaks where food workers have been implicated in the spread of foodborne disease. Part 8. Gloves as barriers to prevent contamination of food by workers
.
J. Food Prot
.
73
:
1762
1773
.
28.
U.S. Department of Agriculture, Food Safety and Inspection Service
.
2003
.
FSIS risk assessment for Listeria monocytogenes in deli meats
.
U.S. Department of Agriculture
,
Washington, DC
. .
29.
U.S. Department of Agriculture, Food Safety and Inspection Service
.
2003
.
Quantitative assessment of relative risk to public health from foodborne Listeria monocytogenes among selected categories of ready-to-eat foods
.
U.S. Department of Agriculture
,
Washington, DC
. .
30.
U.S. Department of Agriculture, Food Safety and Inspection Service
.
2012
.
The FSIS microbiological testing program for ready-to-eat (RTE) meat and poultry products, 1990–2011
.
U.S. Department of Agriculture
,
Washington, DC
. .
31.
U.S. Department of Agriculture, Food Safety and Inspection Service, and Department of Health and Human Services, U.S. Food and Drug Administration
.
2013
.
Draft interagency Listeria risk assessment. Notice and request for comment
.
Fed. Regist
.
28
:
27939
27941
.
32.
U.S. Food and Drug Administration
.
2010
.
FDA report on the occurrence of foodborne illness risk factors in selected institutional foodservice, restaurant, and retail food store facility types (2009)
.
U.S. Department of Agriculture
,
Washington, DC
. .
33.
U.S. Food and Drug Administration
.
2010
.
FDA trend analysis report on the occurrence of foodborne illness risk factors in selected institutional foodservice, restaurant, and retail food store facility types (1998–2008)
.
U.S. Food and Drug Administration
,
Washington, DC
. .
34.
U.S. Food and Drug Administration
.
2013
.
Food Code 2013
.
U.S. Food and Drug Administration
,
College Park, MD
. .
35.
Vorst
,
K. L.
,
E. C.
Todd
, and
E. T.
Rysert
.
2006
.
Transfer of Listeria monocytogenes during mechanical slicing of turkey breast, bologna, and salami
.
J. Food Prot
.
69
:
619
626
.

Author notes

†Present address: Pew Charitable Trusts, 901 E Street N.W., Washington, DC 20004, USA.