TABLE OF CONTENTS

Executive Summary

Recommendations

Introduction: Statement of Charge to NACMCF and the Rationale for the Approach to the Charge

Specific Charge to the Committee

Public Health Focus

Committee's Approach to Answering the Charge

Scope of Committee's Work

General

Background: DOD Procurement

Food Categories

Process Flow Diagrams

 Principles Used in Making the Process Flow Diagrams

 Interpreting the Process Flow Diagrams

 Intended Use of the Process Flow Diagrams

Manufacturing Processes and Opportunities for Loss of Process Controls

 Measuring Insanitary Conditions

Sampling and Testing

 Use of Statistical Sampling Plans in the Supply Chain

 Finished-Product Testing to Aid in the Management and Control of Suppliers

 Process Control

 SPC Limits

 Process Capability

 SPC Monitoring via Microbiological Testing

 Considerations for Finished-Product Testing

 Sampling Plans for Screening and Auditing Suppliers

 Surveillance at Point of Sale

 Microbiological Limits and Criteria

 Development of Limits and Criteria

 Pathogens Important to Public Health

 Indicators that Reflect Loss of Process Control or Insanitary Conditions

 Comments on Microbiological Limits for Specific Food Categories

 Routine and Nonroutine Testing

 Plan of Action If Limits Are Exceeded

 Commodity-Specific Comments on Microbiological Limits

Other Indicators of Process Control and Sanitary Conditions

Glossary

Appendices: Available as supplemental materials

References

EXECUTIVE SUMMARY

The Department of Defense (DOD) purchases a grocery-store array of foods (hereafter to include bottled water and packaged ice) throughout the world. DOD primarily uses the assessment of a supplier's food safety plan, including its hazard analysis critical control point (HACCP) system, to determine whether a supplier is an acceptable supplier to meet its mission requirements. For these suppliers, DOD can rely less on microbiological testing and more on process-oriented, risk-based preventive controls that ensure the supplier's manufacturing process is controlled and sanitary conditions are maintained. However, some mission requirements include the need to purchase foods where suppliers may not have fully developed food safety plans, including HACCP systems. In these instances, DOD has a need for standardized sampling and testing programs that reflect process control and assess sanitary manufacturing conditions. Such programs, defined herein, would enable DOD to monitor suppliers from centralized locations, prioritize supplier audits, and conduct cost-effective and meaningful verification testing.

To assist DOD with its ability to assess suppliers that do not have well-established food safety plans, the National Advisory Committee on Microbiological Criteria for Foods (NACMCF or Committee) has provided microbiological limits for food categories that reflect process control and sanitary manufacturing conditions. These limits are not microbiological criteria for finished products typically found in a product specification but are provided to help DOD assess process control and sanitary conditions in those suppliers without evidence of a documented and functioning food safety plan. Combined with process flow diagrams of manufacturing processes, the microbiological limits also provide guidance to DOD auditors when assisting suppliers with corrective and preventive actions taken when there is evidence of insanitary conditions and lack of process control. The processes for statistical analyses of microbiological data for DOD and suppliers are provided to optimize the use of the data in making decisions affecting process control and sanitation. These limits are based on expert opinion, industry recommendations, and published finished-product microbiological criteria from global sources.

RECOMMENDATIONS

  • DOD should develop and implement a supplier expectations policy and program to address supplier programs, such as crisis management, environmental monitoring, sanitation effectiveness monitoring, pest control, good manufacturing practices (GMPs), HACCP systems, preventive maintenance, the use of statistical process control (SPC), and verification testing, as appropriate to the individual operation.

  • DOD should share the information contained herein with suppliers who do not have documented and functioning food safety plans to begin the process of having them develop SPC charts to demonstrate process control and sanitary conditions. These charts should be based on microbiological limits provided in Appendix J. Suppliers also should examine trends in the data from the supplier's environmental monitoring program (EMP) and sanitation effectiveness monitoring program. A timeline for development and use of these charts should be set.

  • DOD should provide a list of expert consultants that can assist suppliers with development and implementation of the SPC charts and EMP.

  • DOD should develop purchasing specifications that include microbiological criteria for foods purchased through the Worldwide Directory as well as for those foods purchased outside of the Directory. These specifications should be set initially based on consultation with industry experts, shared as draft specifications with the supplier community, and fully adopted after feedback and data confirm that the specifications and the microbiological criteria imbedded therein ensure safe and wholesome products and are realistic and practical.

  • DOD should communicate microbiological standards, specifications, and guidelines to all suppliers and brokers.

  • DOD should request that suppliers document their acceptance of the standards, specifications, and guidelines in manufacturing food for DOD.

  • DOD should require that their suppliers, even if instructed through brokers, use the sampling plan, specified limits, and analytical methods specified in the microbiological criteria (when formally developed and implemented) and maintain documentation for audit purposes.

  • DOD should require Certificates of Analysis and consider the use of Certificates of Compliance with each shipment of product received to verify compliance with the specified microbiological criteria (when formally developed and implemented).

  • If there is a third-party intermediary that is involved in the food supply chain, the intermediary should be required to receive, maintain, and transfer the Certificate of Analysis or Certificate of Compliance with the products.

  • Whenever and wherever possible, meat, poultry, and processed egg products should be purchased from countries with U.S. Department of Agriculture (USDA)-equivalent inspection programs and from manufacturing establishments that meet the requirements of the inspection system. When this is not possible, the manufacturing facility should meet the requirements specified by USDA for production of meat, poultry, and egg products. The product specification for fresh (unfrozen) raw meat and poultry should include a maximum time between slaughter and receipt by DOD.

  • DOD should leverage the implementation of the Food Safety Modernization Act (FSMA) legislation and regulations, requiring all suppliers that would be regulated by the U.S. Food and Drug Administration (FDA) to meet statutory and regulatory requirements as mandated by FSMA and corresponding regulatory rules.

  • DOD should use an information technology solution that requires all suppliers to input key data such as location, contacts, product identification, code dating and traceability program, significant hazards, audit scores, regulatory actions (e.g., equivalent to recalls, market withdrawals, and noncompliance records), SPC data, and microbiological test data. DOD should capture appropriate data in a standardized electronic spreadsheet or database.

  • The risk of potential foodborne pathogens should be considered not only for fresh-cut and frozen fruits and vegetables but also for whole or unprocessed fruits and vegetables.

  • The risk of potential foodborne pathogens should be considered not only for processed nuts, spices, and herbs but also for unprocessed nuts, spices, and herbs.

  • DOD should develop procedures to collect appropriate metadata associated with assay results. Metadata are data about the data, such as methods, sample size, analytical unit, point of sampling, and the reason the sample was collected.

  • DOD should incorporate evaluation of sampling schemes and SPC into audit procedures for those suppliers using the microbiological limits to assess process control and sanitary conditions.

  • DOD should consider enhancing diagnosis and reporting of foodborne illness and integrating this information amongst the Services to help identify potential problems within the supply chain.

INTRODUCTION: STATEMENT OF CHARGE TO NACMCF AND THE RATIONALE FOR THE APPROACH TO THE CHARGE

DOD has specific action levels for various microbiological pathogens (e.g., Salmonella, Listeria monocytogenes, Escherichia coli O157:H7, and Clostridium perfringens) and microbiological toxins in certain raw and processed meat, poultry, egg products, and other products, such as fresh fruits and vegetables, procured globally for U.S. military personnel (Appendix O of the U.S. Army Public Health Command [USAPHC] circular 40-1, Worldwide Directory of Sanitarily Approved Food Establishments for Armed Forces Procurement (24)). Herein, this circular is referred to as the Worldwide Directory. In addition, there are bacteria that, when present in higher numbers, may indicate that processing conditions did not adequately prevent bacterial growth or reduce bacterial contamination of the product. DOD has encountered circumstances where the presence of potential pathogens or the numbers of nonpathogenic indicator bacteria have generated concerns about the safety and/or wholesomeness of products. DOD seeks updated microbiological limits to better evaluate process control and insanitary1 conditions at the point of production.

The Committee agreed with the need to establish microbiological limits to help assess process control and sanitary conditions at DOD suppliers that do not have documented and functioning food safety plans, including HACCP systems. In time, the testing by these suppliers, and to a lesser extent by DOD, should assist these suppliers to develop functioning food safety plans and enable the suppliers to meet the microbiological specifications established by DOD. DOD also expressed interest in the use of criteria such as Staphylococcus aureus and Bacillus cereus levels in ready-to-eat (RTE) products, mesophilic aerobic plate counts (APCs) in raw and RTE products, and other possible indicators (e.g., E. coli, coliforms, Enterobacteriaceae, enterococci, and gas-forming anaerobes) for establishing that food was manufactured with process controls and under sanitary conditions.

SPECIFIC CHARGE TO THE COMMITTEE

Because of the many questions regarding microbiological limits that might indicate poor process control or insanitary conditions, the Committee was asked for its guidance to clarify the following issues.

  • Describe processes and important considerations that could be used to develop a microbiological criterion for a particular product (e.g., bagged leafy greens, dairy products, grain-based products, raw ground beef, and RTE sliced luncheon meat) at various points in the process that might indicate poor process control and/or insanitary conditions. Describe how the processes and considerations could differ in other regions of the world where processing conditions may make certain indicators or levels of indicators more or less appropriate.

  • At the point of production, what levels of S. aureus, B. cereus, E. coli, coliforms, Enterobacteriaceae, enterococci, and/or gas-forming anaerobes in RTE finished products might indicate (i) a possible process control problem or insanitary conditions or (ii) a potentially hazardous product unfit for distribution? How might the levels and the applicability of these criteria vary between different RTE products (e.g., processed meat, poultry, egg products, refrigerated meat or poultry salads, and bagged leafy green salads)?

  • At the point of production, what mesophilic APC in RTE finished products and in nonintact raw meat and poultry products might indicate a possible process control problem or insanitary conditions? How might these criteria vary between different RTE products (e.g., processed meat, poultry and egg products, and refrigerated meat or poultry salads)? How might these criteria vary between different nonintact raw products (e.g., beef trimmings versus ground product)? How might these levels be expected to change during the expected shelf life of the product?

  • Are there other potential indicators (e.g., microbiological, biochemical, or molecular parameters) of process control that should be considered? If so, how might these apply at various points in the process to major product categories (e.g., processed meat, poultry and egg products, bagged leafy green salads, and refrigerated meat or poultry salads)?

  • Discuss various sampling plans (e.g., International Commission for the Microbiological Specifications for Food [ICMSF] two- or three-class plans) that may be applicable for the various analytes and products identified in the questions above.

The Committee notes that the microbiological limits reflecting process control and sanitary conditions requested by DOD should not be misinterpreted as microbiological criteria (specifications and guidelines) for finished food products. It is important that persons reading and using this document do not immediately transfer the limits provided herein to microbiological criteria for foods. Over time, as suppliers without documented and functioning food safety plans, including HACCP systems, use the microbiological limits to establish that their processes are in control and that sanitary conditions exist during manufacturing, they can complement this testing with their development of food safety plans that will demonstrate and ensure that the products purchased by DOD meet the microbiological criteria for finished food products. Once such documented and functioning food safety plans are audited by DOD and found to be effective, testing using the microbiological limits provided herein will be secondary and useful when there is evidence that there is a lack of process control or sanitary conditions and investigative actions are undertaken to determine root causes.

PUBLIC HEALTH FOCUS

With the large number of personnel served by DOD, the wide variety of raw, RTE, and fresh foods procured, and the high number of countries, brokers, and suppliers, the implications for failures in the food safety systems are considerable. While insanitary conditions and process failures can lead to higher numbers of indicator organisms (or classes of microorganisms, such as coliforms or aerobic bacteria detected by APCs; herein referred to as indicator organisms), the greater risks are failures leading to increased prevalence of pathogens in foods.

Verification testing by DOD, while limited in scope and absolute numbers of tests, should provide feedback to suppliers to improve controls where necessary. DOD inspection and auditing staff need to be equipped with tools to assist them in their evaluation of suppliers of a wide array of products. One tool will be process flow diagrams that illustrate points in the manufacturing process where loss of control or insanitary conditions can lead to introduction or growth of microbial contamination.

COMMITTEE'S APPROACH TO ANSWERING THE CHARGE

The Committee leveraged the expertise of the Committee members and additional experts, published literature, and finished-product microbiological criteria to assist in developing microbiological limits indicative of process control and sanitary conditions for food manufacturing. The Committee prepared process flow diagrams to reflect the major food categories purchased by DOD and used these diagrams to predict unit operations that would lead to an increased prevalence of pathogens and levels of indicator organisms or growth of contaminants based on loss of control or insanitary conditions. The diagrams also indicate where in the process there are lethality steps.

SCOPE OF COMMITTEE'S WORK

The Committee focused on major food product categories to address the questions posed by DOD. DOD purchases food products that include what one would find in a retail supermarket. It was not in the scope of the Committee to recommend finished-product microbiological criteria (i.e., product specifications and guidelines with levels of microorganisms describing acceptable, marginally acceptable, and unacceptable products) for the vast array of products. In addition, some food items purchased by DOD will no doubt fall outside of the major food categories included by the Committee. DOD will need to work with food safety experts to address any foods not covered in the major food categories.

The Committee recognized that a food safety program for DOD requires a farm-to-table approach; but the charge did not ask for the Committee to address producer food safety programs, supplier GMPs, broker responsibilities, management of the microbiological data, information technology to optimize use of supplier testing and DOD verification testing, or food service operations managed by DOD or their contractors. All of these components affect food safety and quality of the food purchased and used by DOD and should be included in its comprehensive food safety plan.

The Committee did not address the variability in food manufacturing around the world. The Committee chose to recommend microbiological limits that reflect traditional processes that are in control and running under sanitary conditions. The Committee did not address the consequences for suppliers whose processes are deemed out of control or operating with insanitary conditions. DOD will determine what steps it will take in the event a supplier is unable to substantiate that their process is in control or that sanitary conditions exist for manufacturing. This report is intended to assist DOD in meeting mission requirements, particularly when purchasing from suppliers without documented and functioning food safety plans, including HACCP systems.

In addressing the charge, the Committee did not focus on establishing microbiological criteria as part of purchasing specifications, which DOD does not currently use. The Committee does discuss the use of microbiological limits for both assessment of process control and sanitary conditions and the use of the limits, when and where appropriate, as the initial step toward developing microbiological criteria for lot acceptance.

The Committee did not address the programs and systems for delivering microbiological limits to suppliers, ensuring suppliers implement testing against the limits, reviewing microbiological data from suppliers, targeting of suppliers that do not test or do not meet the limits, collecting and managing data on microbiological quality of the products produced for DOD, and selecting new suppliers or terminating existing suppliers.

GENERAL

While sampling and testing of food products are tools to verify compliance with preventive and prerequisite programs, process control and sanitary conditions, HACCP systems, and microbiological criteria, the results do not guarantee food safety. For all refrigerated and frozen products, temperature monitoring should be done throughout storage and distribution channels as well as at receipt by DOD. Appropriate organoleptic and visual evaluation of the product and the means of conveyance in which it was delivered should occur. Where possible, continuous temperature recording documentation associated with the container delivering these products should be reviewed before accepting the products.

For food products classified under the jurisdiction of FDA inspection, the facilities supplying DOD should meet all applicable regulatory requirements, including those promulgated under the authority of the FSMA with regard to preventive controls and product safety. Meat, poultry, and egg products that would be classified under the jurisdiction of the USDA Food Safety and Inspection Service (FSIS) should meet the regulatory requirements defined by FSIS for the United States and as equivalent for foreign suppliers.

BACKGROUND: DOD PROCUREMENT

DOD procures food products from all 50 states, U.S. territories, and over 60 countries. These food products are made available to active duty and reserve service members and to retirees and eligible family members who choose to purchase from on-post facilities. Clearly the ability to safeguard these food products and ensure high quality is of paramount importance.

The DOD selection and approval process for new suppliers can take 3 months. In some situations where foods are required more rapidly, expedited processes are used to approve suppliers. All purchases of food for the military, whether on bases, at remote locations, on ships, or through commissaries or other commercial establishments, should occur using the Worldwide Directory. Most of the purchases occur through the Defense Logistics Agency, but the Defense Commissary Agency also purchases food products for grocery-type operations. Ship supply officers will purchase food products for their ships. There are instances where procurement occurs outside the Worldwide Directory, especially where fresh foods, including meat and poultry, are purchased. In many instances, these nonstandard situations are corrected when detected; however, ship supply officers are granted more freedom in buying from unapproved sources. It is noteworthy, and potentially problematic, that fresh fruits and vegetables are currently exempt from requirements to purchase from approved suppliers.

Based on the food product and a DOD informal risk ranking, approved suppliers are scheduled for DOD food protection audits on a quarterly, semiannual, or annual basis. Food protection audits encompass an establishment's total food safety and food protection systems and programs. Those facilities receiving a passing score are then listed in the Worldwide Directory. The audit scores are based on observations, with major and critical defects noted and different ramifications on the approval status for each type of finding. Audit documentation is reviewed first at one of the 20 districts, then at one of the five regions, and finally at the USAPHC where new or continued approval is granted. If major or critical failures occur, a corrective action request with a timeframe for completion is made of the supplier. Follow-up is scheduled at a time reflective of the seriousness of the failure.

DOD evaluates the supplier's food safety plan, including HACCP system, to help determine whether the supplier can provide safe and wholesome food products. This evaluation also includes a review of verification testing data that supports the efficacy of the supplier's food safety plan. In instances where a supplier is needed to meet mission requirements but does not have a documented and functioning food safety plan, DOD requires an alternative means to assess the supplier's processes and the sanitary condition of the production environment. Microbiological testing is one of the tools that help with this assessment. The microbiological limits provided herein were requested by DOD to provide guidance on what tests are appropriate for various foods and production processes and what test results may be indicative of process control and sanitary conditions.

Many food manufacturing facilities reference microbiological criteria from various entities or have established their own criteria to monitor the safety and quality of raw or RTE components used to manufacture finished products. The Codex Alimentarius defines a microbiological criterion as consisting of the following components (6, 35):

  • the purpose of the microbiological criterion (e.g., lot acceptance or process control);

  • the food, process, or food safety control system to which the microbiological criterion applies;

  • the specified point in the food chain where the microbiological criterion applies;

  • the microorganism(s) and the reason for its selection;

  • the microbiological limits (e.g., m, M, or other action levels);

  • a sampling plan defining the number of sample units to be taken (n), the size of the analytical unit, and where appropriate the acceptance number (c);

  • depending on its purpose, an indication of the statistical performance of the sampling plan; and

  • analytical methods and their performance parameters.

DOD has established their own action levels (not-to-exceed limits) for finished products to assist auditors in their evaluation of various processing systems and finished products. DOD procurement requires that food products adhere to U.S. regulatory requirements; however, as mentioned above, exceptions to this requirement may be granted under limited circumstances.

Laboratory analysis forms an integral part of the overall mission of protecting military personnel and DOD beneficiary populations from foodborne and waterborne (herein, foodborne will include waterborne) illness. The DOD program allows for testing of food products and the environments in which they are produced. Laboratory testing includes qualitative and quantitative analyses for pathogenic and nonpathogenic bacteria, respectively, as well as verifying other wholesomeness and quality parameters. Food testing equipment is located within each DOD deployable veterinary detachment to provide presumptive (considered level 1 testing by DOD) microbiological testing results, with the staff of each detachment responsible for animal care, food protection, and review of area facilities that supply food. Testing by a food manufacturing facility using an accredited laboratory (e.g., ISO 17025 (14)) is required for DOD procurement. Currently, DOD uses microbiological test results in combination with audit findings to determine the status of an establishment regarding initial and on-going approval or whether product that has been procured is safe and wholesome for military personnel.

Appropriate organoleptic evaluation of food products may be useful to assess quality. While organoleptic examination has its value, it is inherently subjective and dependent upon sensory capabilities that vary from analyst to analyst. Quantification of indicator bacteria, such as by APCs, might be more effective for determining quality of products that may have been stored for a significant period of time. However, fresh produce may have appropriate quality for use while also containing substantial comparatively high levels of aerobic bacteria.

Food processors, including those who supply DOD with RTE multicomponent products (e.g., meals and sandwiches), should be responsible for evaluating individual components (e.g., processed meats, cheese, poultry, egg products, and spices) received at their establishments. In many cases, these components may be included as ingredients in the final product without further processing to inactivate biological hazards. The supplier establishments should perform microbiological testing on these raw materials, require microbiological test results from the secondary suppliers on a Certificate of Analysis, or require the listing of microbiological criteria as elements of a Certificate of Conformance that accompanies the raw materials.

A variety of analytes (e.g., aerobic bacteria, E. coli, Enterobacteriaceae, coliforms, and enterococci) currently are monitored on a limited basis by DOD to suggest potential insanitary conditions or poor process control. This report recommends that this testing should be done by suppliers without documented and functioning food safety plans, including HACCP systems, using the microbiological limits provided herein to demonstrate process control and sanitary conditions. Currently, there is no consensus in the United States on acceptable microbiological limits for indicator bacteria to indicate a process is in control. Such limits may vary by facility, process, and food and may be best determined through the use of SPC as described herein.

FOOD CATEGORIES

Because of the vast array of food products purchased by DOD, categorization is complex. It is beyond the scope of this document to list or cover all foods purchased by DOD. The major food categories and the subcategories covered herein are as follows:

Beverages

  •  Bottled water

  •  Ice, packaged

  •  Juices and drinks, pasteurized, refrigerated

  •  Shelf stable

Dairy

  •  Butter, margarine

  •  Cheese, hard

  •  Cheese, soft, semisoft, surface ripened

  •  Cultured, pH < 4.8

  •  Cultured, pH > 4.8 and <5.4

  •  Dried products (does not include dairy ingredients used to make infant formula)

  •  Frozen desserts

  •  Milk and milk products (fluid)

  •  Processed cheese

Egg Products

  •  Pasteurized, processed

  •  Shell eggs, raw

Grain-Based Products

  •  RTE, baked items, refrigerated or temperature-time controlled for safety (TCS)

  •  RTE, baked items, shelf stable or non-TCS

  •  RTE, cereals

  •  RTE, cold-pressed bars

  •  Non-RTE, dry flour-based mixes

  •  Non-RTE, pasta, dried or refrigerated

Meals and Entrees

  •  Non-RTE, ready-to-cook (RTC) meals, includes raw ingredients

  •  RTE, deli salads, sandwiches, heat-eat meals, sushi

  •  RTE, sous vide, cook and chill

Meat, Pork, Poultry Products

  •  Non-RTE, beef and pork, raw, intact and nonintact

  •  Non-RTE, poultry, raw

  •  RTE, cooked, perishable

  •  RTE, fermented, dried

Nuts and Nut Butters

  •  RTE, not processed for lethality

  •  RTE, processed for lethality

Produce

  •  Fruits and vegetables, cut, frozen or refrigerated, minimally processed

  •  Fruits and vegetables, whole

  •  Mushrooms

  •  Packaged salads and leafy greens

  •  Vegetable sprouts

Seafood

  •  Non-RTE, raw

  •  RTE, fish, cold smoked

  •  RTE, cooked or hot smoked

  •  RTE, raw molluscan shellfish

Spices and Herbs, Coffee and Tea

PROCESS FLOW DIAGRAMS

The generic process flow diagrams for these food categories are included here (Appendix A) to identify for DOD auditors the steps in the manufacturing process where microbiological counts could potentially increase with loss of process control or development of insanitary conditions. In addition, the flow charts illustrate where there are lethality steps that reduce levels of indicator organisms and pathogens.

Principles used in making the process flow diagrams

Steps for receiving and storing packaging materials were omitted to simplify the creation and use of the process flow diagrams. It is expected that a DOD-approved food processing plant would have appropriate control and documentation of these functions, either as part of product-specific preventive controls or HACCP systems or as preventive and prerequisite programs such as standard operating procedures (SOPs) for receiving and storage. It was recognized that a finished food product could move through many storage and distribution facilities as part of the supply chain. Moreover, it is possible that a finished product of one production system could be an input for another production system. The final two steps were denoted “store finished product” and “distribute finished product” to simplify the creation and use of the process flow diagrams.

For several types of food, there are many different possible combinations of manufacturing steps. Rather than try to show all multiple combinations and step sequences, the steps that could be used in the relevant portion of the manufacturing process were listed collectively. For example, in the process flow diagram for yogurt, the “add culture” step also includes the information “(may be preceded by concentration)” and the “process” step also includes “filter, heat, separate, concentrate, stir (optional).” In the coffee process flow diagram the “process raw coffee cherries” step lists the component steps of a wet method and a dry method to process the coffee cherries. The Committee assumes that DOD personnel will be able to recognize the specific steps observed at a food processing plant from among the general manufacturing steps shown on the process flow diagrams.

Interpreting the process flow diagrams

The name of a processing step may be followed by any of the following designations: C, a step at which significant contamination may occur when adequate process controls are not in place; G, a step in the process where growth of microorganisms can occur; K, a step where there is a pathogen kill step; and S, a point where sampling and testing by the supplier are recommended for verification or investigation.

The effectiveness of the expected process controls at preventing contamination may differ considerably from step to step and product to product. For example, there would be a greater likelihood of contamination during the harvesting of coffee cherries than during the packaging of ground roasted coffee beans. Similarly, less contamination might be expected during yogurt packaging than during the packaging of raw, non-RTE seafood.

Programs for minimizing contamination at the identified steps include good agricultural practices (GAPs), sanitation SOPs (SSOPs), GMPs, SOPs for specific steps, and purchasing specifications. Steps denoted as potential contamination points may occur before or after a step causing significant reductions in the numbers of microorganisms present in the food. For example, there may be a high level of concern about L. monocytogenes contamination of RTE foods during the “package” step, and this step will be labeled with a “C”.

Intended use of the process flow diagrams

DOD personnel should use the process flow diagrams to review the general steps to manufacture the food product under evaluation. From the process flow diagram, DOD personnel should determine the step(s) at which sampling should be done by the supplier without a documented and functioning food safety plan to demonstrate process control and sanitary conditions. When microbiological or organoleptic analyses indicate that any supplier may have shortcomings in process or sanitation controls, DOD personnel should use the process flow diagram to determine steps at which contamination could occur or steps at which a failure to achieve the expected destruction of bacteria may be occurring. It shall be important that DOD consider that test results or organoleptic assessments for finished products at the point of use (e.g., commissaries) may not reflect loss of process control or insanitary conditions at the supplier since factors such as temperature control during storage and distribution can affect microbiological results and organoleptic properties and should be taken into account when deriving conclusions about a supplier's manufacturing processes.

MANUFACTURING PROCESSES AND OPPORTUNITIES FOR LOSS OF PROCESS CONTROL

The designation of food categories and subcategories is based on criteria such as the food description itself, type and extent of processing, RTE status, and chemical characteristics of the food. For each subcategory a general process flow diagram depicts the manufacturing process for the foods in that subcategory. If DOD investigates a process following the review of verification test data or as part of an on-site audit, the process flow diagrams provide insights into where in the manufacturing process the investigator or auditor could focus attention.

Measuring insanitary conditions

The Committee believes that the best assessment of insanitary conditions is not necessarily finished-product testing but is typically best achieved through strategic evaluation of the production inputs, cleaning and sanitation practices and their efficacy, and the environmental monitoring and sanitation effectiveness monitoring data generated by the supplier facility as part of their preventive controls program.

SAMPLING AND TESTING

There are various reasons for sampling and testing by DOD itself. While relying primarily on supplier testing, DOD may sample food products at locations such as distribution centers, field locations, or commissaries to determine the microbiological quality of the food product at a particular point in the supply chain. The test results from analysis of these samples can provide insights into supplier compliance with specified microbiological limits, although as pointed out above the results would be affected by the warehousing, distribution, and handling processes and conditions in the supply chain from the time of manufacturing to the point of sampling. For example, the results can provide indirect information regarding temperature control during warehousing and its impact on the shelf life of the food product.

DOD also may take samples during supplier audits. If finished products are sampled, these samples represent verification samples; the test results provide some indication of the ability of the supplier to manufacture safe and wholesome food products and provide an incentive to establish and maintain process control and sanitary conditions. The allocation of verification testing resources should include consideration of the potential presence of biological, chemical, and physical hazards, type of food, supplier characteristics and where the supplier is located, audit results, shelf life, the distribution system and likelihood of temperature abuse, as well as the cost of sampling and testing. DOD has an informal risk ranking process that has been used to define audit frequencies. A more systematic and analytical approach to risk ranking of foods and suppliers by DOD considering the factors specified above would enhance controls over food safety and quality as well as resource allocation.

The DOD process of evaluating suppliers with documented and functioning food safety plans should rely more on the documented evidence supporting effective food safety plans, including verification testing results (EMP, sanitation effectiveness monitoring, and finished-product testing where appropriate) generated by the suppliers, with the DOD sampling and testing used only for periodic verification. For those suppliers without documented and functioning food safety plans, DOD should ensure the suppliers are conducting sufficient sampling and testing to demonstrate process control and to establish that their manufacturing is occurring under sanitary conditions, using the guidance provided in this report. When deemed necessary, more finished product verification testing by the supplier and DOD may be appropriate for these suppliers until they develop functioning food safety plans.

Use of statistical sampling plans in the supply chain

Currently, DOD, through the USAPHC, maintains the Worldwide Directory but does not stipulate purchase specifications, such as microbiological criteria including sampling plans, microbiological limits, and reference methods for specific microorganism-commodity combinations. This section addressing sampling plans is not intended to provide guidance to DOD (or any other entity) for elaborating microbiological specifications for foods. Instead, the aim is to provide some contextual and statistical background for DOD to consider when evaluating food suppliers, their microbiological data, and the extent to which their manufacturing process is in control.

Strategic microbiological testing of foods, as in-process samples or finished products, provides useful information about microbiological quality, safety, sanitation, and the effectiveness and extent of process control. While it is rarely possible to use microbiological testing of foods to ensure safety and wholesomeness, it is possible to design strategic sampling schemes and select appropriate analytes and assays that can aid in the management and control of suppliers. Testing data can be used to help assess manufacturing and monitoring systems such as HACCP and preventive control programs.

In some instances (e.g., immediate need by DOD for a supplier without a documented and functioning food safety system), rapid development and implementation of HACCP systems and preventive control programs by a supplier may not be possible in the short term. In such instances, use of the microbiological limits provided in this report may be useful for suppliers and DOD to evaluate the food safety and quality performance of the manufacturing process. Furthermore, analysis of the data may help identify improvement opportunities. The Committee recommends that a long-term goal be that all approved suppliers develop and implement effective food safety plans, including HACCP systems and preventive control and prerequisite programs. In doing so, suppliers and DOD can rely less on the use of the microbiological limits described herein and finished-product testing and more on data associated with the food safety plan that demonstrate that the manufacturing process is stable and capable and that sanitary conditions are maintained continuously.

SPC methods are powerful tools to evaluate process capability and monitor the extent of control within a manufacturing process. In particular, SPC can be used to identify an out-of-control process and consequently flag events warranting investigation for an assignable cause, corrective action, and potential preventive action. In this document, we focus on sampling schemes that allow the use of SPC to assess process control and sanitary conditions, particularly but not exclusively for suppliers without a documented and functioning food safety plan. Some approaches described herein also may be suitable for a variety of other qualitatively or quantitatively measurable observations such as those identifying chemical hazards or physicochemical measurements; but control of these food process characteristics is beyond the scope of this report.

Finished-product testing to aid in the management and control of suppliers

As mentioned previously, the microbiological limits provided in this report are not microbiological criteria for finished products; although as data generated for SPC accumulate over time, they may help define realistic finished-product criteria that reflect wholesomeness, safety, process control, and sanitary conditions. Finished-product testing does have a role for verification that food is manufactured under sanitary conditions with processes that are under control.

As used herein, finished products refer broadly to food products or ingredients that have completed a manufacturing process by a supplier. It does not necessarily imply an RTE product. For example, beef trim may be considered a finished product from the perspective of a slaughter plant supplying trim to a customer (e.g., a producer of ground beef). Consequently, a finished product of one process may be an input of another.

In order to ensure the integrity of its food supply, DOD should assess a supplier's product as the output of a process that should be under control and delivers wholesome and safe product. This assessment is achieved through reviewing data supporting the supplier's food safety plan, supplier microbiological test data, surveillance of food products at receiving or in distribution, monitoring of process control at the supplier, and supplier audits, among other activities. In what follows here, the elements of process control are reviewed, and guidelines are given for statistically based activities of surveillance and process control monitoring that help ensure process control, sanitary conditions, and high-quality finished products. It is important to understand that assessing process control can take many forms including measurement and documentation of critical processing parameters such as time, temperature, and pressure, documentation of employee compliance to personnel requirements, verification and monitoring programs for SOPs and SSOPs, and evaluation of microbiological, chemical, and physical characteristics of food before, during, and after processing.

Process control

In simple terms as it relates to food manufacturing, storage, and distribution systems, process control can be defined as maintaining the output of a specific process within a desired range. Control of a process (or management of a process in general) requires accomplishment of six basic steps.

  1. The output of the process must be sampled and quantified on key attributes. Even limited information (e.g., above or below target) can be used to establish control, if the sampling rate is high enough. The higher the information content of the measurement (e.g., enumeration versus presence or absence), generally the lower the minimum required sampling rate for control.

  2. There must be predefined relevant process control performance limits and targets traceable to the basic requirements for acceptable outputs (e.g., specifications) and the history of the process.

  3. The actual sample output results must be compared to the relevant process control limits.

  4. There must be a predetermined plan of action (POA; such as a corrective action plan) based on the size and frequency of deviation from relevant limits. This POA should include the conditions under which “take no action” is the proper response to a deviation from control limits. For example, a typical set of POA choices might be: take no action, move to tightened inspection with increased sampling frequency or sample size, conduct a predetermined internal or external audit of the process that is typical for out-of-control variability, and identify an assignable cause through root-cause analysis and take corrective and preventive actions. The corrective actions specified must be validated to ensure they do help to prevent future deviations.

  5. The proper action must be decided upon based on the observed deviation.

  6. The proper action must be promptly taken to adjust the process. Failure to be prompt is equivalent to lowering sampling frequency and reduces the ability to control the process.

Failure to execute any of these steps will obstruct control of the process.

SPC limits

A process is considered under statistical control when its output varies as expected within a standard operating range (SOR) of variation (Appendix B). This refers to common cause variation and represents the random variation inherent in a process. When a process becomes out of control, its average shifts, variation increases beyond the SOR, or both. This loss of control typically is due to the introduction of a disturbance generated by an assignable cause.

SPC limits bracket the SOR and indicate the boundary between controlled and out-of-control operations. The SPC limits may be supplemented by additional statistical rules, such as run tests (i.e., a rule defining loss of control based on a run of sequential observations, such as seven measurements over the center line).

SPC limits typically are determined in one of three ways:

  1. theoretically, from careful scientific analysis of the underlying process;

  2. nonparametrically, from quantiles of the empirical distribution function, derived from historical data; or

  3. parametrically, from quantiles of an assumed model distribution (e.g., lognormal) whose parameters (e.g., mean and standard deviation) are estimated from historical data.

The first way is difficult to carry out successfully, particularly for microbiological data. The third method is typical for nonmicrobiological applications. However, all three may be useful options for establishing SPC limits in various settings.

There is a trade-off involved in the choice of the quantiles used to establish the SOR. If the upper control limit (UCL) is too low (or the lower control limit [LCL] is too high), the corresponding false alarm rate (FAR) will be too high and will monopolize resources in performing corrective actions and searching for assignable causes when actually the process is under statistical control. For example, if the UCL is chosen at the 90th percentile, then 10% of testing can be expected to result in false alarms. If the chosen percentile is too high, the FAR will be too low, and the process may drift out of control too far before it is discovered, or the sampling rate would need to be increased to counteract this effect. Similar arguments apply to the LCL used, if any.

Typical quantiles used for the UCL in SPC are 95, 99, 99.7, or 99.9%. Choice of the quantile is related to the FAR, production lots defined in part by time (e.g., hours, days, or months), and the amount of resources budgeted for dealing with exceptions. Absent other information, a reasonable rule of thumb might be to use 95 or 99% limits if the sampling rate is low (e.g., weekly), so there is no more than one or two expected false alarms per year of production; otherwise, it is conventional to use 99.7 or 99.9% limits.

It is important to note that there is a difference between a process being in statistical control and meeting specifications. A process is considered under statistical control if it is stable over time and the observed variation is due to common, chance causes inherent to the process (e.g., background noise due to normal variation in ambient temperature and humidity) and there is no between-lot variation. A food manufacturing process being under statistical process control does not imply its capability with respect to meeting microbiological specifications. The ideal situation is when a process is both under statistical control and is capable of manufacturing products that meet specifications. However, a process can be in statistical control and not capable of satisfying specifications. For example, the process consistently generates substandard product. Alternatively, a process can be out of statistical control but capable of satisfying specifications. For example, the process is designed to be robust in regard to deviations from the norm, such that it meets specifications despite high variability. Given seasonal and other sources of variability beyond a supplier's control, the latter situation may be particularly relevant to food production processes.

Process capability

Observations that fall within the SPC limits indicate the SOR of production at a facility that is under control. They indicate the typical range of results on product (in-process or finished product samples) produced when the process is under control. Specification limits are different in that they indicate the range of results that indicate company or customer requirements.

The degree by which the SPC limits fall within the specification limits reflects the process capability to meet specifications when the process is in control. If the process UCL exceeds the upper specification limit (USL) or the LCL is less than the lower specification limit, a fraction of the product produced under normal conditions will not meet the specification, even though the process is in control.

Process capability is traditionally quantified by a process capability index (Cp; Appendix B). Typically a recommendation for a new process is Cp = 1.45 and for an established process is Cp = 1.25. Equivalent nonparametric rules would be that the USL corresponds to the 99.999th percentile for a new process or the 99.99th percentile for an established process. In both instances, the USL is higher than the UCL.

SPC monitoring via microbiological testing

SPC monitoring is meant to verify that a supplier's process of production is operating in statistical control (or in terms of previous discussions, there is control of the production process) and therefore is expected to meet microbiological limits where they have relevance in relation to the process control limits. SPC monitoring requires testing at a frequency that makes the data valuable for assessment of stability and capability.

Microbiological testing presents some unique features not present in other applications where SPC is used. Unless a chemical or physical surrogate variable is used, microbiological testing typically results in a discrete value, not a continuous result. The value may be 0 or 1 (i.e., presence-absence testing), or a plate count, or the result of a sequence of serial dilutions. A zero value represents a level below the limit of quantification or detection (e.g., <10 CFU/mL or a negative result in 325 g of sample) for the particular method and test portion size involved.

Because of the discrete value nature of microbiological testing, test results are governed typically by one or more of three distributions:

  1. low prevalence (presence or absence) modeled by the binomial or Poisson distribution;

  2. single dilution plate counts modeled by the Poisson distribution; and

  3. multiple dilution or large plate counts governed by the lognormal distribution.

Examples of control charts (that illustrate statistical analysis of microbiological test results) based on DOD data are provided in Appendices C, D, E, F, G, and H. In addition, other distributions that characterize microbiological populations include the Poisson lognormal distribution. This distribution is a generalization of the Poisson that assumes that the mean level varies log normally rather than remaining constant throughout the product. Furthermore, the combination of low prevalence and a range of levels when the analyte is detected results in a zero-inflated distribution that complicates analysis. Zero inflated refers to a higher frequency of zero values than expected under a parametric distribution. For example, if the microbiological counts in a product follow a simple Poisson distribution with a mean of 0.04 CFU/g, zero counts in 25-g portions are expected with a frequency of 37%. If a higher frequency of zero counts is observed, the distribution may be a heterogeneous mixture in which the microorganism is completely absent from some proportion of the product and present and Poisson distributed in the remainder. The result would be a zero-inflated Poisson distribution.

Considerations for finished-product testing

The microbiological limits provided in this report for DOD are useful to establish process control and sanitary conditions. If suppliers or DOD test finished products, the results may be useful in assessing the microbiological quality of the product. However, to determine finished-product acceptability, additional samples may be required (n > 1), a three-class plan may be more appropriate, and microbiological criteria for a food category (and not provided in this report) shall be required. Considerations for finished-product testing are discussed herein to provide insights and guidance as the suppliers without a documented and functioning food safety plan move from establishing process control and sanitary conditions using microbiological limits to collaborating with DOD to implement microbiological criteria for product acceptance.

Determining the beginning and endpoint of a clearly defined product lot and delineating it microbiologically from other lots is critical. A product lot may be defined using a number of criteria, such as:

  • the food manufactured between defined activities (e.g., cleanup to cleanup);

  • the food manufactured within a period of time (e.g., day, week, or month); or

  • a defined quantity of manufactured food.

The process of defining lots involves thoughtful balancing of various (and sometimes competing) factors such as sampling costs, the likelihood that a lot is rejected by a customer, and the cost of lot rejection. The International Organization for Standardization (15) observes that from the point of view of the cost of sampling inspection, there is an advantage in large lots, provided the same frequency distribution is maintained as lot size increases. However, there are a number of reasons for limiting the lot size, including large lots might result in inclusion of widely varying quality (i.e., heterogeneity of assignable causes), storage and handling might preclude the formation of large lots, and the economic consequences of rejecting or recalling large lots might be unacceptably large. In process control, therefore, there are trade-offs between the increased resolution of frequent testing (e.g., every shift or daily) and the costs of sampling and laboratory analysis. While general rules are available for lot size, frequency of lot sampling, and number of samples per lot, a sampling scheme can be devised to optimize control subject to cost constraints (17).

Lot definition also has implications for SPC when used for assessing the acceptability of a lot. For purposes of SPC, an important consideration is that a lot is produced under reasonably constant conditions so that a lot is a homogeneous volume of contemporaneous production. Statistically, a volume of production is considered homogenous relative to a given characteristic (e.g., concentration of the microorganism) if the characteristic follows the same probability distribution throughout the volume (e.g., lognormal with fixed mean μ and fixed standard deviation σ). It does not mean that the characteristic is the same throughout the volume (3). That is, the conditions result in a homogenous frequency distribution that may or may not produce a spatially uniform distribution within a lot.

A homogenous distribution is often interpreted in food microbiology to indicate a homogenized product with the same mean level throughout (i.e., a Poisson spatial distribution); however, statistically a consistent or homogeneous frequency distribution can result in spatial heterogeneity within a lot (1). For example, if 2 days of production have the same mean level (μ1 = μ2) but substantially different variability (σ1 ≠ σ2), then the two production lots are not characterized by a homogenous (the same) frequency distribution. This concept is important because assignable causes that might occur between lots ought to be different from those that occur within lots. As such, an important aim of SPC methods is to evaluate between-lot variance compared to within-lot variance.

Selection of the appropriate microorganisms when deploying SPC is critical. Typically the best organisms are either (i) those that are predictably present within the sample matrix at some quantifiable level or (ii) those that are neither exceptionally rare (i.e., approaching 0% prevalence) nor ubiquitous (i.e., approaching 100% prevalence) when detected with qualitative assays. In some instances, microorganisms present at low prevalence may be useful for SPC (Appendices D and E).

Sampling frequency

Product samples may be taken systematically based on units of production or by duration of production, e.g., by shift, day, week, month, or quarter. Indicators of process control are best obtained by more frequent sampling. As a general rule, sampling frequency should be high enough to detect the presence of expected assignable causes within the first 10% of their persistence time. SPC cannot function for process control if the sampling frequency is less than twice during the assignable cause persistence time. Cost is associated with sampling and testing, so considerable economic force is exerted to drive the frequency to the minimum possible rate. However, disruptions that cause a loss of process control often persist for only a finite time, and not much is learned if they are either not detected when happening or detected too late for corrective action.

Although DOD currently conducts some sampling and testing during screening, auditing, and surveillance, to develop fully the use of SPC suppliers would need to do sampling and testing at a frequency described above. As such, the supplier needs to have access to a competent laboratory, have the technical ability to collect the appropriate samples, have the financial resources to pay for the program, and have the knowledge of SPC necessary to interpret and use the data.

Even under ideal conditions, a large quantity of data may be required before stable, precise estimates are obtained for process parameters (e.g., mean, variance, and prevalence). Shewhart (19) cautioned that assignable causes of variation are almost always present in the early stages of process control and that a long data sequence (e.g., a total sample size not less than 1,000) may be required to demonstrate that a process is in statistical control. However, acquiring additional data is subject to diminishing returns, and requiring a very long sequence of data may not be economically or technically feasible under operational conditions (Appendix I). For example, the only suppliers of perishable foodstuffs required to support DOD operations in austere areas may be small facilities without long production histories. Also, attainment of process control is often a gradual, stepwise process. Therefore, in practice, a pragmatic compromise is often warranted. As a general rule, Shewhart suggested a data sequence of not less than 25 samples of size 4 (e.g., sampling 25 lots with 4 samples per lot for a total of 100 samples) is the minimum requirement for concluding that a process is in a state of statistical control. Similarly, the ICMSF (13) recommends that a minimum of 30 lots should be examined but cautions that it may be necessary to conduct an initial process control study for longer periods or in phases.

Sampling plans for screening and auditing suppliers

Screening of new suppliers

The first step in screening a new supplier is to have the supplier conduct a self-audit against DOD supplier expectations (currently a preaudit checklist). With the self-audit or upon an initial visit, DOD should request that the supplier provide microbiological data that demonstrate that their production process is under control and occurs under sanitary conditions. The supplier could be asked for verification data supporting its food safety plan or, for those suppliers without a documented and functioning food safety plan, SPC charts that help to demonstrate their level of control (although it is unlikely such suppliers will have these charts and they will need to be provided direction, such as that given in this report). If either type of supplier does not have the information, DOD should consider whether the supplier is willing to begin the process of demonstrating that their process is under control and is operating under sanitary conditions by collecting verification data to support their food safety plan or by using the microbiological limits provided herein to support that their process is in control and production is occurring under sanitary conditions. Suppliers might be accepted under a probationary status. During the probationary period, finished-product testing may be required to assess the acceptability of the supplier's product.

For-cause auditing (directed audits)

When a potential problem has been identified (e.g., failure to achieve a microbiological criterion, prematurely spoiled product, or an outbreak of illnesses associated with consumption of a product), sampling is frequently required to determine the extent and source of the problem. The ICMSF (12) refers to investigational sampling, which includes sampling for this objective. While the sampling conducted in the course of for-cause auditing would typically require more extensive sampling than normal sampling, it differs from tightened inspection in that there are no conventional sampling plans specifically designed for determining the extent of a problem and identifying the underlying cause. The success of such sampling depends greatly on knowledge of the process, product, and microorganism. The process flow diagrams presented in Appendix A should be a useful resource for guiding the sampling conducted during for-cause auditing.

Surveillance at point of sale

DOD performs intermittent point-of-sale surveillance of finished products at locations such as commissaries. The accumulated data are valuable for various purposes such as assessing not only the suppliers' products and processes but also the potential for contamination or abuse during transportation and storage and handling practices throughout the supply chain and at the commissaries themselves. Various sampling plans are appropriate for surveillance purposes, including the sampling and testing being performed currently by DOD. However, improvements in standardization of sampling plans and associated meta-data (characterization of the data and the methods used) are warranted.

MICROBIOLOGICAL LIMITS AND CRITERIA

Development of limits and criteria

The ICMSF describes the establishment and application of microbiological criteria in considerable depth in two publications, Microorganisms in Foods 7: Microbiological Testing in Food Safety Management (12) and Microorganisms in Foods 8: Use of Data for Assessing Process Control and Product Acceptance (13). The details described in these publications will not be repeated here; however, the following discussion relates to how the development of criteria relates to the specific charges posed by DOD.

ICMSF defines three types of microbiological criteria: standards, specifications, and guidelines. Standards are mandatory criteria incorporated into a law or ordinance (normally pathogen oriented). Specifications are part of a purchasing agreement between a buyer and a supplier of a food and may be advisory or mandatory according to use. Guidelines are advisory criteria used to inform food operators and others of the microbiological content that can be expected in a food when best practices are applied (12).

Regardless of where food products are manufactured in the world, the finished-product microbiological criteria indicating safe, wholesome products for DOD would be the same. This presents challenges for DOD because manufacturers around the world do not have the same facility design requirements and standards, processing equipment and technology, sampling and testing programs, regulatory requirements, preventive and prerequisite programs, oversight and auditing, customer expectations, and food safety culture. Further complicating the development of microbiological criteria for finished products purchased by DOD are the large number and variety of products and suppliers.

In contrast to establishing appropriate microbiological criteria, if there were interest or a need to truly reflect how microorganisms are related to process capability for each manufactured product, data would need to be captured over many lots of production at each manufacturing site to determine what levels of organisms measured at various points of production reflect sanitary and insanitary conditions or lack of process control. This requires a site-specific assessment for each product individually to gain an accurate assessment of these data; this resource-intensive effort is not commonly done at manufacturing locations. Setting uniform microbiological limits for process control, while purposeful, may not accurately reflect individual processes and products within that general category. Thus, the suggested microbiological limits (Appendix J) described herein should be considered guidance to DOD representing a provisional starting point for developing empirically based microbiological data and a basis for discussion of DOD expectations with suppliers that do not have documented and functional food safety plans.

Microbiological analyses and comparison of the test results to microbiological limits, for the purpose defined herein or to finished-product microbiological criteria, yet to be fully defined by DOD for the products they purchase, may be used to verify that a supplier's control programs for controlling microbiological contamination are effectively designed and implemented. When there is evidence that the supplier's controls are poorly designed or implemented, it may be prudent to increase the frequency of microbiological testing; this testing may include testing against microbiological limits provided herein, finished-product testing, environmental monitoring, and sanitation effectiveness monitoring. It seems reasonable to expect that appropriate food safety and quality programs are more likely under the following conditions:

  • the food safety regulatory program in the supplier's country has been deemed equivalent to its U.S. counterpart;

  • the supplier has developed, implemented, and documented appropriate preventive and prerequisite food safety programs such as ensuring a safe and properly plumbed water supply, GAPs, GMPs, and SSOPs;

  • the supplier has developed, implemented, and documented a process-oriented risk-based preventive food safety plan, including a HACCP system, that substantially complies with risk-based preventive controls regulations authorized by FSMA; and

  • the supplier's food safety system has achieved third-party certification against standards fulfilling the requirements such as those specified in the Global Food Safety Initiative Guidance Document (9).

Pathogens important to public health

It is somewhat easier to establish microbiological limits, and specifications, for certain pathogens because whenever there is a likelihood of pathogens being present, sampling and testing plans can be designed to require the absence of the pathogen at a given stringency of testing, i.e., quantitative values need not be established.

The Committee considered where pathogens are reasonably likely to occur for each category of food. The pathogens may have resulted from process control failures (e.g., contaminated raw materials and ingredients, inadequate processing conditions and insufficient interventions, or failures in prerequisite programs and preventive programs) or insanitary conditions (e.g., failure in cleaning and sanitation, inferior facility and equipment design, or poor personal hygiene). Combining these analyses with summaries on the causative agents of foodborne outbreaks allowed the Committee to prepare the microbiological limits for pathogens for the major food categories that may reflect loss of process control or insanitary conditions (4).

Indicators that reflect loss of process control or insanitary conditions

Indicator organisms typically used to reflect process control or insanitary conditions include those familiar to food manufacturers, e.g., APC organisms, coliforms, E. coli, Enterobacteriaceae, S. aureus, pseudomonads, and yeasts and molds. The levels of indicator organisms that indicate loss of process control or insanitary conditions during processing are dependent upon factors such as the cleaning and sanitation procedures and products, the types of processes used, the sanitary design of equipment and the facility, and the food being manufactured.

One of the more difficult microbiological limits to establish to reflect loss of process control or insanitary conditions is that for gram-negative bacteria, whether coliforms, fecal coliforms, Enterobacteriaceae, or E. coli. Kornacki et al. (16) provide an historical evaluation of these criteria for foods and their utility based on current knowledge. None of these gram-negative bacteria accurately and consistently reflect fecal contamination of raw and processed foods nor are they useful or reliable as index organisms predicting the presence of pathogens. These criteria may be useful indicators of insanitary conditions and loss of process control; however, these uses are dependent upon many factors such as the type of food, the extent and type of processing, the relationship between bacterial numbers and food quality, and the length of time between production and sampling and testing. Kornacki et al. also reviewed the testing methods and the many variables that affect the accuracy and utility of the results. For these reasons, whichever indicator microorganisms are used, they are generally considered guidelines for use. Based on this current review, in general, the indicator microorganisms of most value would be Enterobacteriaceae followed by E. coli, coliforms, and fecal coliforms.

DOD is at a disadvantage without data from suppliers defining their normal cleaning and sanitation practices, their sanitation effectiveness monitoring program, as well as process control data measured by manufacturers throughout their production runs. Setting arbitrary quantitative limits for indicator organisms for a category of food products is guidance at best and may or may not be reflective of insanitary conditions or lack of process control. For this reason, the microbiological limits provided herein to DOD should be considered guidelines and a starting point for suppliers and DOD to evaluate the process controls and sanitary conditions under which the products were manufactured. The process flow diagrams indicating where bacterial numbers may increase during manufacturing provide some guidance to DOD on questions to ask of suppliers regarding where samples are taken or process control measurements made during processing and what corrective actions might be taken based on the results of such sampling and testing.

Comments on microbiological limits for specific food categories

One of the limitations of microbiological limits as indicators of process control or insanitary conditions is the balance of statistical validity with practicality (Appendices K, L, and M). Microbiological limits and sampling schemes are often dictated by common practice and are not based on statistical design. The guidance below is based on review of the available literature, expert opinion, and industry practice. Consequently, the limits discussed below should be considered provisional starting points toward more formally designed microbiological limits for process control that are updated and revised over time as additional data are acquired.

The tables presented in this document (Appendix J) are intended to provide guidance on microbiological limits, proposed primarily for use by DOD for suppliers without documented and functioning food safety plans, that reflect effective process controls and sanitary conditions used to produce food products using good quality ingredients, validated pathogen intervention strategies and lethality steps, GMPs, and GAPs. Microbiological populations in raw commodities are expected to be higher and more diverse than those in foods produced using a validated lethality process. The limits identified are on a “per gram” or “per milliliter” basis and typically assume a 25-g analytical unit unless otherwise described.

The microbiological limits are intended to help identify when a process is not in control so the manufacturer can investigate causes and implement corrective actions. The limits reported for indicator organism testing are not lot acceptance criteria. In some cases, the action to be taken after exceeding the limit may be to increase sampling to determine the source of contamination or to test for pathogens or other indicators of insanitary conditions. In cases where any microorganism or class of indicator organisms exceeds regulatory limits, then the lot should be evaluated appropriately and typically destroyed or diverted for reconditioning if appropriate. As an example, the FDA dairy compliance policy guide 527.300 (28) considers cheese made with pasteurized milk to be adulterated if the cheese contains 104 CFU/g S. aureus or B. cereus or 100 CFU/g E. coli; these lots should be rejected and additional investigation conducted. If enterotoxins produced by S. aureus or B. cereus are detected, the product also should be destroyed.

Enrichments (such as for pathogens in environmental sponge samples) may be performed on composite samples. However, with compositing, if samples are pulled from multiple locations or over the course of producing several lots of finished products, a positive result for the enrichment would implicate all locations and the lots manufactured during the sampling period. In contrast, enumeration data should be generated from a single sample analytical unit; pooling samples might dilute unacceptable or marginal populations with samples having low populations and thereby provide misleading results.

Assaying for APCs to assess process control and sanitary conditions may be relevant for some RTE foods but not others. APCs used to assess process control and sanitary conditions during production should be low in RTE foods in which all components of the food have received a lethality step (e.g., pasteurization, cooking, or roasting). When RTE foods contain some components that have received a lethality step but then were further handled (e.g., sliced, assembled, or mixed) before preparation of the final food product, APCs would be expected to be moderately higher. In contrast, using APCs to assess process control and sanitary conditions during the production of foods such as fresh fruits and vegetables, fermented or cultured foods, and foods incorporating these has little value as these foods would have an inherently high APC because of the normal microbiota present.

The presence of E. coli in RTE foods is undesirable because it represents poor hygienic (insanitary) conditions or inadequate heat treatment (lack of process control). Thus, E. coli should not be detected in RTE foods; generally, when microbiological specifications are established, a microbiological limit of <10 CFU/g or <3 most probable number/g (the limit of detection of usual test methods) is typical for this microorganism. Levels exceeding 100 CFU/g are typically interpreted as a level of contamination that may be associated with the introduction of pathogens or conditions that allowed pathogen survival.

The Committee concurs with the common practices for environmental monitoring, i.e., testing for Listeria spp. in wet RTE food processing environments, particularly for foods that support growth of Listeria, and for Salmonella in dry RTE food processing environments. Salmonella monitoring in warm, wet RTE food processing environments also may be appropriate depending upon the product and facility. If product contact surfaces (zone 1) are tested, finished product should be held until results are confirmed negative; if testing demonstrates that the product contact surfaces are positive for the pathogen, investigational testing in finished product and corrective action is indicated. As of 2014, the United States maintains a standard of nondetectable L. monocytogenes for all RTE food products. Other countries may allow up to 100 CFU/g L. monocytogenes in RTE foods that do not support growth, e.g., frozen foods, those with pH < 4.4 and water activity (aw) < 0.92, or those with pH < 5 and aw < 0.94) (7, 8).

All dairy food categories listed below are presumed to be made with pasteurized milk to eliminate common vegetative bacterial pathogens. Therefore, the presence of any pathogens when testing for process control or sanitary conditions represents postprocess contamination. Salmonella, E. coli O157:H7, and L. monocytogenes are considered adulterants in RTE dairy products. In the United States, these dairy products are either regulated under the Pasteurized Milk Ordinance (PMO) (32) or microbiological standards are identified in the dairy compliance guidelines (28). Other resources for microbiological specifications and guidelines include the Compendium of Methods for the Microbiological Examination of Foods, “Milk and Milk Products” (2) and the Standard Methods for the Examination of Dairy Products (34). Alkaline phosphatase concentrations in pasteurized fluid bovine milk are limited to less than 2.0 μg phenol equivalent per g in one or more subsamples, whereas cheeses may have higher limits. Actionable limits for S. aureus and B. cereus are set to 104 CFU/g, whereas limits for E. coli or coliforms are product specific.

The general recommendation for DOD procurement of any beef, pork, or poultry product, whether raw or RTE, is to identify an establishment in the country that is authorized to ship that product to the United States and to procure product from that establishment. This will ensure the establishment meets current FSIS performance standards and/or regulatory requirements. If such an establishment cannot be identified, the testing recommended in Appendix J may be used to determine the level of process control and sanitary conditions for establishments not currently authorized to ship the product to the United States.

Microbiological testing of finished products that receive a lethality step, such as baking or cooking, may not be a good indicator of improper storage temperatures and hold times (process controls) of ingredients or blends before the lethality step (such as extended runs between cleanup). Certain ingredients or foods may support microbiological growth and production of heat-stable toxins, such as those produced by S. aureus or B. cereus. Thermal treatments may inactivate the vegetative cells in the final product, but the toxin may remain. As a result, the process must have validated microbiological control steps throughout the production to minimize the risk of toxin being present in the finished product.

Routine and nonroutine testing

In setting the microbiological limits to be used by suppliers that do not have documented and functioning food safety plans, including HACCP systems, the Committee defined the recommended testing frequency as routine and nonroutine. Specific time intervals cannot be set for each indicator organism, class of indicator organisms, pathogen, environmental monitoring, or in some instances, chemical hazard (e.g., mycotoxin). The frequency of routine and nonroutine testing will be dependent upon numerous factors such as the production process, the product being produced, the sanitary design of the facility and the equipment used at the facility, the historical data generated by the supplier, the organism or class of indicator organisms, and the investigative reason for testing. General guidance on the definition of these frequencies is as follows.

Routine testing is defined as testing done at predetermined intervals at sufficient frequency to establish process control or sanitary conditions. The sampling interval may be on a physical lot basis (e.g., 2,000-lb [900-kg] combos for ground beef) or temporal basis (e.g., per shift, daily, weekly, or monthly). Nonroutine testing can be investigational or for verification, validation, surveillance, or qualifying suppliers. Nonroutine testing is less frequent and can be based on time intervals (e.g., weekly, monthly, or quarterly) or based on other indicators of lack of process control or insanitary conditions. For example, if routine testing shows that samples of a pasteurized egg product exceed limits for E. coli, testing for Salmonella may be appropriate. If routine testing of an RTE food that can support growth of L. monocytogenes indicate contamination of the food with Listeria spp., additional testing for L. monocytogenes may be appropriate. When a supplier is manufacturing multicomponent foods (e.g., frozen desserts with inclusions, deli salads, sandwiches, or entrees), routine or investigational sampling and testing may be focused on those components with the highest microbiological risk.

Plan of action if limits are exceeded

The microbiological limits provided in Appendix J are useful to assess process control and insanitary conditions. The action taken by a supplier if indicator organisms in samples taken at the supplier location exceed the specified limits should be to investigate the cause of the high counts, implement corrective and preventive actions, and reevaluate the effectiveness of the actions after implementation. In the cases of a pathogen detected when there has been no additional lethality step, an evaluation of the finished product associated with the sample tested should occur to determine if the product should be rejected or, if appropriate, reworked or diverted for processing that will inactivate the pathogen. Products contaminated with heat-stable toxins typically will be destroyed as reconditioning likely will not eliminate the hazard.

If levels of indicator bacteria in samples assayed during distribution or at the point of sale exceed the limits provided in Appendix J, a more thorough investigation should be conducted by DOD and the supplier to identify the cause of the higher counts. The investigation should note if the food was at the end of the marked shelf life, is considered perishable, if the packaging was intact, and if the chill chain was maintained during storage and distribution. Growth of spoilage microbes is expected to occur during extended storage of perishable items. The higher counts may have resulted from normal growth of spoilage microorganisms or temperature abuse rather than the lack of process control or sanitary conditions during manufacture.

Commodity-specific comments on microbiological limits.

Beverages: bottled water (artisan, mineral, purified, sparkling, spring)—Appendix A, Flow Diagram A.1, Appendix J, Table J.1

The Committee recommends routine coliform testing for bottled water and ice to assess process control and sanitary conditions. In countries where additional microbiological regulations apply, testing for those organisms may be done periodically. A 2013 World Health Organization (WHO) draft report (35) on regulations and standards for drinking water quality recommends routine testing for E. coli or thermotolerant coliforms to provide evidence that these microorganisms are undetectable in a 100-mL sample. Other indicators also were reviewed in this draft report, and the following recommendations were made. The presence of total coliforms immediately after treatment indicates inadequate treatment. C. perfringens (undetectable in 100 mL) can be used as an indicator of the effectiveness of a filtration process to eliminate enteric viruses or protozoan oocysts. Enterococci (undetectable in 100 mL) may survive longer than E coli and can be used as an indicator instead of E. coli. Total heterotrophic bacteria (limit of 100 CFU/mL at 22°C or 20 CFU/mL at 37°C) can be used for operational monitoring of treatment and disinfection and for assessing cleanliness of the distribution system. Pseudomonas aeruginosa, parasites, and enteric viruses were not considered in the WHO report, although their monitoring may be required by individual country regulations.

Beverages: ice, packaged—Appendix A, Flow Diagram A.2, Appendix J, Table J.2

Microbiological testing and limits will be similar to those for bottled water. In countries where additional microbiological regulations apply, periodic testing for the organisms listed in those regulations is appropriate.

Beverages: juices and drinks, pasteurized, refrigerated—Appendix A, Flow Diagram A.3, Appendix J, Table J.3

The Committee recommends routine coliform testing for process control purposes. Fruit juices in the United States are subject to FDA regulations mandating HACCP systems and achievement of lethality against pathogens of significance (E. coli O157:H7 and Salmonella); thus, periodic testing for pathogens may be indicated (25). This category also includes low-acid drinks such as bottled coffees, teas, and vegetable juices. For low-acid juices and drinks, the food safety plan should address the control of pathogenic sporeformers, such as Clostridium botulinum. For products that support the growth of pathogenic sporeformers and where cold-chain management cannot be guaranteed, alternative safety measures could be the inclusion of ingredients that inhibit growth (e.g., blending with acidic juice to reduce pH) or alternative processing such as ultrahigh-temperature processing to destroy spores. High concentrations of patulin can be produced in decaying or moldy apples, and thermal processing does not destroy the mycotoxin. Therefore, apple juice products should be tested for patulin (26).

Beverages: shelf stable—Appendix A, Flow Diagram A.4, Appendix J, Table J.4

Process control of shelf-stable (commercially sterile) beverages is dependent upon control of formulation and verification and monitoring of critical control points rather than routine microbiological testing. If inspection reveals indications of spoilage such as bulging containers, pH changes, and off-odors then further investigation should be done by DOD and the supplier. Methods for investigating failures in processing for commercial sterility are given in the Compendium of Methods for the Microbiological Examination of Foods (5). Shelf-stable apple juice products should be tested for patulin for the reasons described above for refrigerated juices (26).

Dairy: butter, margarine—Appendix A, Flow Diagram A.5, Appendix J, Table J.5

Although whipped butter held under unrefrigerated conditions has been associated with outbreaks of S. aureus intoxication, the low moisture and high salt content or the lactic acid concentrations of many of these products generally preclude microbiological growth. However, routine monitoring of sanitation and process control using indicators such as coliforms should be done. Products containing added seasonings, herbs, or spices may have additional testing requirements as the inclusion of unsafe adjunct ingredients has been linked to foodborne illness. Testing for S. aureus, Enterobacteriaceae, and yeasts and molds is useful under special circumstances, such as the investigation of out-of-specification results. Due to listeriosis outbreaks linked to contaminated butter, routine environmental testing of zone 2 and zone 3 surfaces for Listeria spp. should be done. Although not routinely tested, if zone 1 environmental samples are found to be positive for Listeria spp., investigational testing of the finished product should be undertaken.

Dairy: cheese, hard—Appendix A, Flow Diagram A.6, Appendix J, Table J.6

Although reported cases of foodborne illness have been linked to foods in this category, microbiological safety issues in hard cheeses made with pasteurized milk and active starter cultures are extremely rare. The presence of active cultures in these products makes the use of routine microbiological testing for APC organisms impractical as a tool for evaluation of process controls and sanitary conditions. In contrast, routine testing for coliforms as an indication of sanitary conditions should be conducted. Testing for S. aureus or E. coli is useful under special circumstances such as validation, verification, and investigation when production has occurred without adequate process control. Finally, routine environmental testing of the food production environment for the presence of Listeria spp. is recommended as a verification step for sanitation programs.

Dairy: cheese, soft, semisoft, surface ripened—Appendix A, Flow Diagram A.7, Appendix J, Table J.7

This category represents a broad range of cheeses. Routine environmental monitoring for Listeria spp. in the environment and coliforms in finished product should occur for all products in this category. For products in this category, such as soft cheeses with high pH values that support the growth of L. monocytogenes and have been implicated in illness, in-plant monitoring for this pathogen may be appropriate (18). Testing for S. aureus and E. coli may be used when processing or insanitary conditions indicate a potential increased microbiological risk.

Dairy: cultured, pH < 4.8—Appendix A, Flow Diagrams A.8a and 8b, Appendix J, Table J.8

Rapid acidification and low final pH of these products precludes growth of bacterial pathogens. The presence of active cultures in cultured dairy products make the use of most routine microbiological testing impractical as a tool for evaluation of process controls and sanitary conditions. Routine testing by suppliers for coliforms is recommended to assure compliance with pertinent U.S. regulations and guidance (32). Nonroutine testing for S. aureus is advisable under limited conditions such as evaluating the impact of a slow fermentation processes. Mold and yeast testing may be applicable when producing cultured products without mold inhibitors or when products contain inclusions such as fruit puree that are known to carry spores. Finally, routine environmental testing of the food production environment for the presence of Listeria spp. is recommended as a verification step for sanitation programs.

Dairy: cultured, pH > 4.8 and <5.4—Appendix A, Flow Diagram A.9, Appendix J, Table J.9

The active starter culture and acid content present in these fermented products reduces the growth rate of bacterial pathogens; but because the pH is higher than the aforementioned cultured products with pH < 4.8, prevention of postpasteurization contamination is more critical. The presence of active cultures in these products makes the use of most routine microbiological testing impractical as a tool for evaluation of process controls or insanitary conditions. However, routine testing by suppliers for coliforms is recommended to assure compliance with pertinent U.S. regulations and guidance (32), and routine environmental testing of the food production environment for the presence of Listeria spp. is recommended as a verification step for sanitation programs. Although typically not tested, if zone 1 environmental samples are positive for Listeria spp., finished-product testing for L. monocytogenes should occur. Testing for S. aureus, psychrotrophic microorganisms, yeasts, and molds is useful under the special circumstances described above for cultured products with pH < 4.8, when investigating results exceeding microbiological limits, or during validation and verification efforts.

Dairy: dried products (does not include dairy ingredients used to make infant formula)—Appendix A, Flow Diagram A.10, Appendix J, Table J.10

The low moisture content of dried dairy products precludes microbiological growth. However, routine monitoring of sanitation using coliform counts and APCs should occur. Furthermore, routine testing for Salmonella by suppliers should occur as these products have been implicated in cases of salmonellosis. Nonroutine testing for S. aureus and B. cereus should be done under special circumstances such as during investigation of possible mishandling prior to drying, validation or verification efforts, or an investigation done in response to results indicative of process failures or insanitary conditions.

Dairy: frozen desserts—Appendix A, Flow Diagram A.11, Appendix J, Table J.11

Dairy ingredients used in a dessert mix are pasteurized and will have low microbiological counts; frozen storage will control microbiological growth. Routine testing for coliforms by suppliers should occur to establish process control and monitor sanitation. Although APCs can be used to monitor process control, inclusions such as nuts, cookie dough, and fruits may result in higher populations than the base mix. Periodic testing for Salmonella may be indicated under special circumstances such as when lack of process control is suspected, when the supplier is using inclusions that have been previously associated with outbreaks, or during validation or verification efforts.

Dairy: milk and milk products (fluid)—Appendix A, Flow Diagram A.12, Appendix J, Table J.12

Fluid milk in the United States is produced under the PMO (32), which provides microbiological limits; when evaluated, such as when there is a pasteurization issue, alkaline phosphatase concentration must be <2.0 μg phenol equivalent per g as an indicator of adequate pasteurization. Routine testing for APCs and coliform counts by suppliers should occur to ensure regulatory compliance, to help establish process control, and to assist with evaluating sanitary conditions. Routine environmental monitoring of zone 2 and zone 3 surfaces for Listeria spp. is recommended.

Dairy: processed cheese—Appendix A, Flow Diagram A.13, Appendix J, Table J.13

This product is manufactured by heating cheese with water, emulsifier, and other ingredients to kill vegetative pathogens. Molten cheese may then be hot-filled into loaves or blocks and chilled and cut into individual slices for use. These cheeses are intended to be stored refrigerated. Shelf-stable hot-filled cheese spreads or cheese sauces must be formulated for safety to inhibit C. botulinum. Cooling processed cheese on casting belts or chill rolls may involve a relatively high degree of environmental exposure of the product. The presence of non-sporeforming microorganisms is indicative of postprocess environmental contamination. Low levels of such contamination are inevitable in these cases. Consequently, processed cheese producing facilities need to have robust environmental sampling and control plans for Listeria spp. and Salmonella. Formulae with low levels of salt in the moisture phase could potentially allow growth of enterotoxin-producing Staphylococcus spp., principally S. aureus, likely originating from human contact. The presence of E. coli on processed cheese is reflective of production in an insanitary environment.

Egg products: pasteurized, processed—Appendix A, Flow Diagram A.14, Appendix J, Table J.14

Pasteurized egg products and pasteurized shell eggs receive a lethality treatment during processing and may be used in dishes that are uncooked or lightly cooked. These products may be recontaminated during packaging, handling and storage. These products should be tested by suppliers routinely for S. aureus, coliforms, APC organisms, and Salmonella to verify process control. Periodically, suppliers may test these products for B. cereus and Enterobacteriaceae. Routine environmental testing for Listeria spp. and Salmonella is useful to evaluate sanitary conditions. If samples exceed the microbiological limits, further investigation and correction action should occur. Environmental monitoring of zone 2 and zone 3 surfaces for Listeria spp. is recommended; if Listeria spp. are found, it may lead to testing of zone 1 surfaces for Listeria spp. Finished-product testing should occur for L. monocytogenes if Listeria spp. are detected on zone 1 surfaces (indicative of insanitary conditions) or suspected illnesses are reported.

Egg products: shell eggs, raw—Appendix A, Flow Diagram A.15, Appendix J, Table J.15

Raw shell eggs are not pasteurized and are not intended for consumption without an additional lethality step, such as thorough cooking. Regulations in the United States require that high-volume producers (>50,000 laying hens) test for Salmonella serotype Enteritidis to verify nondetection of this pathogen in the shell eggs (27). High-volume producers supplying shell eggs to DOD should test for Salmonella Enteritidis. For other producers, the Committee recommends only periodic or investigational testing of raw shell eggs, and no microbiological limits are provided. Testing for E. coli, coliforms, or Enterobacteriaceae by suppliers may be useful to assess sanitary conditions or establish process control.

Grain-based products: RTE, baked items, refrigerated or TCS—Appendix A, Flow Diagram A.16, Appendix J, Table J.16

These products are prepared with a lethality step to eliminate pathogens, but the potential of recontamination during handling and the pH and aw ranges (that can support microbiological growth during extended out-of-refrigeration storage) warrant microbiological testing. Routine monitoring of coliforms by suppliers should assess insanitary conditions (including postprocess contamination). APC testing should not be conducted if the products include ingredients that are prepared using starter cultures (e.g., cheese and salami).

Grain-based products: RTE, baked items, shelf stable or non-TCS—Appendix A, Flow Diagram A.17, Appendix J, Table J.17

When manufacturing these products, the dough or batter goes through a baking step that provides lethality against pathogens, and pathogen growth is unlikely during storage due to reduced aw. While routine microbiological testing by suppliers generally is unnecessary, environmental monitoring and in-process sample testing may be appropriate under special circumstances that may increase the microbiological risk (e.g., excessive water due to condensate or roof leaks) or when ingredients are added after the lethality step (e.g., dusting of bread surface with flour).

Grain-based products: RTE, cereals—Appendix A, Flow Diagram A.18, Appendix J, Table J.18

RTE cereals are made from grains that go through a lethality step sufficient to eliminate pathogens of concern. Mycotoxin surveillance testing should be completed on incoming grains to ensure the grains meet the individual country's regulations. These RTE grain-based products do not support the growth of microorganisms due to the very low aw. Routine microbiological testing of finished product by suppliers is not recommended; but routine environmental testing for Salmonella is useful to assess sanitary conditions. Nonroutine testing for coliforms, Enterobacteriaceae, APC organisms, and Salmonella by suppliers is appropriate for verification purposes, for qualifying lines, or when events occur during processing that may increase the microbiological risk (e.g., excessive water due to condensate or roof leaks). If vitamin-containing or other such solutions are sprayed atop cereals after heat processing and depending on the source and processing of these solutions, sampling and testing of these solutions may be a useful measure of process control.

Grain-based products: RTE, cold-pressed bars—Appendix A, Flow Diagram A.19, Appendix J, Table J.19

Cold-pressed bars are made from cooked grains, carbohydrate-based binders, and inclusions such as fruit, nuts, and chocolate. Verification of the microbiological quality of ingredients used in the cold-pressed bar formula is important since the bars will not receive a validated lethality step during manufacturing. Recommendations for finished product and environmental testing by suppliers are the same as those for RTE cereals above.

Grain-based products: non-RTE, dry, flour-based mixes—Appendix A, Flow Diagram A.20, Appendix J, Table J.20

These non-RTE grain-based products harbor a complex and extensive microbiota, and routine microbiological testing by suppliers does not provide useful data to indicate process control and sanitation (20). Flour is a minimally processed commodity that is ground and sifted without any lethality step. These products should receive a lethality step to eliminate pathogens before consumption.

Grain-based products: non-RTE, pasta, dried or refrigerated—Appendix A, Flow Diagram A.21, Appendix J, Table J.21

Pasta is produced by combining flour and water and sometimes other minor ingredients. The microbiological profile may be similar to that of flour, and routine testing by suppliers is not particularly useful. However, the manufacturing process must be controlled to minimize proliferation of naturally occurring microbiota after the introduction of moisture. Nonroutine testing of in-process samples by suppliers may be useful in special circumstances (e.g., evaluation of potential growth of and enterotoxin production by S. aureus during extended down time prior to drying or refrigeration). Although most of these products are intended to be cooked by consumers before consumption, some varieties, such as instant noodles, may be prepared with limited heating. Cooking of refrigerated pasta filled with meat or cheese may be sufficient to cook the outer pasta but not sufficient to provide a validated lethality step in the product interior. Verification testing of raw materials (to support the Certificate of Analysis) and periodic testing of product by suppliers for Salmonella may be appropriate, and environmental testing for Listeria spp. or Salmonella should occur to verify sanitary conditions.

Meals and entrees: non-RTE RTC meals, includes raw ingredients—Appendix A, Flow Diagram A.22, Appendix J, Table J.22

This category includes a wide range of multicomponent (some raw) frozen or refrigerated food products, which are expected to be cooked by the consumer or food service operation. Routine testing of these meals is not recommended; however, manufacturers should be aware of the following points. Suppliers should assess the pathogens and indicator organisms associated with their products and sample and test if there is a reason to do so. Some of these meals and entrees may be improperly prepared by the consumer using conventional or microwave ovens and may not undergo a validated lethality step. Pathogens of concern may vary depending on the specific food. For example, meals prepared with cooked rice may pose a greater risk for B. cereus; E. coli O157:H7 may be of concern for foods including raw nonintact beef, and poultry products may contain Salmonella. Histamine testing may be appropriate when scombroid fish species are present.

Meals and entrees: RTE, deli salads, sandwiches, heat-eat meals, sushi—Appendix A, Flow Diagram A.23, Appendix J, Table J.23

This category includes a wide range of multicomponent, short shelf-life, and refrigerated food products. They are expected to have diverse microbiological populations depending on the ingredients used, may include ingredients which are raw such as fresh produce, and are frequently subjected to multiple handling steps, which can introduce contamination. Routine testing by suppliers of in-process or finished products for E. coli and environmental testing for Listeria spp. and in some instances Salmonella spp. should occur to assess process control and sanitary conditions. As with the non-RTE RTC meals, other nonroutine testing of indicator organisms and pathogens may be appropriate depending on the ingredients used and the type of finished product. Although not routine, if Listeria spp. is found in zone 1 environmental samples, investigational testing for L. monocytogenes may be indicated.

Meals and entrees: RTE sous vide, cook and chill—Appendix A, Flow Diagram A.24, Appendix J, Table J.24

Sous-vide products are prepared with raw or partially cooked foods, which are vacuum packaged in an impermeable bag, cooked in the bag, rapidly chilled, and refrigerated with time-temperature combinations that inhibit pathogen growth. If the cook process does not provide at least a validated 6-log reduction of nonproteolytic C. botulinum spores (10), validation data should be provided by the supplier to demonstrate that the process eliminates vegetative pathogens. Because of the lack of inhibitory barriers in typical sous-vide products and the concern for potential outgrowth of botulinum spores, strict adherence to refrigerated storage after treatment is extremely important. If a validated cook step is used and verified, no routine testing is recommended. In the absence of a validated cook process, testing for vegetative microorganisms should be done by the supplier on postcook samples to verify the thermal process. Testing for E. coli can serve as a verification of thermal processing; periodic testing for coliforms, Enterobacteriaceae, and APC organisms is useful for verification purposes. If cooling deviates from prescribed requirements such as those given in USDA compliance guidelines Appendix B (21), testing for C. perfringens may be useful as a part of the supporting documentation for safety. Routine testing for C. perfringens typically is not done.

Meat, pork, poultry products: non-RTE, beef and pork, raw, intact and nonintact—Appendix A, Flow Diagram A.25, Appendix J, Table J.25

These products include both intact (e.g., nontenderized steaks and chops) and nonintact (e.g., whole muscle destined for ground product, trim, or needle-tenderized steaks) raw beef and pork products. Under normal operating conditions, no routine testing is recommended. When it is necessary to meet a regulatory or customer requirement to confirm production is occurring with process control and sanitary conditions, suppliers should test for E. coli (typical for the United States) or Enterobacteriaceae (typical for the European Union). Those manufacturers supplying DOD with nonintact product should request that their suppliers (secondary suppliers) provide a Certificate of Analysis demonstrating that the raw materials have tested negative for E. coli O157:H7 and other Shiga toxin–producing E. coli (STEC), if appropriate. Suppliers to DOD also may test for Salmonella to meet regulatory requirements or to provide evidence that they are meeting performance standards that indicate production has occurred under sanitary conditions; this testing may typically be done only for ground products.

Meat, pork, poultry products: non-RTE, poultry, raw—Appendix A, Flow Diagram A.26, Appendix J, Table J.26

These products include both intact (e.g., noninjected whole birds or noninjected parts) and nonintact (e.g., injected, “enhanced,” or vacuum-tumbled poultry parts or ground poultry) raw poultry products. Under normal operating conditions, no routine testing is recommended. Production of these foods should include appropriate process controls to reduce pathogens to acceptable levels and to prevent pathogen growth. When it is necessary to meet a regulatory or customer requirement to confirm production is occurring with process controls and sanitary conditions or under specific circumstances when an investigation is underway, suppliers may test for Salmonella and Campylobacter to verify process control and that pathogens are being reduced to acceptable levels. In this case, testing should be performed on the relevant product type such as raw poultry parts if they are the product type purchased. Testing for indicator organisms or classes of organisms such as E. coli, coliforms, Enterobacteriaceae, or APC organisms could provide additional information regarding maintenance of process control and sanitary conditions.

Meat, pork, poultry products: RTE, cooked, perishable—Appendix A, Flow Diagram A.27, Appendix J, Table J.27

This group includes a spectrum of cooked beef, pork, and poultry products that require strict refrigeration for shelf life and safety (e.g., deli meats and hot dogs). While process control is often monitored through routine testing for E. coli, potential contamination with L. monocytogenes is a major concern and should be addressed by the supplier through routine environmental monitoring of zone 2 and zone 3 surfaces for Listeria spp. Although not routine, if zone 1 environmental samples are positive, finished-product testing for L. monocytogenes may be indicated. Nonroutine testing for coliforms, Enterobacteriaceae, APC organisms, Salmonella, and C. perfringens may be useful for additional verification of sanitary conditions or adequate cooling or as periodic verification of process control.

Meat, pork, and poultry products: RTE, fermented, dried—Appendix A, Flow Diagram A.28, Appendix J, Table J.28

These products (e.g., jerky, dried pepperoni, and meat sticks) are characterized by having chemical and physical characteristics (e.g., aw and pH) that ensure the products will not spoil or become unsafe when stored out of refrigeration throughout the manufacturer's specified shelf life. However, it is essential that production of these foods include appropriate process steps to reduce pathogens to acceptable levels and prevent growth of pathogens or the formation of their toxins (e.g., cooking jerky with adequate humidity to prevent surface drying, active fermentation to inhibit growth of S. aureus, and a lethality step to eliminate low-infectious-dose pathogens such as Salmonella and E. coli O157:H7) (11, 22, 23). Suppliers should use E. coli for routine monitoring; coliforms and Enterobacteriaceae may be appropriate for verification monitoring. Testing of products for bacteria such as Salmonella, E. coli O157:H7, and S. aureus may be appropriate when process controls are suspect, e.g., failed fermentation or extended drying times.

Nuts and nut butters: RTE, not processed for lethality—Appendix A, Flow Diagram A.29, Appendix J, Table J.29

Raw nuts (not processed for lethality) may be contaminated with microbiota from orchards, the ground, or equipment and personnel during harvesting, shipping, processing, and handling. Because consumption of raw nuts has been associated with illness, suppliers should test in-process samples and finished products routinely for Salmonella and implement an environment testing program that includes testing for Salmonella. For certain nuts (e.g., peanuts, pistachios, and Brazil nuts), routine testing for aflatoxin B1 should be done. Nonroutine testing for E. coli and aflatoxin B1 (for those not tested routinely for aflatoxin B1) may be done to assess sanitary storage and production and the quality of the raw nuts.

Nuts and nut butters: RTE, processed for lethality—Appendix A, Flow Diagram A.30, Appendix J, Table J.30

In this category, peanuts and tree nuts are processed for lethality (e.g., by dry roasting, oil roasting, or steam processing). Because nuts and nut butters have been associated with illness, routine environmental testing, testing in-process samples, and finished-product testing for Salmonella should be done. For certain nuts (e.g., peanuts, pistachios, and Brazil nuts), routine testing for aflatoxin B1 should be done. Nonroutine testing for E. coli and aflatoxin B1 (for those not tested routinely) may be conducted to help assess sanitary storage and production and the quality of the raw nuts used in manufacturing.

Produce: fruits and vegetables, cut, frozen or refrigerated, minimally processed—Appendix A, Flow Diagram A.31, Appendix J, Table J. 31

Further processing of fresh fruits and vegetables may increase or decrease microbiological populations depending on GMPs, sanitary design of equipment, washing, blanching, or the use of antimicrobials. Routine testing by suppliers of product for E. coli and the environment for Listeria spp. should be done to assess process control and sanitary conditions. Periodic testing by suppliers of in-process or finished products for Salmonella or E. coli O157:H7 (or other appropriate STEC) may be pertinent depending on the commodity, geographic location, and use of GAPs.

Produce: fruits and vegetables, whole—Appendix A, Flow Diagram A.32, Appendix J, Table J.32

Fruits and vegetables are expected to have microbiota associated with them. Whole fruits and vegetables may be washed before introduction to commerce but undergo no other lethality step. Environmental testing in the packing house for Listeria spp. and Salmonella should be done by the supplier to assess sanitary conditions, with the frequency dependent upon factors such as the commodity, geographic location, and use of GAPs. Although not listed in Table J.32 nor routine, the DOD may consider testing (by the supplier or DOD) for Cyclospora cayetanensis, Cryptosporidium parvum, enteric viruses, or Shigella spp. as appropriate when there is knowledge or suspicion of high-risk farming and handling practices (e.g., where evidence of previous contamination exists, water contamination is likely, or contaminated fertilizer is used).

Produce: mushrooms—Appendix A, Flow Diagram A.33, Appendix J, Table J.33

Mushrooms are generally commercially produced indoors on composted substrate. They are grown, harvested, sorted, graded, and packaged and may or may not be sliced. No routine testing of product is typically conducted because populations of indigenous microbiota likely will be high. Routine monitoring and testing of the environment by suppliers for Listeria spp. may be deemed appropriate by DOD to assess sanitary conditions and process control. Such testing would depend on factors such as the type of compost used, the water used, the harvesting techniques, the storage and handling conditions, and the intended end use.

Produce: packaged salads and leafy greens—Appendix A, Flow Diagram A.34, Appendix J, Table J.34

Salad greens are expected to have microbiota that can originate from numerous sources such as irrigation water, insects, birds, animals, and postharvest handling and processing. When salad greens are washed, particularly when a chemical such as chlorine is added to the wash water, some microorganisms can be physically washed off; however, the washing process also can contribute to cross-contamination. Packaged salads and leafy greens generally have a limited shelf life. Suppliers can use testing for E. coli to assess process control and sanitary conditions. Environmental testing for Listeria spp. in processing facilities should be conducted to monitor sanitary conditions.

Produce: vegetable sprouts—Appendix A, Flow Diagram A.35, Appendix J, Table J.35

These are sprouted vegetable seeds before true leaves emerge that may be consumed raw or cooked. Routine testing of in-process and finished products by suppliers for E. coli should be done as an indicator of process control and sanitary production. Appropriate testing of spent irrigation water for Salmonella and E. coli O157:H7 should be conducted to assess potential product contamination. Routine environmental monitoring for Listeria spp. also should occur to assess sanitary conditions.

Seafood: non-RTE, raw—Appendix A, Flow Diagrams A.36a-e, Appendix J, Table J.36

Routine microbiological testing of in-process and finished products by suppliers is not recommended for raw (fresh or frozen) finfish or raw crustaceans for either quality or safety. Nonroutine testing of in-process and finished products for coliforms and Salmonella may be done to verify proper sanitation and process control. A visual inspection for parasites is recommended if the product is intended for raw consumption. Alternatively, the supplier may verify that freezing treatments are applied to destroy certain parasites. For scombroid fish species, testing of finished product for histamine is recommended.

Seafood: RTE, fish, cold smoked—Appendix A, Flow Diagram A.37, Appendix J, Table J.37

Suppliers should conduct routine environmental testing for Listeria spp. to demonstrate that production is occurring under sanitary conditions. The supplier also should test in-process and finished products periodically for L. monocytogenes and Salmonella to demonstrate that the product is produced under sanitary conditions. The pH of pickled herring should be verified periodically. Scombroid species may contain histamine, and products made from these species should be tested to verify that proper temperature control was maintained.

Seafood: RTE, cooked or hot smoked—Appendix A, Flow Diagram A.38, Appendix J, Table J.38

The supplier should apply a validated process that results in at least a 6-log reduction of L. monocytogenes. When such a validated process is used, routine sampling of in-process and finished product for S. aureus and the environment for Listeria spp. should occur to verify that controls are in place to prevent recontamination. If required to further demonstrate that production is occurring under process control and sanitary conditions, the supplier could also test in-process and finished products for coliforms, APC organisms, Salmonella, and L. monocytogenes. If it is apparent that there is a potential for recontamination through mechanical or manual handling, testing finished products for Salmonella and L. monocytogenes should be done routinely. Scombroid fish species may contain histamine if temperature abused and fish decompose; finished products should be tested for histamine per FDA's guidance documents (31).

Seafood: RTE, raw molluscan shellfish—Appendix A, Flow Diagram A.39, Appendix J, Table J.39

Suppliers must demonstrate traceability that establishes that the product was harvested from approved waters in the United States or in countries (Canada, Mexico, New Zealand, and South Korea) that have a Memorandum of Understanding with the United States. Under these conditions, no routine microbiological testing of products is necessary by the supplier. Where the supplier is unable to prove the status of the harvest waters or where contamination is suspected, the DOD should not accept the product. Nonroutine in-process and finished-product testing by suppliers on RTE raw molluscan shellfish from approved waters to demonstrate process control and sanitary conditions may include analyses for APC organisms, fecal coliforms, and Vibrio parahaemolyticus (or other Vibrio spp. if warranted). In addition, Vibrio control plans as outlined in the National Shellfish Sanitation Program (29) may be required if conditions warrant.

Spices and herbs, coffee and tea—Appendix A, Flow Diagrams A.40.a, A.40.b, and A.40.c, Appendix J, Table J.40

Harvested spices are expected to have a varied microbiota associated with them, including sporeforming bacteria and fungi. Also, when a dehydration process is performed outdoors there is the potential to acquire additional contamination. Suppliers should test in-process and finished products routinely for APC organisms and Salmonella to assess process controls and sanitary conditions. The suppliers also should routinely test the environment for Salmonella. Nonroutine testing of finished products by suppliers, when deemed necessary, to assess process control and sanitary conditions may include testing for B. cereus (or other toxigenic Bacillus spp.), E. coli, coliforms, molds and yeasts, and E. coli O157:H7 (or other STEC as appropriate).

OTHER INDICATORS OF PROCESS CONTROL AND SANITARY CONDITIONS

There are microbiological by-products, enzymes, products of decomposition (including those detected through visual observation), and other analytes that may reflect lack of process control or insanitary conditions. The following are examples of some of these indicators.

  • Histamine in scombroid fish at high levels indicates possible temperature abuse, lack of sanitary conditions, and decomposition of these fish.

  • The presence of nonmicrobiological alkaline phosphatase in milk is an indication that the milk has been inadequately pasteurized. Under these conditions microbiological pathogens endemic to raw milk may survive and result in milkborne illness.

  • Peroxidase testing is used to indicate that blanching of fresh vegetables has been adequate. Typical blanching temperatures (195 to 205°F [91 to 96°C] for 3 min) would likely be sufficient to provide a lethality step eliminating vegetative pathogens.

  • The presence of aflatoxin or other mycotoxins is indicative of significant growth of molds. The presence of aflatoxin or other mycotoxins may render the food unacceptable for human consumption or for use in further food processing.

  • Gas formation causing swollen product containers would be indicative of spoilage and potential pathogen growth. Similarly, slime formation, visible mold growth, discoloration, and product leakage from a container would be indicative of spoilage or potential growth of pathogens. Changes in product viscosity may be indicative of microbiological proteolysis or starch hydrolysis; such activity may be the result of postprocessing contamination, temperature abuse, or underprocessing.

  • Peroxide values and concentrations of free fatty acids in nuts exceeding tolerance limits would be indicative of poor storage conditions, extended age, or temperature abuse. In such situations, these changes would not indicate microbiological spoilage or growth but rather oxidation that impacts quality.

  • When free fatty acid concentrations in milk exceed tolerances, this is indicative of hydrolytic rancidity associated with poor raw material control and potential postprocess contamination.

  • Any signs of pests or pest infestation indicate contaminated packaging materials, poor storage conditions within a plant or distribution center, or pest contamination within a transport container or at the location of sampling. These products should be considered compromised and unacceptable.

  • Development of acidity (measured by pH or titration) is critical to the safe production of many fermented products such as cheeses and fermented sausages. Fermentation of these products by harmless starter organisms retards or prevents the growth of pathogenic bacteria such as E. coli, Salmonella, and L. monocytogenes. However, in other products acid development is undesirable, e.g., flat sour defect in canned food resulting from undesirable microbiological growth. Undesirable fermentation can result in expression of purge in RTE meat products.

ACKNOWLEDGMENT

Mention of trade names or commercial products in this publication is solely for the purpose of providing specific information and does not imply recommendation or endorsement by the USDA and other participating agencies.

SUPPLEMENTAL MATERIAL

REFERENCES

2
Bradley
,
R. L.
,
K. S.
Houck
, and
M.
Smukowski
.
2013
.
Milk and milk products
,
p
.
1
16
.
In
S.
Doores
,
Y.
Salfinger
, and
M. L.
Tortorello
(
ed.
),
Compendium of methods for microbiological examination of foods, 5th ed
.
American Public Health Association
,
Washington, DC
. doi:.
3
Codex Alimentarius Commission
.
2004
.
General guidelines on sampling
.
CAC/GL 50-2004
. .
4
Dey
,
M.
,
J. A.
Mayo
,
D.
Saville
,
C.
Wolyniak
, and
K. C.
Klontz
.
2013
.
Recalls of foods due to microbiological contamination classified by the U.S. Food and Drug Administration, fiscal years 2003 through 2011
.
J. Food Prot
.
76
:
932
938
.
5
Elliott
,
P. H.
, and
A.
Kataoka
.
2013
.
Canned foods—test for commercial sterility
.
In
S.
Doores
,
Y.
Salfinger
, and
M. L.
Tortorello
(
ed.
),
Compendium of methods for microbiological examination of foods, 5th ed
.
American Public Health Association
,
Washington, DC
. doi:.
6
Food and Agricultural Organization of the United Nations, World Health Organization
.
2013
.
Proposed draft principles and guidelines for the establishment and application of microbiological criteria related to foods (at steps 5/8)
.
Appendix III (p. 43–49). Joint FAO/WHO Food Standards Programme CODEX Alimentarius Commission 36th session, Rome, 1 to 5 July 2013 and Report of the 44th session of the CODEX Committee on Food Hygiene
,
p
.
27
56
.
7
Food Standards Australia New Zealand
.
2012
.
Standard 1.6.1. Microbiological limits for food
.
F2012C00862
.
Available at: http://www.comlaw.gov.au/Details/F2012C00862. Accessed 5 February 2015
.
8
Food Standards Australia New Zealand
.
2014
.
Criteria for Listeria monocytogenes in ready-to-eat foods
. .
9
Consumer Goods Forum
.
2013
.
Global food safety initiative guidance document, 6th ed., version 6.3
.
Global Food Safety Initiative Foundation
,
Consumer Goods Forum, Issy-les-Moulineaux, France
.
10
Hyytiä-Trees
,
E.
,
E.
Skyttä
,
M.
Mokkila
,
A.
Kinnunen
,
M.
Lindström
,
L.
Lähteenmäki
,
R.
Ahvenainen
, and
H.
Korkeala
.
2000
.
Safety evaluation of sous vide–processed products with respect to nonproteolytic Clostridium botulinum by use of challenge studies and predictive microbiological models
.
Appl. Environ. Microbiol
.
66
:
223
229
.
11
Ingham
,
S.
2008
.
Critical limit summary: validating shelf-stability and prevention of Staphylococcus aureus growth on RTE meat products
. .
12
International Commission for the Microbiological Specifications for Foods
.
2002
.
Microorganisms in foods 7: microbiological testing in food safety management, 1st ed
.
Springer
,
New York
.
13
International Commission for the Microbiological Specifications for Foods
.
2011
.
Microorganisms in foods 8: use of data for assessing process control and product acceptance
.
Springer
,
New York
.
14
International Organization for Standardization
.
2005
.
General requirements for the competence of testing and calibration laboratories. ISO 17025
.
International Organization for Standardization
,
Geneva
.
15
International Organization for Standardization
.
2007
.
Guidance on the selection and usage of acceptance sampling systems for inspection of discrete items in lots
.
Part 1. Acceptance sampling
.
International Organization for Standardization
,
Geneva
.
16
Kornacki
,
J. L.
,
J. B.
Gurtler
, and
B. A.
Stawick
.
2013
.
Enterobacteriaceae, coliforms, and Escherichia coli as quality and safety indicators
.
In
S.
Doores
,
Y.
Salfinger
, and
M. L.
Tortorello
(
ed.
),
Compendium of methods for microbiological examination of foods, 5th ed
.
American Public Health Association
.
Washington, DC
. doi:.
17
Powell
,
M. R.
2014
.
Optimal food safety sampling under a budget constraint
.
Risk Anal
.
34
:
93
100
. doi:.
18
Ryser
,
E. T.
, and
E. H.
Marth
.
2007
.
Listeria, listeriosis, and food safety, 3rd ed
.
CRC Press
,
Boca Raton, FL
.
19
Shewhart
,
W. A.
1986
.
Statistical method from the viewpoint of quality control
.
Dover, New York
.
20
Sperber
,
W. H.
, and
North American Millers' Association Microbiology Working Group
.
2007
.
Role of microbiological guidelines in the production and commercial use of milled cereal grains: a practical approach for the 21st century
.
J. Food Prot
.
70
:
1041
1053
.
21
U.S. Department of Agriculture, Food Safety Inspection Service
.
1999
.
Appendix B. Compliance guidelines for cooling heat-treated meat and poultry products (stabilization)
. .
22
U.S. Department of Agriculture, Food Safety Inspection Service
.
2005
.
Risk assessment of the impact of lethality standards on salmonellosis from ready-to-eat meat and poultry products
. .
23
U.S. Department of Agriculture, Food Safety Inspection Service
.
2014
.
FSIS compliance guideline for meat and poultry jerky produced by small and very small establishments
.
2014 Compliance guideline
. .
24
U.S. Department of Defense
.
2013
.
Department of Defense food safety and quality assurance laboratory action levels, appendix O
.
In
Worldwide directory of sanitarily approved food establishments for Armed Forces procurement, USAPHC circular 40-1
.
U.S. Department of Defense
,
Washington, DC
.
25
U.S. Food and Drug Administration
.
2004
.
Guidance for industry: juice HACCP hazards and controls guidance, 1st ed
. .
26
U.S. Food and Drug Administration
.
2005
.
CPG sec.510.150
.
Apple juice
,
apple juice concentrates, and apple juice products—adulteration with patulin
. .
27
U.S. Food and Drug Administration
.
2009
.
Prevention of Salmonella Enteritidis in shell eggs during production, storage, and transportation: final rule. 21 CFR parts 16 and 118
.
Fed. Regist
.
74
(
130
):
33029
33101
.
28
U.S. Food and Drug Administration
.
2010
.
Compliance policy guide sec. 527.300. Dairy products—microbial contaminants and alkaline phosphatase activity
. .
29
U.S. Food and Drug Administration
.
2013
.
National shellfish sanitation program (NSSP)
. .
30
U.S. Food and Drug Administration
.
2013
.
Food Code 2013
.
31
U.S. Food and Drug Administration
.
2014
.
Fish and fishery products hazards and controls guidance, 4th ed
. .
32
U.S. Food and Drug Administration
.
2015
.
Grade “A” pasteurized milk ordinance (includes provisions from the grade “A” condensed and dry milk products and condensed and dry whey—supplement I to the grade “A” PMO)
. .
33
U.S. Food and Drug Administration
.
2017
.
Current good manufacturing practice in manufacturing, packing, or holding human food
.
21 CFR 110
.
U.S. Food and Drug Administration
,
Silver Spring, MD
.
34
Wehr
,
H. M.
, and
J. F.
Frank
.
2004
.
Standard methods for the examination of dairy products, 17th ed
.
American Public Health Association
,
Washington, DC
.
35
World Health Organization
.
2013
.
Water sanitation health—drinking water quality
. .
1

The terms insanitary and unsanitary are considered as one and the same in this document. Insanitary is a word that has been used in regulatory language. In this document insanitary is used as this term was provided in the charge to the NACMCF Committee.

Competing Interests

Participating agencies include the U.S. Department of Agriculture, Food Safety and Inspection Service; U.S. Department of Health and Human Services, Food and Drug Administration and Centers for Disease Control and Prevention; U.S. Department of Commerce, National Marine Fisheries Service; and U.S. Department of Defense, Veterinary Service Activity. This article may be reproduced without prior permission.

Supplementary data