Correlation between Environmental Monitoring and Product Bioburden

Environmental monitoring (EM) in modern-day medical device and pharmaceutical manufacturing is used to demonstrate a state of control over the manufacturing environment. Fred B. Kilmer was one of the first individuals to bring to light the criticality of controlling the environment during the production of gauze, a steam-sterilized product.1 Kilmer's research and practices were considered the gold standard and changed the manufacturing landscape for all types of products required to be free from microorganisms.

As the importance of EM was further understood throughout the 1900s, organizations that developed industry standards documented these best practices for consistent use throughout the pharmaceutical and medical device industries. The current-day standard ISO 14644-1:2015 addresses environmental control by limiting the number of airborne particulates allowed in a manufacturing environment. The standard states: “Cleanroom and associated controlled environments provide for the control of airborne particulate contamination to levels appropriate for accomplishing contamination-sensitive activities.”2 These levels are dictated as classifications and require airflow volume (velocity) and air pressure differences to be measured each year. This is consistent with Kilmer's thoughts on the source of microorganisms and how they are “readily disseminated through the air by the medium of dust. The air of a crowded room is always laden with bacterial life.”1 

Cleanroom classification is a general way to categorize the environmental conditions established for a manufacturing environment; however, data collection for the classification occurs only a few times per year. To address this gap, guidance documents and standards provide additional EM program requirements that help to demonstrate the control of the manufacturing process in relation to sterility assurance.

Regulatory bodies also recognized the benefit of these standards and began requiring manufacturers to implement environmental control within their manufacturing processes. The Food and Drug Administration requires that medical device and pharmaceutical companies that market products within the United States must comply with 21 CFR 820.70(c) (environmental control), which states that manufacturers must “establish and maintain procedures to adequately control” environmental conditions that “could reasonably be expected to have an adverse effect on product quality.”4 The position of the regulatory bodies and industry for EM can be understood through a number of standards and guidance documents (Table 1).

These standards and guidance documents are intended to reduce the risk to patients by requiring additional testing to demonstrate that the manufacturing environment is maintained in a state of control (e.g., testing for viable microorganisms [e.g., in the air or on surfaces, gowns, gloves/hands, door knobs], compressed air particulates, water quality testing). The data should be used to identify potential sources of microbiological risk to the product from the manufacturing environment and not as a routine measurement for product quality. Alert and action levels typically are statistically based, and when out-of-specification (OOS) results are observed, an investigation is launched to determine potential product impact.

Data generated from EM is indirect evidence that the finished device quality could be compromised and not direct evidence that the quality has been compromised.

The standards and guidance documents listed in Table 1 provide manufacturers with instructions for when to conduct OOS investigations and assess microbiological risk to products. However, the execution of the investigation is at the discretion of the manufacturer, and the value of the investigation outcome is dependent on the quality of the program and the technical experience of the individuals analyzing the data.

AAMI TIR52:2014/(R)2017 states that “an excursion (in well controlled processes) will rarely be directly correlated with an increase in the product bioburden,”9 leading manufacturers to question how an indirect measure of product quality (EM) and a direct measurement of product quality could be related. The investigation described in this article sought to explore that relationship.

In this study, five years of viable EM and product bioburden data for medical devices were analyzed using the assumptions described in Table 2. The dataset included 50 terminally sterilized device families produced in four controlled environments from two manufacturing facilities. Each data point for the product bioburden contained at least 20 data points (10 for aerobic and 10 for fungal plate results).

The EM sampling strategy and product families for bioburden evaluation remained consistent over the course of the data collection period. A linear regression model (α = 0.05) was used (Minitab 17; Minitab, State College, PA) to demonstrate a significant correlation between EM and product bioburden and to reject the null hypothesis of “no correlation between product bioburden and viable EM.” To reject the null hypothesis, the following acceptance criteria were used: P < 0.05, R² > 0.7 or 70%.

Graphs (Figures 1–3) were generated using the individual bioburden results for each device sampled and the maximum EM value for the same sampling time. The maximum EM value is represented by the solid red line, and the individual uncorrected bioburden results are depicted by open circles. The different color circles represent the various product families.

Facility 1 was used to manufacture 21 product families in two cleanrooms. The 20 quarters of bioburden data and EM data were graphed in Figure 1 and evaluated for statistical correlation. (Note: Facilities and products within this subset of data were validated to current industry standards and continued during the sampling frame to operate within a state of control. All product bioburden and EM data spikes were investigated independently via an OOS investigation and corrective action for root cause was identified and implemented successfully.)

This sample size of 3,595 bioburden data points yielded an α of 0.05, showing 5% probability of rejecting the null hypothesis as true. Whereas our null hypothesis was “no correlation between product bioburden and viable EM,” with a P value of 0.128, R2 value of 0.001, and negative slope, evidence was insufficient to demonstrate a linear relationship between product bioburden and EM data (Figure 1).

Facility 2, cleanroom I was used to manufacture 22 product families. The 20 quarters of total product bioburden and sample size of 2,783 data points yielded an α of 0.05, P value of 0.35, R² value of 0, and positive slope, thereby resulting in insufficient evidence to demonstrate a linear relationship between product bioburden and EM (Figure 2). Therefore, the null hypothesis for this cleanroom also was unable to be rejected.

Facility 2, cleanroom II was used to manufacture seven product families. The total bioburden sample size for 20 quarters of 1,079 points yielded an α of 0.05, P value of 0.007, R² value of 0.007, and negative slope (Figure 3). Although the P value suggested that one can reject the null hypothesis for this cleanroom, the linear regression did not demonstrate a strong relationship between bioburden and EM.

A best-fit line of EM plotted against product bioburden did not show the presence of a linear relationship, despite the P value. The slope for this relationship presented as a negative relationship, indicating that as EM increases, bioburden decreases (Figure 3).

In addition, the equation for the regression would not have predicted any time points at which the product bioburden would have been above the 2σ and 3σ level. Therefore, this relationship cannot be practically used to demonstrate a failure to control or meet the microbiological quality of the product.

Further analysis of the linear regression models shown in Figures 1–3 confirmed that evidence was insufficient to demonstrate a relationship—and that it may not be possible to do so.

Analysis of the linear regression model demonstrated that EM did not provide enough sensitivity to accurately predict large changes in product bioburden. In addition, the trends seen in Figure 2 are contrary to the hypothesis that EM is correlated with product bioburden.

The data also were analyzed to understand whether predictive trends could be visualized. Therefore, even if statistical correlation using a linear line could not be established, other means of data analysis could provide useful tools for the manufacturer. The data in Figure 1 provided circumstances in which EM and bioburden spikes occurred during the time period evaluated. The graph failed to show product bioburden increasing as EM increased, even on delayed timing. It appeared that as the overall EM level decreased, the variation in product bioburden increased for an entire room.

Viable EM and product bioburden programs are designed to assess the potential microbial impact from the manufacturing process on a product. However, for terminally sterilized products, product bioburden monitoring is a more sensitive test when determining the microbiological quality of the product, as it is a direct evaluation of how the manufacturing inputs, including the manufacturing environment, have affected the product.

Sampling plans currently required by standards for EM programs may not provide enough information to monitor all potential risk points affecting product microbiological quality and, therefore, cannot be considered independently as indicators of product microbiological quality. According to TIR52, “Sampling plans should be tailored to the level of control required to protect the process and the product. The mode of sterilization might influence this process.”9 If historical EM data are statistically improbable of monitoring the microbiological impact for the product, the formal program design should be reevaluated to take into account potential sources that could affect microbiological quality.

When developing sampling plans for EM programs, a product risk assessment can help manufacturers remain aware of the manufacturing inputs that may affect the microbiological quality of the final product. The manufacturing environment is only one input, and although it may not show a direct correlation, as discussed in this study, it should be used as an indicator that the manufacturing environment remains in control.

When the microbiological quality of the final product is compromised, the combined evaluation of the EM program (e.g., viable air/nonviable air, water quality, compressed gases), raw material screening, and product bioburden can provide manufacturers with evidence to truly understand root causes and implement a meaningful corrective action.

As manufacturers prepare for manufacturing, it is important that they remain agile and able to react to trends. Testing the product at the end of the process (e.g., bioburden) restricts the manufacturer's ability to react. Instead, manufacturers should move to an end-to-end philosophy where in-process EM and product input evaluation allows them to make real-time adjustments and to demonstrate that their manufacturing environment and process are controlled. Doing so will allow manufacturers to consistently produce microbiologically safe products.

The data analyses yielded insufficient evidence to reject the null hypothesis, resulting in a low statistical probability for the existence of a correlation between EM and product bioburden with this dataset.