Radiation Dose Audit Failures: Truth and Consequences

The validation of terminal sterilization using radiation involves the establishment of an initial sterilization dose as well as routine maintenance of that sterilization dose. The procedures for routine maintenance, typically carried out by performing bioburden and periodic dose audits, are outlined in the ANSI/AAMI/ISO 11137 series of standards^1,2  as well as ANSI/AAMI/ISO TIR13004.3 The methods defined in these documents have been successfully used for several decades to determine and maintain sterilization doses. These documents provide guidelines for recommended actions based on the number of positive tests of sterility (TOS) from a dose audit. Although some wording and instructions have been modified slightly over the years to provide additional clarification, the general actions to be taken have remained the same throughout the revision history of the standards.

The terms implied throughout the radiation sterilization documents for the TOS outcomes of the verification experiment performed during a dose audit are “acceptable verification” and “unacceptable verification,” although one clause is titled “Failure of a sterilization dose audit.”^2,3  In the strictest sense, the term “failure” should not apply to the initial positives in a TOS but should only apply when those positives are determined to be true survivors of the verification dose and exceed the acceptable limits described in the standards. However, the terms widely used in the healthcare product industry are “pass” and “fail” of the dose audit based on positive TOS results. For this discussion, and to be more in keeping with industry understanding, the industry-accepted terms of “pass” and “fail” will be used here, as well as the terms “acceptable” and “unacceptable.”

When failure of a dose audit occurs, many companies typically assume the TOS positives are indicative of a catastrophic failure. The natural inclination is to assume the dose audit is directly evaluating the routinely sterilized product and, consequently, to assume a failed dose audit means that the fully processed product has also “failed” or is nonsterile. The belief that dose audit positives or failures mean fully sterilized product is nonsterile is absolutely not true. This article will outline the truth of what a dose audit failure means in terms of the sterility and sterility assurance of the product. In addition, it will explore the consequences of a dose audit failure on the sterilized product associated with it.

The dose audit TOS results are reported as the number of positives in a set of samples, typically 100 samples for Method 1 and Method 2 and 10 samples for VD_max—the most widely used dose determination methods. The number of TOS positives determines the action to be taken. In all radiation sterilization dose audits, one positive TOS for 10 samples and 2 positive TOS for 100 samples are acceptable, and these results confirm the validity of the sterilization dose for the specified sterility assurance level (SAL). Although a positive TOS is acceptable and expected, sometimes any TOS positive will be wrongly viewed as unacceptable or in a negative context. This is tied to the misconception that the verification TOS positive is somehow directly reflective of the sterility of fully sterilized product. It is also tied to the misconception that the verification test of sterility is the same as a lot release test for sterility. The verification dose is intended to provide a 10⁻¹ to 10⁻² SAL (sometimes referred to as “sublethal”), and therefore one or two positives in the set of samples is considered a statistically acceptable outcome of the experiment and should never be viewed as an indication of a problem with the product.

When the number of TOS positives exceeds the specified level for acceptability, different actions are to be taken, depending on the number of positives. For Methods 1 and 2, a repeat test is called for, based on the number of initial TOS positives. The initial test in this case is not considered a dose audit failure and might only indicate that the results fall slightly outside statistical acceptability, with the repeat test potentially confirming that fact. When using VD_max, the repeat test is called a “confirmatory test.” (Note that a VD_max document currently under development will not include the “confirmatory test” term). Again, the initial test in itself is not considered a dose audit failure, and, as with Methods 1 and 2, might only indicate that the results fall slightly outside statistical acceptability. The outcome of the confirmatory test will determine whether the dose audit should be considered passing or failing. These two situations outlined for Method 1, Method 2 and VD_max will be referred to as “repeat/confirmatory” tests in the remainder of this article. In the case of a repeat/confirmatory test, the additional results will confirm whether the initial test was a statistical expectation, and whether the sterilization dose continues to be valid. Hence, positive TOS results that call for repeat/confirmatory tests are not “failures” and should not be referred to as such.

Apart from this scenario of a repeat/confirmatory test, a certain number of TOS positives in the initial dose audit is considered failing and will call for augmentation of the sterilization dose (increase in sterilization dose to address the number of positives observed) or cessation of sterilization for products impacted, while corrective action is taken and the sterilization dose is reestablished. Each dose determination method specifies those criteria and the corresponding actions. For this discussion, only those outcomes that indicate a confirmed dose audit failure will be addressed—not outcomes prior to completion of a repeat/confirmatory test, or outcomes of an acceptable repeat/confirmatory test.

It should be noted that the truths and the consequences to follow are not relevant for a situation where there is a gross dose audit failure due to an extremely high or out-of-control bioburden, or an excessive number of TOS positives (i.e., a number beyond that which calls for augmentation). In these situations, the principles presented here do not apply and, based on the actual failure situation, there might be a significant impact to the sterility of the product because of total loss of control of the manufacturing process.

The first truth about dose audit failures is that there should not be an immediate assumption that fully sterilized product is nonsterile or not safe for use simply because there are TOS positives after exposure to the verification dose. The incorrect assumption by many companies is that a failing number of TOS positives means that product that has been fully processed is nonsterile. This is incorrect, in that the TOS positives were based on irradiation at a verification dose, which is a much lower dose than that used for sterilization and realistically is expected to result in one or more TOS positives (e.g., one positive out of ten tested demonstrates a 10⁻¹ SAL). The verification dose is set for the purpose of experimenting at a 10⁻¹ or 10⁻² SAL, not at the 10⁻⁶ or other SALs used for full sterilization. Therefore, verification dose positives arise from a 10⁻¹ to 10⁻² SAL dose that statistically can result in positives, and not from a full sterilization dose where the expectation of a positive is, for example, one in one million (i.e., 10⁻⁶ SAL).

One cannot compare the safety of product that has been irradiated at a one in one million probability of a viable microorganism to that of product irradiated at a one in ten or one in one hundred probability of a viable microorganism. The failed dose audit might indicate that the calculated SAL of the fully processed product is not exactly at 10⁻⁶, but, if calculations are performed based on D-values, it is most often shown that the SAL is only slightly different than that claimed. Therefore, even if the SAL of the sterilized product is slightly different than a claimed SAL of 10–6, the SAL might be only something such as 10^−5.8, a probability of a viable microorganisms of about one in six hundred fifty thousand, which is certainly not a level that can be called nonsterile as far as patient safety is concerned.4 In ANSI/AAMI ST67, there are SALs other than 10⁻⁶ that are recognized as “sterile,” such as 10⁻³, 10⁻⁴ and 10⁻⁵, which are acceptable based on certain criteria.⁵  Because of this, the issue is not whether the fully processed product is sterile—it is sterile as far as patient safety is concerned. The primary concern relates to a compliance perspective in demonstrating the label claim, as explained further.

An important truth about a dose audit failure is that there should not be an automatic assumption that all TOS positives are true survivors of the verification dose. It is also possible that a TOS positive is the result of postprocessing contamination. Postprocessing contamination is a significant factor in the potential for a sterility test positive, and the aspects that can contribute to this factor need to be recognized.

The probability of lab contamination in a sterility test—resulting in false positives—is a valid concern.⁶  In published information, the false positive rate varies, but can be as high as 0.5%, depending on the level of environmental control and the competency of the lab. Issues such as product design, fatigue, the test environment, manipulation, materials, technician error, and incubation conditions all come into play in evaluating the likelihood of contamination. Labs that are not set up for strict aseptic practices for sterility testing, such as the sterility test isolator practices used in the pharmaceutical industry, will likely have higher contamination rates. This expected laboratory contamination rate is one reason sterility testing of terminally sterilized product is not recommended.⁶  There is even a likelihood that contamination will occur during the incubation process, where container integrity may be an issue during incubation and sample examination.

Postprocessing contamination can include more than just lab contamination—it can be essentially anything that happens to the test samples from the time the product exits the irradiation process all the way through the final examination of containers in the testing process. Of utmost importance is the package that maintains the sterility of the samples. A breach in the package integrity—either a pre-existing issue or unknown/undetected damage—could also negate the results of the TOS, especially considering the handling and stresses involved in shipping (e.g., samples are frequently packaged in material other than what is validated for the finished product).

Another truth about dose audit failures is the statistical probability that one in approximately 11 or 12 dose audits will have results that fall outside the acceptable number of TOS positives, even if the bioburden has not changed.⁷  Dose determination is based on statistical probabilities, as previously discussed, such as the likelihood of survival of one in one million viable microorganisms. Based on the probabilities of occurrence of numbers of positives presented in the 11137 series dose determination methods, there is around a 92% probability that there will be the acceptable number of positives, which leaves about an 8% probability that the number of TOS positives will not be acceptable, according to the interpretation criteria. This is the basis for the repeat or confirmatory options in dose audits.

In real life terms, for quarterly dose audits, this probability of unacceptable results can happen at any time during the course of performing dose audits—not only after the 11th or 12th dose audit. This is one reason why the repeat/confirmatory option is specified—to demonstrate whether or not the failure was within the probability of occurrence. If this truth is known and understood, it should prevent a company from jumping to the wrong conclusion about the meaning of unacceptable results. For this truth, the assumed failure should be assessed in light of it potentially being a statistical probability, and therefore actions should be focused on determining this and demonstrating that the product is in a state of control, rather than only taking remedial actions for a problem that might not exist.

The next truth about a dose audit failure is that, where there is a possibility of not achieving the specific SAL claimed, that possibility only pertains to one small portion of a sterilization load. For this discussion we will assume a 10⁻⁶ SAL. In all radiation sterilization a dose range will be delivered, which is typically the sterilization dose plus, for example, 10–20 kGy, such as 20–32 kGy or 25–40 kGy. A dose range is specified because, due to the physics of the irradiation process, the exact dose cannot be delivered throughout the product itself or the product configuration. The selected SAL, such as 10⁻⁶, corresponds to the sterilization (lowest) dose, whereas all the higher doses in the dose range will translate into SALs exceeding the SAL for the sterilization dose. Therefore, the vast majority of a given sterilization load will possess SALs ranging from 10⁻⁶ to 10⁻⁹ or 10⁻¹⁰. This fact demonstrates the truth that the vast majority of a sterilization load will have received the intended SAL of 10⁻⁶, because a slight change in SALs of 10⁻⁷, 10⁻⁸, or 10⁻⁹ will not approach that of 10⁻⁶. This is the reason that the majority of a sterilization load will achieve the designated SAL, leaving only a portion of the load—in the minimum dose zones—where the SAL might have been affected.⁸ 

A final truth is that, in many cases, the minimum sterilization dose delivered to the product load is higher than the sterilization dose that was determined (validated or substantiated). For instance, if a 25 kGy sterilization dose is substantiated, the routine sterilization process might show that previous loads received minimum doses higher than 25 kGy, such as 26 or 27 kGy. In these cases, for a dose audit failure, the truth is that there is potentially no impact to achieving the corresponding 10⁻⁶ SAL, because the minimum dose of 25 kGy was always exceeded, and therefore it could be demonstrated by calculations that the 10⁻⁶ SAL was always achieved.

Calculations for D-value and SAL based on bioburden and radiation dose can show that, for example, 25.5 kGy provides a 10^−6.3 SAL, in which case a slight change in that SAL due to a true dose audit failure still would mean the SAL of 10⁻⁶ was achieved. A series of calculations can be performed for each load to definitively show what the theoretical SAL is, based on bioburden data coupled with the minimum delivered dose for each load. Depending on the circumstances, it is often appropriate to specify a minimum dose to the sterilization site that exceeds the validated minimum dose. If it is possible to specify the dose range for sterilization as a minimum dose slightly above the dose that was determined (e.g. 26–40 kGy versus 25–40 kGy), this could become the critical factor in defense of the nonimpact of a dose audit failure.

Understanding the truth about dose audit failures is critical to understanding the consequences of a dose audit failure. As previously explained, a dose audit failure is not automatically an indication of nonsterile product in current or previous sterilization runs. Nor is a dose audit failure automatically a recall situation. The consequences of a dose audit failure depend entirely on a) the magnitude of the failure, b) the increase or change in bioburden numbers and types, and c) the actual sterilization (minimum) dose that has been delivered to the processed product.

In facing a dose audit failure, the first and foremost question to address that will dictate consequences is: Is it truly a failure or is there some other reason for the TOS positives? The likelihood of the TOS being invalid is often higher than typically assumed, and it can be attributed to many things—manufacturing, handling, packaging, shipping, or testing.⁹  Before taking dose audit failure actions, one should always determine whether the failure has a reason for invalidation.

There are three central questions to ask while conducting an investigation into a dose audit failure:

1. Is this the result of a statistical expectation, in which a retest/confirmatory test is performed?

2. Is this the result of an invalid experiment (nonrepresentative samples, incorrect dose delivery, contaminated media, breach of test container integrity, etc.) where the test experiment should be invalidated and a new experiment performed?

3. Is this a true failure due to a change in product bioburden, where augmentation or reestablishment of the sterilization dose is required?

An initial step in the investigation requires identification of the positive microorganisms from the TOS, which can go a long way in concluding whether there is a true failure. For example, a microorganism with very low resistance to radiation (e.g., Staphylococcus sp.) should be questioned more than a microorganism known to have a higher resistance to radiation (e.g., Bacillus sp.). Concurrently there should be an investigation into the laboratory, manufacturing and packaging processes, sterilization, and postprocess handling. Improper manufacturing of samples (components, handling, or packaging related only to the verification samples) can cause a dose audit failure that is related only to the samples made for the dose audit. Packaging can be compromised—either during the process or after—and this can lead to contamination of the samples after irradiation at the verification dose. For products that promote growth, a delay in irradiation of the samples that extends the irradiation time of routine product could result in continued growth of microorganisms in the samples that typically would not be present, based on the standard time to irradiation. Each of these deficiencies might only be related to the dose audit samples and not necessarily the routine product. Contamination can occur during test preparation, execution, and incubation. The test of sterility is not infallible, and the occurrence of contamination should always be considered.

The information gathered from the investigation will lead to one of two actions: 1) the performance of a new verification test due to invalidation of the TOS based on an identified root cause or 2) the pursuit of actions dictated by the number and identification of TOS positives, assuming they are true survivors of the verification dose. The first action does not qualify as a consequence because it is not a dose audit failure at this point. The second action indicates a true dose audit failure has occurred; therefore, several consequences might apply.

The following consequences apply only to true dose audit failures for which a preliminary investigation rules out any cause for invalidation.

For a true dose audit failure, the first consequence one must consider is the sterilization dose going forward. The 11137 and TIR13004 guidelines indicate whether the sterilization dose should be augmented and by how much, or whether sterilization must be halted and a new dose reestablished. The radiation guidelines for how to proceed in this case are straightforward. For augmentation, the calculations are outlined based on the number of TOS positives, and the augmented dose is to be continued until either the underlying issue is resolved or the dose is reestablished. For results that indicate cessation of sterilization, dose reestablishment must be pursued immediately for sterilization to resume.

In addition to the sterilization dose going forward, another consequence is the sterilization of batches prior to the dose audit failure. The 11137-2 guidelines specify that one must consider the validity of the sterilization dose in retrospect. In essence, this means considering the validity of the sterilization dose for the product that was sterilized since the previous passing dose audit. The guidelines state that “...the effect of processing product at the sterilization dose that has failed sterilization dose audit on the achievement of the specified SAL for previously processed batches of product shall be considered and a risk assessment undertaken on their suitability for use.”²  Depending on the data available, such as minimum delivered doses, bioburden determinations, and dose mapping, the consideration and the risk assessment might be simple.

In performing a risk assessment, one would initially document the product batches under review—as well as the minimum dose delivered to these batches during sterilization—and compare these data with the calculated augmentation dose, where appropriate. Many of the batches may have already met the augmented dose, depending on the target dose determined by the sterilizer and the actual delivered dose reported. For example, a product might have a sterilization dose of 15 kGy, with a minimum specified dose of 15.5 kGy for irradiation, and an actual delivered minimum dose of 15.7 kGy for all loads under review. For loads that indicate an augmented dose is required, it may be determined upon review that the augmented dose was already achieved in routine processing, because the calculated augmented dose was actually achieved for all batches processed at the minimum delivered dose. Where the augmented dose was not achieved for certain loads, a calculation of the theoretical SAL might be appropriate to determine how different the calculated SAL is from the designated SAL.

Once a theoretical SAL range has been determined for each batch of product using minimum dose delivered and maximum dose delivered, then the percentage of product at the minimum dose, intermediate doses, and maximum dose can be evaluated through the product dose mapping performed during performance qualification. Armed with this information, a company can then assess if any product is at risk and how significant that risk might be, based on the calculated SAL for minimum dose locations, as well as the percent of product that might not meet the SAL claim, if applicable. In these calculations, if the bioburden has changed, the D-value of the population and the calculation of SAL as indicated above might not be appropriate. However, a change in bioburden would naturally be assessed, initially, as a critical factor in the overall investigation.

There will be cases where sterilized product might be augmented, as specified in the 11137 series and TIR13004. A company must assess whether this is a possibility based on several factors, such as time elapsed since sterilization and whether the additional dose will result in exceeding the maximum dose for the product. Based on the findings laid out above and product use, a company should assess the risk to patient and determine what, if any, action is warranted. The considerations discussed should be undertaken with technical experts either within the company or contracted when determining the path forward.

A final consequence that must be considered for a true dose audit failure is the impact to other product in the family. The ISO 11137-2 standard states that in the event of a dose audit failure “…all members of that family shall be considered to be affected.”^2,3  Therefore, actions taken for the dose audit failure—augmentation, dose reestablishment, or cessation of sterilization—will have to apply to all members of the family. Considering the many variables that could apply to a family, it would be impossible to cover all potential approaches here. Suffice it to say that there are several options for separating out certain family members or sub-groups and confirming the validity of a sterilization dose for those members or sub-groups apart from the product in the dose audit failure. The same level of scrutiny and assessment must be applied to all members of the product family. Additionally, if a root cause is determined that could be systemic, assessment across product families might be warranted.

Table 1 is a summary of the preceding sections concerning truths and consequences of dose audit failures.

In conclusion, for a confirmed dose audit failure there are several truths and several consequences. The truths are that: a) product that is processed at the full sterilization dose is sterile as far as it concerns patient safety—an individual product in a portion of the load might simply not possess exactly the designated SAL; b) there is a potential that the TOS positives are not true survivors of the verification dose; c) only a small portion of a sterilization load might be affected by the question of SAL, because the load always receives a dose range; d) there is a statistical probability that there will be a dose audit failure over time; and e) the actual delivered dose for sterilized product—versus the validated sterilization dose—might show that the product does possess the designated SAL.

The real consequences of a confirmed dose audit failure are that a) the sterilization dose going forward must be considered, augmented if required, and guidelines for reestablishment followed; b) the sterilization dose in retrospect must be considered and a risk analysis performed; and c) the impact to and subsequent action for other product in a family must be considered.

The overarching fact is that having positives in a dose audit test of sterility does not automatically mean product that has received the full sterilization dose is nonsterile or unsafe for use. In many cases, those sterilized products possess an acceptable SAL where patient safety is concerned.