Regulatory Wrap: Reducing Time to Market and Promoting Innovation while Ensuring Device Safety

Per the International Organization for Standardization (ISO) standard on medical device risk management, ISO 14971:2007,2 “Safety is freedom from unacceptable risk.” ISO 14971:20123 further adds that (per EU Directive 93/42/EEC4) all reasonably foreseeable risks should be reduced as far as practicable, disclosed properly, and have benefit-risk analyses performed to ensure benefits outweigh risks before concluding residual risks are acceptable.

Europe's new Medical Device Regulation (effective in 2020) states (in Annex I, General Safety and Performance) the following: “All known and foreseeable risks, and any undesirable side-effects, shall be minimized and be acceptable when weighed against the evaluated benefits to the patient and/or user arising from the achieved performance of the device during normal conditions of use.”5 

U.S. regulation 21 CFR 860.7 states, “There is reasonable assurance that a device is safe when it can be determined, based upon valid scientific evidence, that the probable benefits to health from use of the device for its intended uses and conditions of use, when accompanied by adequate directions and warnings against unsafe use, outweigh any probable risks.” The regulation also identifies four relevant factors that should be considered when assessing safety and effectiveness: “(1) The persons for whose use the device is represented or intended; (2) The conditions of use for the device, including conditions of use prescribed, recommended, or suggested in the labeling or advertising of the device, and other intended conditions of use; (3) The probable benefit to health from the use of the device weighed against any probable injury or illness from such use; and (4) The reliability of the device.”6 

For the general public, it would be reasonable to expect that a safe medical device should not cause unnecessary, uninformed, or intolerable safety issues (relative to its benefits) while providing the functions and health benefits as claimed. Having said that, a number of important contextual considerations exist:

1. From an engineering perspective, risk identification, control, and assessment need to be performed within the context of a defined device and use environment. This includes, but is not limited to, adequate documentation of the device's intended use, use conditions, design, and proof of the verified and validated physical device, as described and specified (i.e., the final device is verified and validated). For example, we cannot conclude all risks associated with a device are adequately identified if we don't have adequate information on what the device is, including what it is used for, how it is designed, and how it is used. This contextual information is equally important for evaluating the benefits of using the device.

2. Both the benefits and risks of a medical device have some inherent level of uncertainty7 and are relative to other existing benefits and risks (e.g., existing practice or technologies). Risk is a combination of the probability and the severity of a potential harm. Probability in this framework means the probability of the harm (Ph) arising from the use of a device (from the device user population) during its intended use life and for its intended use and use conditions. Per the example in the informative section of 14971, Annex E, Ph is composed of P1 (the probability of hazardous situation) and P2 (the probability of the hazardous situation leading to the harm) (Figure 1). As described by Wu and Kusinitz,8 harm severity and P2 should be assessed by people with clinical/medical background, while P1 may be assessed by multidisciplinary engineers. Benefits should be assessed based on the magnitude of the benefits, as well as the probability and duration of the benefits.9 The probability of the benefits should take into consideration both the device's clinical effectiveness and its capability to reliably perform the functions as needed.

3. Reliability also is an important component of safety and effectiveness (e.g., in life-supporting or -sustaining devices).10 Traditional reliability is the probability that the device will function as specified, under specified use conditions, and for a specified period of time. Reliability plays an important role in assessing P1 because the assessment of P1 for a hazardous situation is technically the assessment of the failure rate of a hazardous situation (e.g., a life-supporting function failure rate). The assessment of P1 is practiced without strictly following the reliability disciplines. For example, when we rank the likelihood of a hazardous situation (e.g., incorrect dose by a drug delivery device), we use scales (such as “unlikely” or “impossible”) instead of specific probability numbers with confidence levels; thus the overall probability for a hazardous situation (e.g., incorrect delivery) typically is estimated pragmatically rather than logically calculated or directly demonstrated by reliability testing results. It is understandable that a reliability estimate, such as quantitative risk assessment using probabilities, may not always be accurate due to the lack of valid methods for assessing the probabilities of certain events or issues related to human factors or software. Nevertheless, reliability engineering disciplines should be applied to the extent that is reasonably practical. In contrast, the formal reliability methods for evaluating essential performance (defined per IEC 60601-1:2005, section 3.27, as “Performance of a clinical function, other than that related to Basic Safety, where loss or degradation beyond the limits specified by the manufacturer results in an unacceptable Risk”11), should be strictly utilized in order to have adequate confidence of a device's safety and effectiveness in providing the clinical functions for achieving the claimed benefits. For example, at minimum, a fault tree analysis should be performed for each essential performance function, and associated testing should consider all possible use and environmental conditions, including the potential worst cases during the device use life. Of important note, the identification of a device's essential performance requirements (including the evaluation of “unacceptable risk”) should be performed before and without accounting for any risk mitigation.

4. The criterion for benefits outweighing risks implies that the device provides the benefits as claimed and reasonable assurance exists that the risks are adequately identified and assessed. If the device does not provide any benefits, then any risks introduced would be unacceptable. If the risks are not adequately identified and assessed, then the claim that the benefits outweigh the risks may not be justified.12 On the other hand, if a device is functionally and clinically effective, and all risks are acceptably controlled, then it would be reasonable to conclude that the device's benefits outweigh residual risks. If risks are not controlled to acceptable levels, then a benefit-risk analysis plays a key role in determining and justifying the risk acceptability. However, even in this case, the justification should include the assurance that risks are reduced as far as possible (the new EU MDR clarifies it as “without adversely affecting the benefit-risk ratio”).

All medical devices have risks, and absolute safety does not exist. Safety and risk acceptability for medical devices are relative concepts. For example, for certain small patient populations, the acceptable risk level may be different from the general public when weighing the benefits of a device and the patients' health conditions. Therefore, justification for risk acceptability (i.e., safety) is an essential part of the safety reasoning.

To summarize, safety means that, for a given medical device with a defined intended use (including intended patient population) and use specification, which is both verified and validated, reasonable assurance exists that possible risks associated with the device are adequately identified, disclosed, and acceptably controlled when weighed against the evaluated benefits. This includes the consideration that safety-critical clinical functions are adequately defined with adequate reliability and that the health benefits of using the device outweigh the risks.

Clarifying what a “safe medical device” means allows for a foundation to assess and ensure safety. Given the complexity of today's medical devices, their use environments, and involvement of multidisciplinary engineering sciences, it is almost impossible to verify safety as a whole. To tackle this issue, we need to take a risk-based approach and apply critical thinking to break down the general safety expectations into subelements, to the point that the subelements are specific enough that they can be satisfied with valid scientific evidence.

General safety expectations can be broken down into more specific criteria via three approaches. (1) With a deterministic (deductive) approach, the breakdown can be established based on logic. This is similar to a typical fault tree analysis, which is a top-down, deductive failure analysis based on logical relations of low-level events. (2) With a probabilistic (inductive) approach, the breakdown can be justified from the probabilities. For example, an acceptable risk is justified by the probability of the harm occurring being extremely low. (3) With a qualitative (abductive or defeasible) approach, the breakdown is the best explanation based on best knowledge and, therefore, is presumed to be sufficient until there is cause not to believe it. For example, adequate design controls can be broken down into adequate design and development planning, adequate design input, adequate design output, adequate design review, adequate design verification, adequate design validation, adequate design transfer, adequate design changes, and adequate design history file. Table 1 illustrates an example of how general safety criteria can be broken down by applying these approaches jointly.

As a regulated industry, medical device development creates a large amount of data as part of the industry's compliance efforts. These data are suitable as evidence of compliance for quality management systems but are not necessarily suitable or sufficient to justify safety. Furthermore, it is important to recognize that there are process (e.g., 14971, IEC 62304:200613) and product (e.g., 60601 series) standards. For the areas that are heavily regulated through process standards, adequate examination of the process output and product itself is an essential element of safety assessment. For example, it is common for an organization to have records demonstrating compliance with 62304 by following the same software standards of practice across its software projects but to experience a varying level of quality with software products in the field. Similarly, 14971 is the medical device risk management standard; many organizations don't have issues staying compliant with it, but they may experience unequal safety results. Compliance with process standards helps to provide the confidence that the process has been followed properly, but the output (e.g., the software code written, the hazards and risk controls identified, the product design) attributes still need further examination to ensure that the process was effective in achieving the intended product objective. Table 2 provides an example of the types of data sets associated with each of the criteria.

The use of ambiguous words such as “adequately” exemplifies our inability to directly verify safety in the same manner that we would verify a product requirement. The traditional practice for safety assessment has been that industry developers provide the data, such as those listed in Table 2, and regulators review and analyze the data and draw conclusions (Figure 2).

The challenge for regulators often is that the data provided are in certain formats, as a result of compliance, but do not point directly to why a desired safety conclusion should be drawn or how the data collectively support the desired conclusion. Data sets are growing increasingly large and complex commensurate with technology (e.g., connected devices, devices using artificial intelligence [AI] or machine learning technologies, device-drug combination products). The time and effort needed for regulators to review, connect the dots, and draw conclusions are challenging. This situation calls for developers to provide additional background and justification as to why the data supports the safety conclusion. This typically leads more structured data being provided to explain and justify the adequacy. Table 3 provides a high-level display of the justifications that can be provided for each safety subcriteria identified previously.

Figure 3 illustrates a more detailed example, including how a risk assessment report can be structured with justifications explaining how residual risks are adequately controlled.

If developers can provide structured data with the necessary justification as described, then the information provided by the developer is more easily assimilated and regulatory reviewers can focus more clearly on assessing whether the data and explanation provided are sufficient, trustworthy, and reasonable. Regulators can apply their knowledge (which is not necessarily available to the developers) to review and challenge as needed. As such, the regulatory review becomes an effective process of checks and balances. This approach will help regulators promote innovation in healthcare technology by empowering developers to create their own arguments for safety, rather than restricting developers to predefined regulatory frameworks, which may not necessarily be suitable to all technologies.

For example, for emerging healthcare technologies such as AI- or machine learning–driven medical applications, it may take years for the relevant industry standards or regulations to be fully established. With this approach, developers can craft their own justification for safety with considerations of a device's potential benefits to convince regulators rather than waiting for regulations to be fully ready. This will encourage developers to do their own thinking to assess and ensure safety; it also provides opportunity for the regulators to continuously build their knowledge base and stay current as the industry advances.

Scientific thinking involves the application of empirical evidence, logical reasoning, and a skeptical attitude toward a subject. In practicing regulatory science to assess and ensure the safety of medical devices, regulators and developers can collaboratively practice this scientific (and critical) thinking.

As shown in Figure 4, industry partners and regulators can establish a common safety assessment and assurance framework. Using this construct, developers can provide structured data-driven evidence, including reasoning, and the regulators can perform knowledge-based review and critical thinking to balance risk with benefits and draw regulatory conclusions. With data and reasoning being structured, the reviewers can productively perform a high-level review of the safety argument, followed by detailed reviews on high-risk or randomly sampled areas. This is important considering that regulators' reviews are time constrained, and it may not be practical at times for regulators to review every detail.

This type of collaborative scientific thinking can bring developers and regulators to the same conclusion in terms of what is important to safety, what developers should do, how to share information, and what (and how) regulators should examine in the review—resulting in a safety value-added process. For complex or innovative medical products using emerging technologies (for which many questions may exist), this collaborative scientific thinking–based method provides a solution to reduce regulatory burden and time to market.

Some international standards and Food and Drug Administration (FDA) initiatives have used this scientific thinking–based method. Examples include the technical information report AAMI TIR38:2019,14 infusion pump total product life cycle guidance from the FDA's Center for Devices and Radiological Health,15 FDA's Accreditation Scheme for Conformity Assessment,16 FDA's Knowledge-aided Assessment & Structured Application,17 IEC/TR 80001-2-9:2017,18 and ANSI/AAMI/UL 2800-1:2019.19

As healthcare technology continues to advance rapidly, the application of scientific thinking in regulatory science is key to reducing time to market and promoting innovation while ensuring medical device safety.

To Sandy Weininger, safety reviewer at the FDA CDRH; Winifred Wu, regulatory consultant and former vice president of regulatory affairs at Medtronic; Yi Zhang, senior researcher at Massachusetts General Hospital and former FDA scientist; and Edwin Bills, principal consultant of ELB Consulting for valuable input and comments.