Kevin,
FMEA is a useful tool, when used within its limitations. Among them is that it is a "single fault tool", which does not align with the requirements of ISO 14971, that is "identify known and foreseeable hazards ...in both normal and fault conditions". It does not meet those requirements totally, but can provide valuable information on single fault condition hazards, just not all fault condition hazards. And you have to interpolate the information in such a way that you meet the 14971 definition of risk, which is not that same as true FMEA definition of risk (combination of probability of occurrence of failure-not harm as in 14971, and severity of effect of failure-not severity of harm as in 14971).
Another limitation is that FMEA cannot be performed until you have a good idea of the detailed design, which means you have to be in Design Output portion of the Design Controls. ISO 13485 requires in 7.3.3 (c) that you provide output from risk management (ISO 14971) as Design Inputs, thus earlier than Design Output. So you must use other risk tools, one specifically that meets this requirement is Preliminary Hazard Analysis (PHA), others can be used as well. FMEA is a valuable tool to provide a check that you have identified all possible events that might lead to a hazardous situation that could lead to harm. It is late in the design process, and if that is the only tool used it can be costly, as you find risks in the device that have to be controlled after the design is nearly complete rather than while it is "still on paper" (or on screen). Estimates of 10X greater expense to fix issues in design at each stage can add up to time delays and expose of scrapping tooling, revising manufacturing processes, etc, when fixing things late in the process.
Back in 2004, the late Mike Schmidt, one of my fellow quality engineers at Hill-Rom and later at Ethicon penned an article on the use of detectability in MDDI. Basically, Mike suggested that detectability can be used in Process FMEA to identify process changes that can prevent or reduce the escape of failures. But it is not a risk control that can be used to allow the user to detect the problem and reduce risk, as it was often done in the early days of EN 1441 and ISO 14971-1 (pre-2000). That is a recipe for big product liability payouts.
RPN has been defined in IEC 60812, FMEA reliability analysis standard as probability of occurrence of failure X severity of effect of failure X detectability of failure and used to rank the importance of failures to the reliability of the design, and which should be fixed first to improve the reliability. It was never intended to provide a ranking of safety, which is the subject of ISO 14971, where we chose to allow the manufacturer to determine what risk policy they would use to determine how to establish what risks were acceptability. One long discussion I remember was that the risk of a bathtub fall was 1 in 800,000 and was commonly an acceptable risk (nearly 1:1,000,000 which was used in sterilization to establish limits). We looked at a number of risks in that context to try and identify what might be acceptable and determined it had to be in the context of the intended use of the device and the state of the art of the technology available, as well as competing therapies, including pharmaceuticals. It could not be an arbitrary scale like RPN, which has no basis in science, but more in convenience.
I might point to the elimination of the use of RPN in automotive FMEAs in the recent edition of the AIAG FMEA standard, and replacement by another method, which is still not acceptable in 14971. Risk Acceptability Criteria (which is what RPN purports to be, in ISO 14971 consists of only two elements probability of harm and severity of harm. It must be based on the Risk Policy established by the manufacturer (ISO 14971:2019 4.2 and further explained in ISO TR 24971 4.2.2 and Annex C), none of which refer to RPN or Detectability. Where detectability has value (outside ISO 14971 and risk management) is in identifying where failures might be detectable in the manufacturing process, prior to release of the device, so that inspection and testing can be established at the proper place in the process to reduce the probability of escapes occurring.
One of the things I have seen is that these tools are often misused, as you indicated. Not every failure in an FMEA (of any type) needs to be a part of risk management as many relate to improvements that can be made in the manufacturing process for yield or the reliability of the design, as opposed to product safety, which is 14971. After all, FMEA has been developed as a tool for reliability analysis and not safety analysis, although it may be used with some utility to identify areas where the design may be the source of a safety issue.
Which leads me to another thought, Normal condition hazards may occur when design is working just fine and meets its design criteria, but FMEA will not identify them. Most often these issues are found when examining usability and looking at readably predictable human behavior. Of course, this is the subject of IEC 62366 and also AAMI HE 75. Another long discussion for that topic beyond what I can offer proper expertise.
Anyway, Kevin, I enjoy our discussions and hope that others can take away some gems of knowledge that might be useful. Thanks for providing the opportunity.
------------------------------
Edwin Bills MEd, CQA, RAC, BSc, CQE, ASQ
Principal Consultant
Overland Park KS
United States
elb@edwinbillsconsultant.comPrincipal Consultant
------------------------------
Original Message:
Sent: 02-Aug-2021 14:30
From: Kevin Randall
Subject: Software anomaly with unacceptable RPN score
Hi Ed,
My experience is that when used properly, "RPN"s aren't prohibited by ISO 14971, nor is the consideration of a "detectability" attribute when applied properly. It seems like "RPN" and FMEA have gotten bad names for themselves in recent years due to potential misuse; yet they should nonetheless not be banned or canceled. Wondering if you can let me know your thoughts on the supporting rationale below.
In practicing risk management, my experience has been that it's often necessary to dynamically integrate FMEA (or something similar) into the ISO 14971 risk management process. Specifically, my experience has been that it's often not possible to fully elucidate ISO 14971 sequences of events, hazardous situations, and their individual contributory probabilities (that feed the overall ISO 14971 probability of occurrence of harm) without employing some form, or adaptation, of FMEA. And in so doing for that FMEA-style step when used as intended, "detectability" has traditionally been an optional attribute that, at the manufacturer's discretion, may or may not be utilized for that part of the assessment. If FMEA is used properly (i.e., to help elucidate sequences of events, hazardous situations, and their contributory probabilities), then it seems ISO 14971 doesn't prohibit consideration of "detectability" for the FMEA-like steps.
My experience is that a similar dynamic exists regarding "RPN". When industry evolved from the old EN 1441-style assessments into modern ISO 14971 risk analysis and management, my experience was that companies continued to use the "RPN" vernacular when they applied ISO 14971 for semi-quantitatively expressing various combinations of the probability of occurrence of harm and severity of harm (i.e., to express ISO 14971 "risk"). While true that traditional RPNs from the FMEA portion of the evaluation shouldn't be equated with the ultimate ISO 14971 risk acceptance criteria, it also seems true that this shouldn't cause us to create a universal ban against RPNs or the term "RPN".
Indeed, in alignment with ISO 14971's core principles, "RPN" may have different meanings for different companies. What I mean is that, while true that "RPN" is not a term used in ISO 14971 as a method of identifying risk acceptability, neither are any other such terms. I believe that is deliberate in order to allow each manufacturer to derive risk acceptability policy/methods appropriate for the manufacturer's unique needs. Some key paraphrased / recited principles for this from ISO 14971 and ISO/TR 24971 are that:
- The criteria for risk acceptability are established by the manufacturer based on the manufacturer's policy for determining acceptable risk.
- The risk acceptance policy and its elements are tailored by the manufacturer to fit the specific needs of the manufacturer's organization.
- The manufacturer needs to establish risk acceptability criteria that are appropriate for the particular medical device.
- The criteria for risk acceptability can include combinations of qualitative requirements and quantitative limits for specific properties.
- The criteria for risk acceptability are recorded in the risk management plan.
- The criteria for the acceptability of the overall residual can be the same or different from the criteria for acceptability of individual risks.
- Whatever is/are the manufacturer's method(s) and policy for evaluating risk acceptability, the method(s) is/are to be defined and documented in the risk management plan.
Accordingly, when applying the ISO 14971 principle that criteria for risk acceptability can be based on combinations of qualitative, semi-qualitative, quantitative, or semi-quantitative requirements/limits, it seems that ISO 14971 permits the manufacturer to characterize those combinations in whatever way works for the organization. And if the manufacturer's chosen approach happens to be based on assigning numeric scores for probability of occurrence of harm and severity of harm, and then ranking various combinations as "RPN"s with respect to risk acceptability, then I think ISO 14971 permits such an approach as long as the manufacturer has properly defined the approach and limits in the risk management plan.
Looking forward to hearing your insights as always.
------------------------------
Kevin Randall, ASQ CQA, RAC (U.S., Europe, Canada)
Principal Consultant
Ridgway, CO
United States
© Copyright 2021 by ComplianceAcuity, Inc. All rights reserved.
Original Message:
Sent: 01-Aug-2021 09:53
From: Edwin Bills
Subject: Software anomaly with unacceptable RPN score
Since ISO 14971:2019 and ISO TR 24971:2020 were mentioned, I will jump in and state you will not find RPN mentioned anywhere in either document. RPN is not recognized in the standard as a method of identifying risk acceptability as it has several problems, one is its use of "detectability" which is not part of the definition of "risk" in the standard.
Another issue is the definition of "risk" itself in FMEA. FMEA RPN uses the probability of failure and the severity of the effect of the failure and the detectability of the failure. This is not the same as ISO 14971, which is "the combination of the probability of occurrence of harm and the severity of harm".
So in doing a risk-benefit analysis RPN is not a part of this analysis. You will find discussions of risk-benefit in Clause 7.4 for individual risks and Clause 8 of the standard, with much more explanation in the technical report ISO TR 24971 in the same clause numbers.
Incidentally, since you are working in software, Annex F of ISO TR 24971:2020 discusses risk ind data and Cybersecurity, which are now part of software risk management.
I would also recommend you use ISO TR 80002-1 which discusses software risk management in the context of software lifecycles such as defined in ISO 62304. It will be a great help for you in performing software risk management. If you are using Agile software development methods, AAMI TIR 45 addresses the use of Agile in medical device software development.
The bottom line is drop the use of RPN in software risk management and use your policy for risk acceptability to identify your risk acceptability criteria as required in ISO 14971:2019.
------------------------------
Edwin Bills MEd, CQA, RAC, BSc, CQE, ASQ
Principal Consultant
Overland Park KS
United States
elb@edwinbillsconsultant.comPrincipal Consultant
Original Message:
Sent: 31-Jul-2021 02:28
From: Christopher Erwin
Subject: Software anomaly with unacceptable RPN score
If you are applying ISO 14971:2019 there is instruction regarding the application of benefit-risk analysis to a residual risk once risk control measures have been implemented. Going through and documenting this exercise may help demonstrate that the benefit of the intended use of the device outweighs its risks. Such an activity should be carried out in the context of your company's risk acceptability policy and the risk management plan; ISO/TR 24971:2020 has some good guidance on this as well.
------------------------------
Christopher Erwin
Scottsdale AZ
United States
Original Message:
Sent: 28-Jul-2021 23:18
From: Anonymous Member
Subject: Software anomaly with unacceptable RPN score
This message was posted by a user wishing to remain anonymous
Hi RAPS members,
I want to get some clarity on regulatory concern about a software anomaly that has unacceptable RPN score.
The latest version of the SW is impacted by a known anomaly (existed in previous version). The harm caused as a result of this anomaly is negligible, but it has unacceptable RPN score due to the high occurrence rate. The SW will be released with this unresolved anomaly. This issue results in a recoverable fault and the user is able to complete the procedure without much (3-4 minute) delay.
The unresolved anomaly list will be submitted to the regulatory body together with the clinical risk benefit analysis. Please share your thoughts and guidance on this situation.
Thanks