© Copyright 2023 by ComplianceAcuity, Inc. All rights reserved.
Original Message:
Sent: 05-Oct-2023 15:52
From: Edwin Bills
Subject: Probability of Occurrence
Yes, Kevin that is what I meant. ISO TR 24971:2020 refers to IEC 60812 as the FMEA standard. FMEA uses definitions for risk, probability of occurrence and severity that are different from ISO 14971. They talk about "probability of failure" and "severity of effect". The input to Risk Analysis is "hazard" and FMEA is useful in identifying failure effects which may be hazards, but only in single fault conditions. ISO 14971 requires identification of "normal condition" hazards and all hazards, not just st single fault. So there are limitations to its usefulness. Another issue is that FMEA requires Design Outputs in order to perform its analysis, yet ISO 13485:2016 7.3.3 c) requires "outputs of risk management" be Design Inputs. So FMEA is late in the game, and relying on it, as so many companies do, adds cost and time to projects when they wait until after the design is firmed up to perform risk analysis. This causes costly redesigns and may impact already selected suppliers and tooling for example.
BUT, FMEA is a good tool to identify any overlooked single-fault hazards that may appear in the Design Output.
FMEA is also a great tool for its intended purpose, improving design reliability. Each of the tools identified in ISO TR 24971:2020 Annex B has its own Benefits and Limitations. It is important that we recognize those and use the best tools for the job. Early in Design-Development the best tool is Preliminary Hazard Analysis. For Human Factors/ Usability a new tool not shown in Annex B (because it's new) is Usability Related Risk Analysis (URRA), and while pFMEA has use in manufacturing another tool that intersects with Process Validation and Process Control and Monitoring is Hazard Analysis and Critical Control Points (HACCP).
While not widely used in medical devices, HACCP is growing in Pharma and originated in Foods (all juices and seafood processed in the US uses HACCP). HACCP can be used in connection with DOE to determine inputs to a process that have greatest impact and should be controlled. Then these are controlled in Process Validation, and finally the process outputs are monitored as part of HACCP to retain Process Control.
Anyway those are my comments on your question. Use FMEA within its limitations just like all tools, use them to your benefit.
------------------------------
Edwin Bills
Edwin Bills Consultant
ASQ Fellow CQE, CQA, CQM/OE, RAPS RAC
elb@edwinbillsconsultant.com
Original Message:
Sent: 05-Oct-2023 09:56
From: Kevin Randall
Subject: Probability of Occurrence
Edwin, when you said that FMEA is not risk analysis, I'm wondering if you meant to say that it is not risk management? ISO 14971 and ISO/TR 24971 maintain that FMEA is a valid risk analysis technique, yet not risk management.
------------------------------
Kevin Randall, ASQ CQA, RAC (Europe, U.S., Canada)
Principal Consultant
Ridgway, CO
United States
© Copyright 2023 by ComplianceAcuity, Inc. All rights reserved.
Original Message:
Sent: 05-Oct-2023 09:43
From: Edwin Bills
Subject: Probability of Occurrence
Dan has done a great job with his explanation, but much more can be said. First of all P1 and P2 was a concept developed as an educational tool in the Informative Annexes of ISO 14971 to explain why the probability of a failure occurrence was not the same as the probability of harm occurring. It was not intended to be a requirement.
Second, ISO TR 24971:2020 5.5.2 explains that Quantitative Probability should only be used when there is sufficient confidence in the data. Otherwise, Qualitative estimates should be used. Remember in product development we are making best guesses on the expected performance of the device and until we have Real World Evidence the Probabilities are only Estimates based on the information we have available.
For products that have Predicates we may have data with sufficient confidence. But brand new devices will not have that level of confidence until exposed to human use, Design Validation at the earliest.
Also as Dan indicates FMEA is not risk analysis. It may be used to identify hazards as input to a risk analysis. It also is late in the process as it requires Design Output to perform and ISO 13485:2016 7.3.3 c) requires the "outputs of risk management" are Design Inputs. FMEA is a good check tool to make sure no single-fault hazards are missed. Which brings up another issue as ISO 14971 does not restrict the process to single-fault hazards, but requires identification of "known and foreseeable hazards…in both normal and fault conditions". So the usefulness of FMEA in Risk Management is limited, as are all the tools. So use the risk tools within their limitations.
------------------------------
Edwin Bills
Edwin Bills Consultant
ASQ Fellow CQE, CQA, CQM/OE, RAPS RAC
elb@edwinbillsconsultant.com
Original Message:
Sent: 04-Oct-2023 16:24
From: Dan O'Leary
Subject: Probability of Occurrence
There are no circumstances in which you should use P1 and P2. The concept is pedagogical and not practical.
First, ISO 1497:2019 Annex C is informative. It doesn't create any requirements. Figure C1 shows a conceptual diagram, not a practical application.
In practice there are a few difficulties.
The process is to identify a hazard, a sequence of events leading to a hazardous situation, a hazardous situation leading to a harm.
P1 is the probability of a hazardous situation associated with the hazard mediated by the sequence of events. The steps in the sequence of events should, collectively, be a small probability – say one occurrence in 10,000 times. This means that all items in the sequence must occur at the same time. (Think of Reason's Swiss Cheese Model). In practice, it is a hard number to estimate. In addition, it is usually characterized as a point estimate. However, it should really be a statistical distribution.
P2 is the probability that the hazardous situation leads to a harm. This results when the patient or user is exposed to the hazardous situation. Again, it should be a small probability – one occurrence in 10,000 times. It is a hard number to estimate. In addition, a point estimate is the wrong model it should be a distribution.
P is the product P1×P2. If you have point estimate then multiply them together. However since they are really distributions, their product is the convolution of the distributions. That integral can be very hard to compute. Consider how you would multiply two normal distributions with different means and different standard deviations to get the resultant distribution.
The problem, in the point estimate case, is that you multiply a very small number by another very small number to get a very very small number.
In my class I do a thought experiment. There is a problem with a wall outlet in the meeting room. An electrician comes in, troubleshoots the problem and leaves. Unfortunately, she forgot to put the cover plate back. There is now exposed wire with AC mains. This is the hazardous situation. How do you estimate the probability of failure to put the cover plate back – P1?
Next, a person plugs something into the outlet without the cover plate, touches an exposed wire, and gets a shock (the harm). How do you estimate the probability of touching the exposed wire – P2?
In practice, just estimate P. The convention is that P is not a point estimate, but a range of values typically covering an interval that is an order of magnitude. The underlying distribution in uniform across the interval.
So, in practice, estimate the interval for P or do a lot of contortions that end up with an estimate of the interval for P.
You mention use and design FMEAs. Typically, an FMEA uses P only and usually defines it in an interval.
However, an FMEA is not the right tool for medical device risk management. It only considers single point failures (fault condition). In medical device risk management the hazard could be in either a normal or a fault condition. In addition. The sequence of events means it is not a single point event. Multiple things need to happen at the same time.
------------------------------
Dan O'Leary CQA, CQE
Swanzey NH
United States
Original Message:
Sent: 02-Oct-2023 07:43
From: Anonymous Member
Subject: Probability of Occurrence
This message was posted by a user wishing to remain anonymous
What is the consensus on the use of separate probabilities (P1, P2) for expressing the probability of occurrence of harm (ISO 14971:2019 Annex C) or just a single probability (P)?
I believe that the electronics industry has already moved in this direction but conventionally the medical device industry has used one (P) in the use and design FMEAs, but is there a move to separate probabilities, (P1) for the probability of a hazardous situation occurring and (P2) probability of a hazardous situation leading to harm?
Has anyone in the medical device industry moved to the use of separate probabilities and if so what has been your experience with the regulators – how have they perceived it?