Great insight, Edwin, and thanks for the background! I do actually recommend quantitative, qualitative, and SW-only breakdowns in risk management processes for P1 and both quantitative and qualitative in P2 as well. I totally agree that it is tough to estimate probabilities (and even likelihoods) in brand new devices, with no post-market data to check against. The conversion to quantitative probabilities is much more doable, with post-market data available.
As I stated, the SW-only P1 is a likelihood measure and not a probability measure, given that most software is deterministic and specific probabilities of code-path execution are tough to get below that deterministic threshold. I put a small disclaimer at the bottom of the P1 table recognizing the statements in 62304 regarding probability and stating the likelihood substitution for SW.
Thanks again to both of you for a great tangent to the original question. Totally worth the discussion.
------------------------------
Eric Henry
King & Spalding
Washington DC
United States
------------------------------
Original Message:
Sent: 15-Mar-2019 07:38
From: Edwin Bills
Subject: Ask Me Anything Session: Risk Management Principles
I want to support Dan's use of the term "conceptual". As a member of the technical committee that wrote the standard we included a discussion of P1/P2 in the informative annexes (not requirements) to explain why just because a hazard is present does not automatically mean that harm results. There has to be a hazardous situation occur that exposes the hazard in such a way that harm may (or may not) occur. It depends on a sequence of events that may allow the harm to occur. In the next edition of ISO 14971 we will point out that you do NOT have to identify P1/P2 in your risk analysis. Dan pointed to the difficulties in establishing values of the two.
Our preference was to use the term "likelihood" instead of "probability", but it would not translate into other languages sufficiently to use the term. Probability did translate diffidently to do the job. Unfortunately English-speakers took the term quite literally and want to identify quantative values where they have insufficient data to support their use. So I might also point out that the standard allows either qualitative or quantitative values to be identified. For a PMA type of device you would not normally be able to identify quantitative values of P with any confidence until at least Design Validation when you gather use data with the device in actual use. You would have higher confidence in the data from actual use when the device is released to the market. In this case you would use qualitative levels of probability until you have sufficient data to establish confidence in quantative estimates in your values of probability. As soon as you have high confidence you should change from qualitative to quantitative values of P.
------------------------------
Edwin Bills MEd, CQA, RAC, BSc, CQE, ASQ
Principal Consultant
Overland Park KS
United States
elb@edwinbillsconsultant.com
Original Message:
Sent: 14-Mar-2019 07:55
From: Eric Henry
Subject: Ask Me Anything Session: Risk Management Principles
Dan,
Interesting point of view, and thanks for the comment. I have helped deploy risk management with companies using P1/P2 successfully and particularly using P2 across severities to create a workable distribution and initial and residual risk profile, so it was good to hear the other side of the story. We have also used them as probabilities and not as frequencies of occurrence. The exception to this being software risk management, where P1 is effectively translated to a P1a focused on occurrence so that the overall risk profile is not skewed solely towards severity thus causing overmitigation of software-driven hazardous situations.
Thanks again,
------------------------------
Eric Henry
King & Spalding
Washington DC
United States
Original Message:
Sent: 13-Mar-2019 18:44
From: Dan O'Leary
Subject: Ask Me Anything Session: Risk Management Principles
Eric,
You asked if the P1/P2 method from ISO 14971:2007 would help meet the EU-MDR requirements for risk management.
The short answer is NO, they are not related. There are, in theory, two approaches. In one approach, estimate the probability that the hazard situation occurs, then estimate the probability that if the hazardous situation occurs patient or user harm results. Calculate the product to estimate the probability of harm with the stated severity.
In the second case, directly estimate the probability of harm with the stated severity; do not break the estimate into these components.
Regardless of the method, there is nothing that I can see in the EU-MDR that would change based on the method employed. By this I mean either method would satisfy the EU-MDR.
However, there are other issues with the P1/P2 method that, in my opinion, make it impractical.
The implicit assumption is that P1 and P2 are point estimates, so their product is a point estimate. In practice, they are probability distributions, so their "product" is not a simple multiplication.
Also, P1, P2, or their product are not probabilities, but statements of frequency of occurrence. (Once and event happens, its probability of occurrence is 1.)
Because people tend to use powers of 10 in setting the frequency of occurrence, there is an implicit assumption about the resulting distribution. Using Table D.4, it appears, for example, that Occasional means that any frequency of occurrence between 0.0001 and 0.00001 are equally likely to happen. This is a uniform distribution.
In addition, there is a conditional probability, making the calculation a little more difficult.
Even if they were point estimates, each number is small, so their product is smaller. In ISO 14971:2007 Table D.4 a Frequent occurrence is about 1 time in a 1,000 or 0.001. This is the product of P1 and P2. If P1 and P2 were equal, each would be about 0.0316.
I believe that P1/P2 is a wonderful conceptual model. In my risk management course, I teach people to consider the sequence of events leading to the hazardous situation. (In my opinion neither ISO 14971:2007 nor ISO 14971:2019 puts enough emphasis on the sequence of events.) Breaking the sequence of events makes P1 = 0, so the hazardous situation cannot occur so patient or user harm cannot result.
In practice nobody has good estimates for P1 or for P2, so there are no good estimates for their product.
My recommendation is that P1/P2 is a good conceptual model but is not a practical method for calculation.
------------------------------
Dan O'Leary CQA, CQE
Swanzey NH
United States
Original Message:
Sent: 13-Mar-2019 13:34
From: Eric Henry
Subject: Ask Me Anything Session: Risk Management Principles
Posting the two questions I posed in the other thread:
Two questions I frequently have to answer are: (1) Does the P1/P2 probability estimation method recommended by ISO 14971 help me with meeting EU MDR requirements for risk management (especially as it regards the requirement for better post-market feedback into risk management)?; (2) How do I address software risk management more effectively than just considering severity since this could lead to over-mitigation of highly unlikely hazardous situations?
I have guidance I provide in both of these areas, but I would be very interested in your perspective as well.
Thanks!
------------------------------
Eric Henry
King & Spalding
Washington DC
United States
Original Message:
Sent: 13-Mar-2019 13:32
From: Meredith Smith
Subject: Ask Me Anything Session: Risk Management Principles
Hi, I'm Meredith. Looking forward to answering your questions!
------------------------------
Meredith Smith
Thousand Oaks CA
United States
Original Message:
Sent: 13-Mar-2019 13:30
From: Emily Stamm
Subject: Ask Me Anything Session: Risk Management Principles
Hi Members,
Recently, RAPS released its' Quarterly Regulatory Focus Article Series, "Risk Management Principles: A Global Perspective. Today, we have three risk management experts, @Darin Oppenheimer, @Meredith Smith, and @Anne Walsh, who will be discussing the articles in real-time and available to answer your questions.
Please use this thread to ask the experts your questions.
Can't wait to read the discussions!
Best,
Emily
------------------------------
Emily Stamm
Community Manager
Regulatory Affairs Professionals Society®
regex@raps.org
------------------------------