Regulatory Open Forum

 View Only
Expand all | Collapse all

Ask Me Anything Session: Risk Management Principles

  • 1.  Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 13:30
    Hi Members,

    Recently, RAPS released its' Quarterly Regulatory Focus Article Series, "Risk Management Principles: A Global Perspective. Today, we have three risk management experts, @Darin Oppenheimer, @Meredith Smith, and @Anne Walshwho will be discussing the articles in real-time and available to answer your questions.

    Please use this thread to ask the experts your questions.


    Can't wait to read the discussions!

    Best,
    Emily ​​​​

    ------------------------------
    Emily Stamm
    Community Manager
    Regulatory Affairs Professionals Society®
    regex@raps.org
    ------------------------------


  • 2.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 13:32
    ​Hi, this is Anne Walsh.  Looking forward to a good discussion.  Send your questions!

    ------------------------------
    **************************
    Anne K. Walsh
    Hyman, Phelps & McNamara, P.C.
    700 Thirteenth St., N.W., Suite 1200
    Washington, D.C. 20005
    awalsh@hpm.com
    Direct: (202) 737-4592
    Fax: (202) 737-9329
    Cell: (202) 834-2462

    See HPM's FDA Law Blog: http://www.fdalawblog.net/
    ------------------------------



  • 3.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 13:33
    ​Hi, I'm Meredith.  Looking forward to answering your questions!

    ------------------------------
    Meredith Smith
    Thousand Oaks CA
    United States
    ------------------------------



  • 4.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 13:34
    Posting the two questions I posed in the other thread:

    Two questions I frequently have to answer are:  (1)  Does the P1/P2 probability estimation method recommended by ISO 14971 help me with meeting EU MDR requirements for risk management (especially as it regards the requirement for better post-market feedback into risk management)?; (2) How do I address software risk management more effectively than just considering severity since this could lead to over-mitigation of highly unlikely hazardous situations?

    I have guidance I provide in both of these areas, but I would be very interested in your perspective as well.

    Thanks!

    ------------------------------
    Eric Henry
    King & Spalding
    Washington DC
    United States
    ------------------------------



  • 5.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 13:41
    Hi Eric,

    Could you please clarify the second half of your question:  are you referring to software risk management in the context of the new EU MDR requirements specifically?

    Thanks.​

    ------------------------------
    Meredith Smith
    Thousand Oaks CA
    United States
    ------------------------------



  • 6.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 13:42
    No.  The second question is separate and gets into the contention from IEC 62304 and TR 80002 that software risk analysis can only consider severity due to its deterministic nature.

    ------------------------------
    Eric Henry
    King & Spalding
    Washington DC
    United States
    ------------------------------



  • 7.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 13:46
    Hi Eric,

    Great question. I would like to follow up with you  after the discussion.

    Thanks,

    Darin

    ------------------------------
    Darin Oppenheimer FRAPS, RAC
    Executive Director
    Medford NJ
    United States
    ------------------------------



  • 8.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 17:04
    ​Hello Eric

    My response is narrowly focused on the second part of your query, regarding software risk management…

    While IEC 62304: 2006 simply looks at the Severity to arrive at the Software Safety Classification (i.e. Class A/ B/ C), Amendment 1: 2015 clarifies that while software failures may lead to a Hazardous Situation, if the Risk Control Measures external to the software do not eventuate in or serve to reduce the Risk, then such mitigation can be considered to allow a lower Software Safety Classification.  The flowchart from Sec. 4.3 of Amendment 1: 2015 clarifies, stressing that the Risk Control Measures must be external to the software, for example hardware watchdog. 

    See also:
    https://blog.cm-dm.com/post/2016/05/06/Is-my-software-in-class-A%2C-B-or-C-2015-reloaded

    Hope this helps.

    ------------------------------
    Homi Dalal RAC
    Regulatory Affairs Leader
    Christchurch
    New Zealand
    ------------------------------



  • 9.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 14-Mar-2019 08:01
    Homi,

    Thanks for the response and comment.  I'm familiar with the change in Amendment 1 to broaden the control measures, which may reduce safety classification, this still doesn't address my issue with leaning so heavily on severity.

    The advice I give is to use P1/P2 to calculate probability for all hazard/hazardous situation pairs and to have a separate software-only column for P1 (including a written disclaimer to address the restrictions in 62304), which effectively converts P1 into P1a defined as the likelihood of occurrence (as opposed to probability).  This allows a more reasonable approach to risk controls especially high-severity rare events.

    I was looking for other possible approaches to the risk analysis process to broaden exposure to various points of view.

    Thanks again for the response, and if you have a particular view on this issue, please feel free to respond in kind.

    ------------------------------
    Eric Henry
    King & Spalding
    Washington DC
    United States
    ------------------------------



  • 10.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 18:44

    Eric,

    You asked if the P1/P2 method from ISO 14971:2007 would help meet the EU-MDR requirements for risk management.

    The short answer is NO, they are not related. There are, in theory, two approaches. In one approach, estimate the probability that the hazard situation occurs, then estimate the probability that if the hazardous situation occurs patient or user harm results. Calculate the product to estimate the probability of harm with the stated severity.

    In the second case, directly estimate the probability of harm with the stated severity; do not break the estimate into these components.

    Regardless of the method, there is nothing that I can see in the EU-MDR that would change based on the method employed. By this I mean either method would satisfy the EU-MDR.

    However, there are other issues with the P1/P2 method that, in my opinion, make it impractical.

    The implicit assumption is that P1 and P2 are point estimates, so their product is a point estimate. In practice, they are probability distributions, so their "product" is not a simple multiplication.

    Also, P1, P2, or their product are not probabilities, but statements of frequency of occurrence. (Once and event happens, its probability of occurrence is 1.)

    Because people tend to use powers of 10 in setting the frequency of occurrence, there is an implicit assumption about the resulting distribution. Using Table D.4, it appears, for example, that Occasional means that any frequency of occurrence between 0.0001 and 0.00001 are equally likely to happen. This is a uniform distribution.

    In addition, there is a conditional probability, making the calculation a little more difficult.

    Even if they were point estimates, each number is small, so their product is smaller. In ISO 14971:2007 Table D.4 a Frequent occurrence is about 1 time in a 1,000 or 0.001. This is the product of P1 and P2. If P1 and P2 were equal, each would be about 0.0316.

    I believe that P1/P2 is a wonderful conceptual model. In my risk management course, I teach people to consider the sequence of events leading to the hazardous situation. (In my opinion neither ISO 14971:2007 nor ISO 14971:2019 puts enough emphasis on the sequence of events.) Breaking the sequence of events makes P1 = 0, so the hazardous situation cannot occur so patient or user harm cannot result.

    In practice nobody has good estimates for P1 or for P2, so there are no good estimates for their product.

    My recommendation is that P1/P2 is a good conceptual model but is not a practical method for calculation.



    ------------------------------
    Dan O'Leary CQA, CQE
    Swanzey NH
    United States
    ------------------------------



  • 11.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 14-Mar-2019 07:56
    Dan,

    Interesting point of view, and thanks for the comment.  I have helped deploy risk management with companies using P1/P2 successfully and particularly using P2 across severities to create a workable distribution and initial and residual risk profile, so it was good to hear the other side of the story.  We have also used them as probabilities and not as frequencies of occurrence.  The exception to this being software risk management, where P1 is effectively translated to a P1a focused on occurrence so that the overall risk profile is not skewed solely towards severity thus causing overmitigation of software-driven hazardous situations.

    Thanks again,

    ------------------------------
    Eric Henry
    King & Spalding
    Washington DC
    United States
    ------------------------------



  • 12.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 14-Mar-2019 08:11

    Eric,

    Thank you for the response. I'm curious how you get useful estimates of P1 and P2. For example, assume a hot surface. How would you estimate P1 for the surface being exposed, the hazardous situation, and P2 for a patient or user touching the hot surface and receiving a burn, the harm?

    Do you use point estimates or do you try to determine a distribution for P1 and for P2 and then combine them to provide an entry into the risk matrix (assuming you use the traditional risk matrix to estimate risk and define acceptability)?



    ------------------------------
    Dan O'Leary CQA, CQE
    Swanzey NH
    United States
    ------------------------------



  • 13.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 15-Mar-2019 07:39
    I want to support Dan's use of the term "conceptual".  As a member of the technical committee that wrote the standard we included a discussion of P1/P2 in the informative annexes (not requirements) to explain why just because a hazard is present does not automatically mean that harm results. There has to be a hazardous situation occur that exposes the hazard in such a way that harm may (or may not) occur. It depends on a sequence of events that may allow the harm to occur. In the next edition of ISO 14971 we will point out that you do NOT have to identify P1/P2 in your risk analysis. Dan pointed to the difficulties in establishing values of the two.

    Our preference was to use the term "likelihood" instead of "probability", but it would not translate into other languages sufficiently to use the term. Probability did translate diffidently to do the job. Unfortunately English-speakers took the term quite literally and want to identify quantative values where they have insufficient data to support their use. So I might also point out that the standard allows either qualitative or quantitative values to be identified. For a PMA type of device you would not normally be able to identify quantitative values of P with any confidence until at least Design Validation when you gather use data with the device in actual use. You would have higher confidence in the data from actual use when the device is released to the market. In this case you would use qualitative levels of probability until you have sufficient data to establish confidence in quantative estimates in your values of probability. As soon as you have high confidence you should change from qualitative to quantitative values of P.

    ------------------------------
    Edwin Bills MEd, CQA, RAC, BSc, CQE, ASQ
    Principal Consultant
    Overland Park KS
    United States
    elb@edwinbillsconsultant.com
    ------------------------------



  • 14.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 15-Mar-2019 07:57
    Great insight, Edwin, and thanks for the background!  I do actually recommend quantitative, qualitative, and SW-only breakdowns in risk management processes for P1 and both quantitative and qualitative in P2 as well.  I totally agree that it is tough to estimate probabilities (and even likelihoods) in brand new devices, with no post-market data to check against.  The conversion to quantitative probabilities is much more doable, with post-market data available.

    As I stated, the SW-only P1 is a likelihood measure and not a probability measure, given that most software is deterministic and specific probabilities of code-path execution are tough to get below that deterministic threshold.  I put a small disclaimer at the bottom of the P1 table recognizing the statements in 62304 regarding probability and stating the likelihood substitution for SW.

    Thanks again to both of you for a great tangent to the original question.  Totally worth the discussion.

    ------------------------------
    Eric Henry
    King & Spalding
    Washington DC
    United States
    ------------------------------



  • 15.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 13:34
    Hello,

    I am excited to hear your questions.

    ------------------------------
    Darin Oppenheimer FRAPS, RAC
    Executive Director, GRACS Device and Digital Health Merck
    United States
    ------------------------------



  • 16.  RE: Ask Me Anything Session: Risk Management Principles

    This message was posted by a user wishing to remain anonymous
    Posted 13-Mar-2019 13:34
    This message was posted by a user wishing to remain anonymous

    This question relates to OEMs - If the legal manufacturer receives the device in bulk <g class="gr_ gr_9 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling multiReplace" id="9" data-gr-id="9"><g class="gr_ gr_6 gr-alert gr_spell gr_inline_cards gr_disable_anim_appear ContextualSpelling" id="6" data-gr-id="6">non</g> sterile</g> form and then packs/sterilizes the device before supplying under their brand name, in order to meet the essential requirements, are they able to use any of the verification/validation testing done by the OEM? What is the normal process when <g class="gr_ gr_5 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" id="5" data-gr-id="5">OEMS</g> supply bulk <g class="gr_ gr_10 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling multiReplace" id="10" data-gr-id="10">non sterile</g> items? Who normally does the design verification/validation?


  • 17.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 13:42
    Hello,

    Great question and a challenging one at that. The burden will typically fall on the legal manf. The information you may be able to obtain from the supplier can be useful but this often depends on many factors such as tests performed, standards utilized, quality of testing, and other aspects that may contribute to the validity of the test information. This information may be useful as supportive information to help you identify what the overall strategy could be regarding supplemental testing.

    The responsibilities are frequently outlined in the contractual obligations between the partner and the manf so it is clear who will provide what information and objective evidence to support product testing, verification or validation.

    ------------------------------
    Darin Oppenheimer FRAPS, RAC
    Executive Director
    Medford NJ
    United States
    ------------------------------



  • 18.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 13:53
    @Meredith Smith​ ​Are there any resources available to guide the development of risk minimization tools to minimize medication errors associated with drugs intended for use in pediatric patient populations?

    ------------------------------
    **************************
    Anne K. Walsh
    Hyman, Phelps & McNamara, P.C.
    700 Thirteenth St., N.W., Suite 1200
    Washington, D.C. 20005
    awalsh@hpm.com
    Direct: (202) 737-4592
    Fax: (202) 737-9329
    Cell: (202) 834-2462

    See HPM's FDA Law Blog: http://www.fdalawblog.net/
    ------------------------------



  • 19.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 14:00
    ​Hi Anne,

    The EMA released a Good Practice Guide in 2015 on Risk Minimization and Prevention of Medication Errors.  This is an excellent resource for sponsors to use in developing risk minimization measures to reduce medication errors, including measures for use within pediatric patient populations.  Another excellent resource is the EMA's Guideline on GVP Module XVI- Addendum I, "Educational Materials" which is a supplement to the GVP Module XVI on developing and evaluating risk minimization measures.  That addendum was released in 2015 as well.

    ------------------------------
    Meredith Smith
    Thousand Oaks CA
    United States
    ------------------------------



  • 20.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 14:02
    @Meredith Smith ​For the purposes of characterizing a risk associated with a product indicated for pediatric patients, can clinical trial data from the adult indication be used?

    ------------------------------
    Darin Oppenheimer FRAPS, RAC
    Executive Director
    Medford NJ
    United States
    ------------------------------



  • 21.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 14:11
    @Darin Oppenheimer:  Adequate characterization ​of risks is a key step in the risk management planning process for products intended for pediatric patient use.  A range of different data sources can be used, included pre-clinical evidence (i.e., general tox studies, pre- and post-natal animal studies and juvenile animal tox studies), and clinical trials in both pediatric and adult patients.  One caveat in using data from adult clinical trials is that such studies may not capture the full range of adverse events or clinical manifestations seen in children.

    ------------------------------
    Meredith Smith
    Thousand Oaks CA
    United States
    ------------------------------



  • 22.  RE: Ask Me Anything Session: Risk Management Principles

    This message was posted by a user wishing to remain anonymous
    Posted 13-Mar-2019 14:04
    This message was posted by a user wishing to remain anonymous

    Hello,

    Are there currently any guidelines or best practice documents that describe expected training and/or experience for those involved in the risk management process?  For example, do regulators typically expect that a manufacturer would employ a clinician who would be involved in determining the risk-benefit of a medical device?  Are there specific training programs or resources available for non-engineers and non-clinicians who may be involved in the risk management process?

    Thank you.


  • 23.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 14:18
    ​Hi Anonymous -

    Yes, FDA has issued a few guidance documents specific to risk management principles. For example, https://www.fda.gov/downloads/Guidances/ucm073511.pdf

    With respect to your specific question about whether a medical device manufacturer needs to have a clinician on staff, I am aware of several companies that do not employ a medical officer on staff.  But these companies will need to have access to a medical opinion for various functions (like FMEA or HHEs). 

    And re training, I am sure RAPS offers training programs in this space.  There may be other opportunities for training as well.

    Best,
    Anne


    ------------------------------
    **************************
    Anne K. Walsh
    Hyman, Phelps & McNamara, P.C.
    700 Thirteenth St., N.W., Suite 1200
    Washington, D.C. 20005
    awalsh@hpm.com
    Direct: (202) 737-4592
    Fax: (202) 737-9329
    Cell: (202) 834-2462

    See HPM's FDA Law Blog: http://www.fdalawblog.net/
    ------------------------------



  • 24.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 14:22
    Another area of interest for additional training in risk management is through different Universities. I know several University Graduate Regulatory Science programs have specific courses in the areas of Risk Management and Risk Tools and Techniques.

    ------------------------------
    Darin Oppenheimer FRAPS, RAC
    Executive Director
    Medford NJ
    United States
    ------------------------------



  • 25.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 14:21

    @Anne Walsh What are the risks of failing to comply with a negotiated REMS program How can companies minimize those risks?

    ------------------------------
    Meredith Smith
    Thousand Oaks CA
    United States
    ------------------------------



  • 26.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 14:25
    FDA considers a drug misbranded if there is a failure to comply with any of the elements of a REMS program: timetable, medication guide, communication plan, Elements to Assure Safe Use, or implementation system.  A violation of REMS can result in administrative action (like a Warning Letter or Untitled Letter), or even civil or criminal liability.  Recently, there has been interest by DOJ in bringing False Claims Act cases premised on a violation of the REMS program.  In 2017, Novo Nordisk paid $58 million to settle its FCA claims, and Aegerion paid $35 million related to its REMS violations.

    Companies can minimize risk by aligning on the purpose of the REMS program (e.g., divorcing it from the sales functions), training employees and external stakeholders involved at all points of the distribution process, and establishing a track record of swift corrective action should a violation be identified.

    ------------------------------
    **************************
    Anne K. Walsh
    Hyman, Phelps & McNamara, P.C.
    700 Thirteenth St., N.W., Suite 1200
    Washington, D.C. 20005
    awalsh@hpm.com
    Direct: (202) 737-4592
    Fax: (202) 737-9329
    Cell: (202) 834-2462

    See HPM's FDA Law Blog: http://www.fdalawblog.net/
    ------------------------------



  • 27.  RE: Ask Me Anything Session: Risk Management Principles

    Posted 13-Mar-2019 14:36

    Thanks for joining us for this wonderful session and a big thank you to our experts @Darin Oppenheimer@Meredith Smith, and @Anne Walsh


    As a reminder, you can find the Regulatory Focus Article Series "Risk Management Principles: A Global Perspective" here.


    Feel free to keep the conversation going by adding your question to this thread.

    Thanks again!

    Emily

    ​​​​​​​​​​​​​​​​​​

    ------------------------------
    Emily Stamm
    Community Manager
    Regulatory Affairs Professionals Society®
    regex@raps.org
    ------------------------------