Regulatory Open Forum

 View Only

  • 1.  Risk Control Implementation and Verification of Effectiveness

    Posted 13-Jan-2020 09:50
    Hi everyone

    Risk control measures serve as design inputs into design and development.  If somebody can speak on the risk management process, can you clarify the differences in testing requirements between risk control implementation and verification of effectiveness? 

    Thanks!

    ------------------------------
    Karen Zhou
    ------------------------------


  • 2.  RE: Risk Control Implementation and Verification of Effectiveness

    Posted 13-Jan-2020 10:42

    The implementation verification basically addresses "How do I know the risk reduction measure is in the device that ships?"

    The methodology is straight forward. Bring the risk control measure over as a design input. This will drive a design output. Design verification ensures the design output matches the design input. The design verification report becomes the implementation verification report. There is an implicit assumption that all design outputs correctly transfer to production specifications.

    One place that may need an adjustment is for risk control measures in the manufacturing process. Since they are not in the device, design verification might not work. Check that the process step is in the manufacturing plan and the PFMEA. Look for implementing procedures and work instructions.

    The effectiveness verification basically addresses "Why do I think the risk reduction measure will work?"

    There are multiple methods, one of which is design validation. The design validation report is the effectiveness verification report. Other methods include use of product standards, literature searches showing the measure works, and team consensus. ISO/TR 24971:2013 includes a discussion of using product safety standards for effectiveness verification. I expect that ISO/TR 24971:2020, when published, will cover this as well.



    ------------------------------
    Dan O'Leary CQA, CQE
    Swanzey NH
    United States
    ------------------------------

    Original Message


  • 3.  RE: Risk Control Implementation and Verification of Effectiveness

    Posted 14-Jan-2020 02:42
    Think of risk control as a process with the risk management process; sometimes including multiple steps in the process.  Once hazards have been identified, the organisation will then identify risk control measures that should be taken to reduce the hazard to acceptance.  Then risk control is implemented through maybe product testing, verification testing, or quality control inspection, etc.  Checking effectiveness of the risk control is ensuring the hazard is indeed being controlled.  Also think of risk control effectiveness as just not a one time event, but continually occurring through post market.  The post market data is feeding back into the risk management process, also confirming the risk control measures are still good and/or effective.

    ------------------------------
    Richard Vincins RAC
    Vice President Global Regulatory Affairs
    ------------------------------

    Original Message


  • 4.  RE: Risk Control Implementation and Verification of Effectiveness

    Posted 15-Jan-2020 08:36
    AAMI just announced availability of ISO DTR 24971:2020.  The draft document is an advance copy of the guidance for medical device risk management as defined in ISO 14971:2019. The document advertisement indicates immediate download availability and the purchaser will get the final document when it is released. The document carries a date of 2 February 2020.  The document is 100 pages of informative annexes, much of which has been updated to annexes formerly in the standard and the previous edition of ISO TR 24971:2013.  There is a great deal of new information as well. It is available at www.aami.org at the AAMI store.

    ------------------------------
    Edwin Bills MEd, CQA, RAC, BSc, CQE, ASQ
    Principal Consultant
    Overland Park KS
    United States
    elb@edwinbillsconsultant.com
    ------------------------------

    Original Message


  • 5.  RE: Risk Control Implementation and Verification of Effectiveness

    Posted 14-Jan-2020 14:22
    A bit of background here.  ISO 14971 in all the editions, including the 3rd (2019) edition was developed to work with a quality system (e.g. ISO 13485)or without a quality system for those cases where none was required from the regulators.  So the requirements for verification were developed to line up with the quality system requirements.  The risk controls should be documented as design inputs, requiring design output(s) for implementation. And remember ISO 13485:2016 indicates that the outputs of risk management (risk control measures) are design inputs.  The  verification of implementation would then be objective evidence the requirement was met in either the case for a quality system or no quality system.   The verification of effectiveness was required to provide objective evidence that the implemented risk control(s) did I actually reduce the risk to the level estimated in the Risk Management File.  This objective evidence may be acquired from a number of sources from bench testing to animal testing to clinical trials, depending on the particular evidence to be provided. In some cases this would be performed in Design Validation, again a part of a quality system requirement.  

    Don't look at the risk management system as a separate requirement or a checkbox activity.  If you have a quality system (and the MDD did not require one) then integrate it as part of the quality system throughout the lifecycle of the product.   ISO 14971:2019 especially strengthens the production-postproduction requirements in Clause 10 and uses the GHTF CAPA guidance as the basis for the revised requirements the same as ISO 13485:2016 did on Clause 8.  Again, the objective was to harmonize the risk management standard with the quality system standard to allow integration of the risk management system into the quality system.

    ------------------------------
    Edwin Bills MEd, CQA, RAC, BSc, CQE, ASQ
    Principal Consultant
    Overland Park KS
    United States
    elb@edwinbillsconsultant.com
    ------------------------------

    Original Message


  • 6.  RE: Risk Control Implementation and Verification of Effectiveness

    Posted 15-Jan-2020 08:35
    Edited by Adam Atherton 15-Jan-2020 08:36
    Agree with everything Dan, Richard, and Edwin said.

    As far as the distinction between risk control implementation and verification of effectiveness, it is a matter of scoring. In your risk assessment appropriate risk controls will be identified. Teams can assume effectiveness via the post mitigation score, but verification is lacking. It is best practice to uniquely identify the RC and link it up to design inputs (I always wonder why folks do not use best practice). Once the risk control is implemented, execute the V&V activity and confirm the risk control performs as expected. Again, this is where scoring comes into play. The V&V activity may show areas where further improvement is needed or perhaps the risk control is not as effective as initially believed or the implemented risk control unintentionally broke something else, i.e. collateral damage. Just because a risk control is implemented does not mean it is effective as the scoring indicates. Confirm the reduction in score matches the performance of the implemented risk control.

    ------------------------------
    Adam Atherton
    Farragut TN
    United States
    ------------------------------

    Original Message


  • 7.  RE: Risk Control Implementation and Verification of Effectiveness

    Posted 16-Jan-2020 12:36
    First of all, I wish to state tat there is no such thing as V&V.   The only commonality between Design Verification and Design Validation is that they both contain the letter "V".  These are completely different requirements and activities that have no relationship.  Except perhaps in software, where IEC 62304 lumps them together in a process.  In Design and Development these are separate and unrelated activities.

    The second thing is that ISO 14971 is designed to be integrated into the quality system and that means we developed verification of implementation to be integrated into the Design Verification step of ISO 13485 7.3.6 when following the standard, as the quality management standard also expects that risk control measures will be Design Inputs 7.3.3 (c) and therefore must be verified as implemented in 7.3.6 as Design Outputs in 7.3.4 must be verified as fulfilling Design Inputs.   So in one step required in the quality system standard you are fulfilling a requirement in another standard, ISO 14971 7.2 Implementation of risk control measures.  Separately the same Clause, 7.2 requires verification of effectiveness.  This becomes complicated because the verification of effectiveness, which provides objective evidence that the estimated residual risk after the risk control is implemented, actually results from the implemented risk control.  This verification step can be performed, depending on the method used and the residual risk being verified, in either Design Verification or in Design Validation.  This complicates the discussion of these four activities, Risk Control Verification of Implementation, Verification of Effectiveness, and Design Verification and Design Validation.  It took me a long time, even participating in the development of the two standards to completely understand and separate all these activities.  However, once I got through my thick brain what was happening, it became clear what was being accomplished. 

    Other than the V&V confusion Adam seems to have reached the same point it took me a long time to arrive.  I also teach AAMI Quality Systems and Design Control courses and we try to emphasize that there is no such thing as V&V and they are different activities.

    ------------------------------
    Edwin Bills MEd, CQA, RAC, BSc, CQE, ASQ
    Principal Consultant
    Overland Park KS
    United States
    elb@edwinbillsconsultant.com
    ------------------------------

    Original Message


  • 8.  RE: Risk Control Implementation and Verification of Effectiveness

    Posted 16-Jan-2020 13:00
    We are in violent agreement , Edwin. I appreciate the pushback and need for clarification.

    I completely understand the diff between verification and validation (Like you, I have been teaching and using design controls, risk management, and quality systems for many years). Please pardon the grouping V&V as I didn't want to presume what controls were in place and, therefore whether tweaked requirements needed to be verified and/or affected user needs or intended use addressed via validation/usability. Perhaps an example will help. 

    A simple, dual actuation autoinjector may require pushing the needle end onto the skin prior to activating. If the force required to activate is too high the manufacturer could take steps to reduce the force necessary. Verification would demonstrate the reduced force met the requirement, whereas validation would demonstrate the reduced force is appropriate for the intended user, keeping in mind it was not appropriate previously. In this case, I would want to perform both verification and validation. Certainly, there will be cases where only verification is needed. Indeed, this is the most likely scenario. The manufacturer needs to determine and document the most appropriate course of action.

    Are we aligned, Edwin?

    ------------------------------
    Adam Atherton
    Farragut TN
    United States
    ------------------------------

    Original Message


  • 9.  RE: Risk Control Implementation and Verification of Effectiveness

    Posted 16-Jan-2020 16:31
    Adam,

    I think largely we are in agreement.  I just am sensitive to the differences between verification and validation after having taught the AAMI Quality Systems course for 17 years and seek the confusion.  We kept trying different approaches until today we have a whole set of approaches that spend time discussing the differences from several viewpoints and it seems to get it across.  One diagram that points out one difference is the widely used Health Canada diagram that shows that verification is comparing design output back to design input requirements to see that those requirements are satisfied by the design outputs.  Validation is shown to take the final medical device back to the user needs to show the final device meets those needs.  Two different activities, that may require different techniques to satisfy the requirements of ISO 13485 and the US FDA 21 CFR 820.30.  They are require different personnel with different qualifications to perform the activities.  

    Anyway, I think we agree, just different ways of saying the same thing.  Language gets in the way sometimes, especially American English.  Just ask a Brit!

    ------------------------------
    Edwin Bills MEd, CQA, RAC, BSc, CQE, ASQ
    Principal Consultant
    Overland Park KS
    United States
    elb@edwinbillsconsultant.com
    ------------------------------

    Original Message


  • 10.  RE: Risk Control Implementation and Verification of Effectiveness

    Posted 16-Jan-2020 17:05
    Hi Edwin,

    I think you are referring to the following drawing that is also included in FDA's Design Control guidance doc. I agree it is helpful. Thank you for the exchange.


    ------------------------------
    Adam Atherton
    Farragut TN
    United States
    ------------------------------

    Original Message


Related Content