Regulatory Open Forum

 View Only
  • 1.  Use of prototypes for Design Verification and Design Validation

    Posted 20-Aug-2015 16:41

    Fellow RAPS members,

     

    I would like your opinion on how "final" devices need to be for Design Verification testing and Design Validation testing.  I have always gone by the rule that Design Verification testing needs to be conducted on final design equivalent devices and that Design Validation testing needs to be conducted on final design and process equivalent devices, and any differences in the design or manufacturing process between the final marketed device and the devices that were tested will need to be documented and need a justification why the difference don't impact the testing.

     

    In 21 CFR 820.30(g) it requires the use of initial production devices or their equivalents for design validation.  

    Does IQ/OQ/PQ of equipment need to be completed before building Design Validation units?

    Is there any similar regulation for Design Verification units?  

    What is the expectation of a manufacturer when justifying equivalence?

    Are there any requirements in ISO 13485 regarding this issue?

     

    Thank you for your help,

     


    ------------------------------
    Andrew Rodenhouse
    ------------------------------



  • 2.  RE: Use of prototypes for Design Verification and Design Validation

    Posted 20-Aug-2015 19:42

    I usually start any discussion of design verification with Warning – Heresy Alert!! My recommendations are not in line with current thinking, and, I expect, is the genesis of your question.

    You say that you go by the “rule that Design Verification testing needs to be conducted on final design equivalent devices”. I disagree that this is the appropriate rule. First you don’t, necessarily, need to do conduct design verification testing, nor is there any requirement that when you choose testing it must be from initial product units. Recall the question answered by design verification; does the design output meet the design input requirements. There are many ways to do this including document review, comparison with previous designs, and alternate calculations.

    Here is a simple example of “document review” that involves neither testing nor initial product units. The design input is “The color of the molded plastic case must match the corporate branding document for color, i.e., Pantone xxxxx. The design output is a drawing that defines the molded plastic case, and, in particular, specifies the color of the case as Pantone xxxxx.

    In design verification I read the input and locate it in the associated output. If they align, the design verification is successful. If not, there is a nonconformance that must be resolved. Notice that, in this simple case, I never had to produce a molded plastic case. The results would state how I plan to perform the design verification, the identification number of the design input, the identity of the drawing (number and revision), and the location of the color information (e.g., Note 3, feature in Zone D5, etc.).

    The recent trend confuses manufacturability with design verification. This leads them to make “production” units, invoke elaborate formulas for sample size, etc. In many cases, this is not necessary, ergo my heresy.

    To continue the example, assume production of the molded plastic case is an outsourced process. The drawing (design output) moves to purchasing, is approved as a document under 820.40 and becomes purchasing data under 820.50(b). Supplier management identifies and selects a suitable supplier under 820.50(a).

    Supplier management issues a purchase order for the molded plastic case. The supplier produces and ships them.

    In parallel, the responsible person sets up the receiving acceptance activities under 820.80(b). In this case, the receiving inspection document requires product verification for the case, defines the verification method, and the number to verify (100% of all units received, a sampling plan, certificate of conformance, etc.)

    In the end we know a few very important things:
    a) The design is correct, based on the design verification in 820.30(f)
    b) The supplier requirements (purchasing data) are correct based on the purchasing data approval in 820.40
    c) The material received from the supplier and put into stock is correct based on the incoming acceptance activities in 820.80(b).

    I’ll address Design Validation in a separate post.

    ------------------------------
    Dan O'Leary
    Swanzey NH
    United States
    ------------------------------




  • 3.  RE: Use of prototypes for Design Verification and Design Validation

    Posted 21-Aug-2015 03:16

    Dan, please correct me if I am wrong, is the point it depends what requirement you are aiming to verify. The simple example you highlighted can be verified by review. However, a couple of examples that probably can’t

    • 'the device must function as intended after a drop from 1m onto a surface’ (I know this requires more detail). It may be possible to verify by analysis (math model) or simulation (finite element analysis) but you will need to validate these tools. So the question is what manufacturing process is acceptable for the manufacture of DV parts, if ultimately the marketed device is going to be moulded then rapid prototype or SLA parts are still not representative enough, therefore pilot tools / process is the minimum requirement
    • Verification of performance after 2 year shelf life or xx months in use, verified through the testing of accelerated and/or RT aged devices. Again it will be very difficult to demonstrate that rapid prototype parts or SLA are representative.   

    WRT ISO 13485 the current version does not go into that level of detail. However it is believed that ISO 13485 3rd edition due to be published early 2016 will provide more detail regarding design verification  

     

    ------------------------------
    Mark Di Cioccio
    Cambridge
    United Kingdom
    ------------------------------




  • 4.  RE: Use of prototypes for Design Verification and Design Validation

    Posted 21-Aug-2015 07:54

    It is a strange world where objective, logical thinking and common sense are considered heresy! 

    I fully agree with your assessments.  I will also add that there are some tests that can be done on prototypes and even subassemblies before the final design is complete that can be valid, and used as Verification for the finished design.  Design verification assessments and testing are a continuous and iterative process.  In fact, the results of some Design Verification testing become "inputs" into the design during the design process. It is not always necessary, nor desirable to wait until there are "validation" lots to confirm that the Design Inputs have been met. 

    On the other hand, when it comes to those attributes that can only be verified by testing a fully manufactured finished product, Andrew's approach is reasonable and supported by the regulation. The devices tested can be hand assembled prototypes, if the company can justify that the difference to the commercial product do not obviate the test results.

    Lee 



    ------------------------------
    Lee Leichter RAC
    President
    P/L Biomedical
    Fort Myers FL
    United States
    ------------------------------




  • 5.  RE: Use of prototypes for Design Verification and Design Validation

    Posted 21-Aug-2015 22:37

    Dan, I'm more inclined to say that you absolutely do need to do design verification testing, and then attribute the "heresy" problem to the mindset that there is only one kind of test, and it's an engineering test.  Apparently a lot of people have managed to find their way into the industry without ever having taken a single test in school. :)

    ------------------------------
    Julie Omohundro RAC
    Durham NC
    United States



  • 6.  RE: Use of prototypes for Design Verification and Design Validation

    Posted 21-Aug-2015 09:09

    I agree basically with earlier comments. I also do not understand the perceived need for final units to carry out most verification testing (and do not consider this to be heresy). It may be useful to think of the verification as part of a chain of documentation that is usually useful in the design process. The inputs, often stated in general terms (but which should be measurable themselves) are turned into specifications, which are more detailed, and each of which is verifiable by testing. A trace matrix should track each input into the resulting specifications and each of those specifications will in turn be traced into the appropriate verification tests. The great majority of those tests will not require the finished device, although as noted in earlier comments, some may require a complete device for that purpose. The trace matrix will include the validation information as well, but normally a small number of validation tests will serve to validate a large number of specifications.

    In justifying equivalence the first consideration is that the manufacturer must itself be convinced of the equivalence. The manufacturer knows the device better than anyone else, and therefore the first test is whether you are yourselves convinced of the equivalence. There should be objective evidence that you can provide to an inspector that you base this opinion on.

    With regard to IQ/OQ/PQ there is a risk-based decision involved. If the IQ/OQ/PQ is not completed before building the design validation units and then in completing the equipment validation there is a problems, you will have to deal with the possibility that your design validation has been jeopardized by using units that may not represent the final product. After you fix the equipment issues you may have to redo the design validation. On the other hand, if your equipment validation goes as planned, you will have saved time in your design process by doing the design validation at least partly concurrently with the equipment validation.

    As may be expected, there is less prescriptive detail in ISO 13485 than in 820. The DIS2 of the new 13485 does have some more detail than the current. It requires plans for both verification and validation to include methods, acceptance criteria and sample size. It also requires for both v and v to include testing related to interfacing with another device if it is intended to be used with that other device. The final will probably be like the DIS2. However, it does not seem as if these details impinge significantly on the questions you are raising.

    ------------------------------
    William White
    Senior Consultant
    Quality System Strategies LLC
    Elkhart IN
    United States
    ------------------------------




  • 7.  RE: Use of prototypes for Design Verification and Design Validation

    Posted 21-Aug-2015 11:10
    Andrew,

    Practical perspectives have been shared by others. 

    From what I have observed and what I think it is important is to make sure:
    • we understand key definitions - "finished device," "design verification," and "design validation" within the meaning and scope of 21 CFR 820.3, 21 CFR 820.30(f) and 21 CFR 820.30(g)
    • we adequately translate our understanding into our actual practice during device development
    Design Verification can be done at any given time, when appropriate and necessary, during device development as we are trying to ensure whether "we are designing our device in a right way" using materials, sub-assemblies, assemblies, etc.

    Design Validation can be done at some points using sub-assemblies, assemblies, prototypes, finished devices, etc as we are trying to ensure whether "we made the right device/product (e.g., subassemblies, assemblies, prototypes, finished device)." 

    However, to best address whether device specifications meet the user needs/intended uses as a device system, it would be best achieved when the finished device ("capable of functioning" or "suitable for use" even as a prototype) is used (e.g., under defined operating conditions on initial production units, lots, or batches, or their equivalents (prototypes) unless justified otherwise. 

    Please have a chance to look at common citations concerning design control at http://regulatorydoctor.us/common-design-control-citations-in-483s/

    For anyone's interest, my design control training seminar titled "Demystifying Design Control Including DHF, DHR, and DMR."

    s/ David
    ____________________________________________________________
    Dr. David Lim, Ph.D., RAC, ASQ-CQA 
    President and CEO | REGULATORY DOCTOR and GCS
    Phone (Toll-Free): 1-(800) 321-8567


    "Knowledge is power only when it is practiced and put into action." - Regulatory Doctor

    NOTICE: This communication (including any attachments) may contain privileged or confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this communication and/or shred the materials and any attachments and are hereby notified that any disclosure, copying or distribution of this communication, or the taking of any action based on it, is strictly prohibited.





  • 8.  RE: Use of prototypes for Design Verification and Design Validation

    Posted 21-Aug-2015 17:43

    This is my second installment in my response. In this case, I’m responding to the design validation issues.

    You ask, “Does IQ/OQ/PQ of equipment need to be completed before building Design Validation units?” This question is, on its face, full of issues.

    First, in QSR there is no requirement to perform IQ/OQ/PQ. Instead, this is one of ways to perform process validation. While it is not the only approach, it is by far the most common.

    Second, IQ/OQ/PQ is not performed on equipment, but on processes. The IQ portion tends to focus on equipment, but it is redundant with 820.70(g).

    As a result, I’ll rephrase the question as, “Does process validation need to be completed before building design validation units?” The short answer is NO. Consider the role of process validation. It provides a “high degree of assurance” that the process produces conforming product when the product cannot be (destructive test) or is not (sampling plan) verified. Before using an initial production unit for design validation, one should ensure it is conforming. As a result, the question of process validation does not arise.

    You ask, “What is the expectation of a manufacturer when justifying equivalence?” The best answer comes from the QSR preamble where section 81 says, “When equivalent devices are used in the final design validation, the manufacturer must document in detail how the device was manufactured and how the manufacturing is similar to and possibly different from initial production. Where there are differences, the manufacturer must justify why design validation results are valid for the production units, lots, or batches.”



    ------------------------------
    Dan O'Leary
    Swanzey NH
    United States
    ------------------------------




  • 9.  RE: Use of prototypes for Design Verification and Design Validation

    Posted 21-Aug-2015 22:58

    Some excellent thinking and insights here.

    I agree with this statement:  "any differences in the design or manufacturing process between the final marketed device and the devices that were tested will need to be documented and need a justification why the difference don't impact the testing."

    Your verification and validation results must support the device that is marketed. This does not mean that the devices used for verification and validation must be identical to the device that is marketed.  It does mean that you need to be able to show why they are adequate support the marketed device in spite of any differences.

    So my answer to this question has always been that it is entirely up to you.

    Sometimes there seems to be no compelling reason to wait for final design, especially when the particular output you are verifying is unlikely to change in the final design. Other times, I think this is a matter of "pay me now or pay me later"...a desire to (appear to) move forward faster by not waiting for final design, mostly due to financial and other external pressures. But sometimes you can lose as much or more development time addressing all the changes on the back end as you would have lost by simply waiting for final design on the front end.  

    And of course there is always the risk that, after the dust settles from a rush to verify and validate, you find that you can't actually provide a good justification as to why the verification and validation results still support the marketed device, because they don't.

    I think Dan's point about confusing design verification with manufacturability is well taken. I have seen a similar tendency to confuse design validation with process validation. Based on other things I've seen, I wonder how much of this might due to a tendency to confuse manufacturing engineers with design engineers.



    ------------------------------
    Julie Omohundro RAC
    Durham NC
    United States
    ------------------------------