Thanks Kevin for your in depth answer. I think we will have to disagree on this one. but I think we both agree 100% on the requirements to have your test method qualified.
I also would not combine calibration with test method validation, they are very different and would not sit in the same part of my QMS
Under 820.75, it states that were a process can not be fully verified, it must be validated. Why TMV does not sit comfortable there is that test methods are verification methods. We use them to verify a product specifciation. So if we consider a test method asa process under 820.75, we are considering verification of a verification. it does not really fit.
if you look at the GHTF guidance for process validation as well, it gives guidance on making decision to whether conduct process validation and it bases this decision on the process output. For a test method, there is no product output, its a method to verify the output.
820.72(a) states "suitable for its intended purposes and is capable of producing valid results".
its that requirement to show our method meets it intended purposes that drive us to qualify our test method. And the "capable"requirement drives us to conduct Gage R&R etc.
I agree that there are many examples of warning letters that link test methods to 820.75 but just does not fit there comfortably. I actually teach this subject too, and many times with the FDA. And we do cover TMV under 820.72.
Although we might differ, I think the most important thing is that we both agree on is the need to assess your test methods and demonstrate that they meet your needs
Seb
------------------------------
Sebastian Clerkin
Consultant
Cork
Ireland
------------------------------
Original Message:
Sent: 29-Jul-2021 20:30
From: Kevin Randall
Subject: Test Method Validation
Thanks, Sebastian for that explanation. Very interesting indeed.
My experience has been that FDA views a test method as a process that produces an output. And if such output can't be fully verified, then it must be validated. Since 820.72 is not (to my imperfect knowledge) related to "validation" (a very specialized, very robust, very deliberate term defined by 820.3), I've seen FDA handle process validation (be it for a production process yielding a tangible physical output or for a test method producing a data output) via 820.75. For example, in a February 19, 2002 Warning Letter, FDA states, "...[we]...collected information that revealed serious regulatory problems...as follows:...Failure to validate with a high degree of assurance a process that cannot be fully verified by subsequent inspection and test as required by 21 CFR 820.75(a). For example:...The testing procedure used for...has not been validated..."
I would be very hesitant to try and integrate the principles of "validation" into my instrument metrology program. That's not something I've seen before in the many metrology programs I've encountered. FDA and its explanations of 820.72 are (to my imperfect knowledge) aimed at assuring metrological adequacy of instruments rather than the adequacy of test/measurement processes.
Another way to put my experience is that 820.72 is related to the principles embodied by ISO ISO 10012 / EN ISO 10012 clause 7.1 (metrological confirmation, i.e., "calibration"), whereas "validation" is instead related to the principles embodied by that standard's clause 7.2 (measurement process development and validation). Those sections provide a nice distinction between instrument metrology vs. measurement process (e.g., test method) validation. They show that clause 7.1 (and I would say 820.72) is to ensure that metrological instruments are fit for their intended use, whereas 7.2 (and I would say 820.75) ensure that a process (e.g., a measurement process, i.e., a test method) is fit for the process's intended use.
Because of these principles, my experience is that FDA generally associates test method validation with 820.75. As mentioned above, I've seen FDA issue test method validation audit citations against 820.75 rather than 820.72. It seems FDA demands that we perform test method validation pursuant to 820.75 rather than 820.72. If we applied the principles of 820.72 for test method validation, then I would get ready for a real wallop from FDA and other regulators like ISO auditors.
------------------------------
Kevin Randall, ASQ CQA, RAC (U.S., Europe, Canada)
Principal Consultant
Ridgway, CO
United States
© Copyright 2021 by ComplianceAcuity, Inc. All rights reserved.
Original Message:
Sent: 29-Jul-2021 15:55
From: Sebastian Clerkin
Subject: Test Method Validation
Hi Kevin,
no I meant 820.72. 820.75 refers to process validation. Process validation refers to process that produce an output. this is an output of the manufacturing process and is usually to our product specifications. For example it could be a dimensional characteristic, such as the outer diameter, or maybe a material characteristic of the actual product itself.
820.72 refer to both the gauges that we would use too monitor/control our manufacturing process such as a pressure gage or thermocouple, and also the test methods that we would use to monitor/verify the outputs of our manufacturing process.
For the gauges that monitor/control processes, usually calibration is sufficient. But for methods that are used to measure a key characteristic of our product, calibration by itself would be insufficient. we need to doing further qualification.
hope that helps
Seb
------------------------------
Sebastian Clerkin
Consultant
Cork
Ireland
Original Message:
Sent: 29-Jul-2021 11:02
From: Kevin Randall
Subject: Test Method Validation
Good comments Sebastian. Question: When you said 820.72, did you instead mean 820.75? The FDA uses 820.72 in regards to metrology controls for measurement equipment, whereas FDA associates test method validation with 820.75.
------------------------------
Kevin Randall, ASQ CQA, RAC (U.S., Europe, Canada)
Principal Consultant
Ridgway, CO
United States
© Copyright 2021 by ComplianceAcuity, Inc. All rights reserved.
Original Message:
Sent: 29-Jul-2021 04:04
From: Sebastian Clerkin
Subject: Test Method Validation
Hi Shimon,
you are getting lots of different responses, which is reflective of the fact that there is lots of different types of test methods.
In the med device sector, we can break them down into three main groups.
- Compendial or test methods that meet a specific standard.
- Analytical test methods
- Physical test methods.
Each of these have different strategies to validate. For example, for test methods that are based on a standard, i would look to demonstrate that these methods are equivalent to the standard. For physical test methods, you would be looking to use statistics such as Gage R&Rs
Per the regulations 21 CFR part 820.72, you must show your test method meets it intended use. so you will need some level of qualification for any test methods used. As part of any validation, also look to see if you can identify a worst case that will cover multiple product codes.
let me know if you need more info or pointers to publications on this topic
Seb
------------------------------
Sebastian Clerkin
Consultant
Ballincollig
Ireland
Original Message:
Sent: 28-Jul-2021 09:26
From: Shimon Vaknin
Subject: Test Method Validation
Thanks Kevin!
Original Message:
Sent: 27-Jul-2021 15:35
From: Kevin Randall
Subject: Test Method Validation
Note also the FDA's more recent method validation guidance attached (from the pharma side, but the basic principles of which are useful for the device industry).
In general, my understanding is that any analytical test method, whether for devices or drugs, must be proven (validated) to be suitable for its intended use. For example, the U.S. FDA (including via device Warning Letter) and AAMI have a history of including test methods amongst the processes that are candidates for requiring validation. The only filter of which I'm aware is that some methods may be officially recognized (such as via the USP) and thus may not be subject to additional validation. But the others (so called "alternative" methods like custom methods developed in-house) would generally seem to require validation.
------------------------------
Kevin Randall, ASQ CQA, RAC (U.S., Europe, Canada)
Principal Consultant
Ridgway, CO
United States
© Copyright 2021 by ComplianceAcuity, Inc. All rights reserved.
Original Message:
Sent: 27-Jul-2021 14:26
From: Kevin Randall
Subject: Test Method Validation
Many years ago I wrote a white paper on the topic, focused primarily on the medical device sector, but taking its roots from the pharma sector. I've attached it in case it's of value to the Forum.
------------------------------
Kevin Randall, ASQ CQA, RAC (U.S., Europe, Canada)
Principal Consultant
Ridgway, CO
United States
© Copyright 2021 by ComplianceAcuity, Inc. All rights reserved.
Original Message:
Sent: 23-Jul-2021 08:41
From: Shimon Vaknin
Subject: Test Method Validation
Hi Colleagues,
I recently saw a proposal from a consulting firm to do test method validation for one former client of mine.
The plan looks like a very big project (19 TMV's) and would take months to complete.
My questions:
1. How do you determine when TMV is necessary? what has been your experience / best practice.
2. Can you avoid using fancy statistics (Gage R&R / ANOVA)? no offense intended for statisticians 
3. Can you do one TMV report for a product line instead of multiple TMVs which would take much longer?
Thanks in advance.
Shimon