Regulatory Open Forum

 View Only
  • 1.  Recording quantitative inspection data

    This message was posted by a user wishing to remain anonymous
    Posted 21-Mar-2022 09:44
    This message was posted by a user wishing to remain anonymous

    Does anyone have experience, or other insight, into whether the FDA requires quantitative inspection data?  The QSR Final Rule comments/Preamble (section 147) indicates that "the regulation does not specify quantitative data, but simply requires that the results be recorded."


  • 2.  RE: Recording quantitative inspection data

    Posted 21-Mar-2022 10:16

    You are correct that QSR does not require quantitative data. Unfortunately, there are FDA Warning Letters to device manufacturers for recording only pass/fail, for example, when the inspection step measured the items in the sample. Recording pass/fail is a common practice when using attribute sampling plans such as Z1.4 or c=0.



    ------------------------------
    Dan O'Leary CQA, CQE
    Swanzey NH
    United States
    ------------------------------



  • 3.  RE: Recording quantitative inspection data

    Posted 21-Mar-2022 15:13
    Edited by Kevin Randall 21-Mar-2022 15:20

    I've seen at least four Warning Letters citing firms for recording "pass" without the supporting objective data.  Accordingly, although the QS Regulation preamble comment group 147 certainly does contain the aforesaid narrative, it nonetheless seems to contain a bit of double-speak, or at least language that can be easily misinterpreted.

    Indeed, FDA reminds us therein about the criticality of recording "evidence" that will "clearly show" whether the product has passed or failed according to "defined acceptance criteria".  Remember that with FDA, there's only one kind of "evidence", and that's objective evidence (i.e., that which can be independently verified).  It isn't possible to clearly (i.e., objectively, unambiguously) show conformity with defined acceptance criteria if the record only records "pass" without also recording the supporting data.  We tend to dilute the integrity of our scientific method when we don't record the actual observed results by instead just marking "pass".

    This seems to be the basis behind FDA's history of Warning Letters citing records that only recorded "pass".  It is more likely that when FDA in the preamble said it doesn't specify quantitative data, the agency meant that the objective evidence data may either be quantitative (numeric) or qualitative (such as is the case with visual inspection).

    As an aside for cases where qualitative evidence is appropriate (e.g., visual appearance, color, etc.), it should nonetheless be presumed as a general rule that "pass" won't satisfy the "objective" and "unambiguous" requirement.  Indeed, the aforesaid Warning Letter citations include qualitative inspection scenarios where the firms were cited for recording only "pass" rather than the objective results.

    As another aside, FDA officially defines the acronym "QSR" to mean Quality System Record (i.e., a Quality Manual).  It's best to state "QS Regulation" for the current regulation, or "QMSR" for the new proposed revisions that will align the Regulation with ISO 13485.



    ------------------------------
    Kevin Randall, ASQ CQA, RAC (U.S., Europe, Canada)
    Principal Consultant
    Ridgway, CO
    United States
    © Copyright 2022 by ComplianceAcuity, Inc. All rights reserved.
    ------------------------------



  • 4.  RE: Recording quantitative inspection data

    Posted 22-Mar-2022 06:22
    Indeed quantitative date is expected when available.  Always keep in mind Good Manufacturing Practices or cGMP where quality and regulatory activities evolve over time as the industry progresses in knowledge, awareness, and compliance.  There are many current regulatory agencies: US, Korea, Japan, Brazil who expect if there are quantitative results these are recorded as part of testing activities (development and manufacturing alike).  A pass/fail can be recorded if there is justification for why the quantitative results are not needed to be recorded or retained, such as if is true pass/fail or go/no-go where the value does not have any specific meaning.  Most regulators are quite keen on monitoring and measuring, so the main question asked is how do you monitor and measure if the value is not recorded and analysed?

    ------------------------------
    Richard Vincins ASQ-CQA, MTOPRA, RAC
    Vice President Global Regulatory Affairs
    ------------------------------



  • 5.  RE: Recording quantitative inspection data

    Posted 23-Mar-2022 08:58
    Hi Anon.

    Let me preface this comment with the fact that I am not a device person.  But I think my comment will assist anyone with this quandry.

    My rule of thumb has always been your specification should have 2 separate spaces in the specification - one for the "result" and one for the "disposition" of each test.  That way you enter the "result" (whether that is quantitative or maybe even a simple statement that the product powers on for a device) in the result section and the "disposition" (pass/fail/investigate/etc.) in the disposition section.  I always worry about specifications that commingle these two concepts because there is no way for a reviewer to accurately check the result obtained when the only result is essentially a disposition on the test.

    Even in the drug or cosmetics world where visual inspections are almost always part of the specification for color, form, etc., I am a strong believer that you at minimum need to write down what you saw.  If you simply say "pass" or "fail" you have lost an amazingly important opportunity to trend the inspection results in the future should you begin an investigation (or even to provide trending to your own organization about how the production process is functioning!).  

    I realize that your quality team might not like the idea of having to write down everything that they see or every beep that happens with the machine when it powers up but really that is the proof that the machine is (or is not) functioning as intended and that is really what the idea of testing against a specification is all about - proving that the product at the time of testing was (or was not) functioning properly.  

    I may be in the minority here (although from the other answers my comments seem to mirror the others pretty well) but the fix to this is to not allow the testing team to "short circuit" the data gathering and documentation requirements by simply allowing the team to commingle the idea of test results and test disposition.

    ------------------------------
    Victor Mencarelli
    Global Director Regulatory Affairs
    MelvilleNY
    United States
    ------------------------------



  • 6.  RE: Recording quantitative inspection data

    Posted 23-Mar-2022 10:04

    The question here is the regulatory requirement in QSR and is NOT expectation, best practice, etc. When I get a question in this area it is usually the result of an FDA Inspection in which the FDA Investigator wants to issue a 483 observation for recording pass/fail instead of actual measurements when performing attribute sampling.

    This is another example of a Quality Auditor or FDA Investigator writing a nonconformance because the manufacturer didn't implement the auditor or Investigator's opinion on how to do something. This is especially troublesome for an FDA Investigator, because a 483 observation is an allegation of criminal activity – failure to follow the FD&CA.

    As a case study consider the July 22, 2009 Warning Letter to Ohio Medical Corporation for failure to document acceptance activities as required by 21 CFR §820.80(e). (I was not involved in this, but use it in my course.)

    "For example, in March 2009, 464 incoming bottles … were received, visually inspected, and measured for wall thickness. There is no documentation to show inspectional results including values for measured wall thickness and results of the visual inspection; results are simply reported as passing."

    In this case the Investigator failed to follow QSR and, instead, "invented" a requirement and then accused the firm of failure to follow the Investigator's expectation.

    The firm should have objected at the Inspection closing meeting, but in my experience the Investigator seldom changes the 483 at that point.

    In responding to the 483, the firm should have pointed out the error in the observation.

    Most likely, the firm instead changed its procedures, created a lot of non-value work, and did not improve the medical device as a result.

    I'm an advocate of data, data analysis, etc. and always recommend that companies collect the data and analyze it. However, it is not a regulatory requirement to record the measured values, only to record the pass/fail status.

    In this case, it appears the incoming inspection looked at all the bottles, found them all in specification, and accepted the lot. There is no requirement to record even the pass/fail for each bottle, let alone the actual measured value. They could have been sorted at the inspection station and recorded the final results: 464 inspected and 464 (100%) passed.



    ------------------------------
    Dan O'Leary CQA, CQE
    Swanzey NH
    United States
    ------------------------------



  • 7.  RE: Recording quantitative inspection data

    Posted 28-Mar-2022 19:29
    Edited by Kevin Randall 28-Mar-2022 19:32
    Remember that FDA's QS Regulation makes a very clear regulatory distinction between recording pass/fail status (i.e., 820.86) as distinguished from objectively recording the inspection results (i.e., 820.80) on which the pass/fail status is based.  It is no coincidence that the QS Regulation's 820.80 (recording of inspection results, among other requirements) immediately precedes, and is fundamentally separate from, 820.86 (recording pass/fail status).

    Accordingly, if a firm unwisely chooses to argue with FDA by asserting that there is no regulatory requirement for recording inspection results distinct from inspection status, then that would surely exacerbate an already difficult Form FDA 483 or Warning Letter scenario.

    On the other hand, despite a scary and often misunderstood reputation, remember that FDA's primary goal via a Form FDA 483 or Warning Letter is to achieve voluntary compliance as a means of avoiding more formal enforcement actions such as criminal investigations.  While Form FDA 483 Observations and Warning Letter citations might record opinions and evidence that FDA could eventually, yet very rarely, and only very carefully, if at all, be used by the agency in later efforts to establish criminality, those observations and citations are not automatically considered allegations of criminal activity.  At best they could be considered a means of "prior warning" before criminal investigations begin.  FDA's Regulatory Procedures Manual (RPM) is recommended reading to help folks realize a proper understanding of such matters.

    So, don't let such sweeping declarations frighten you.  For example, FDA's inspectional policy is that, in no circumstance, should an FDA inspection (i.e., most common source of Form 483 Observations) be conducted solely to obtain evidence to support a possible criminal case.  In other words, FDA inspections are to be principally triggered based on the routine monitoring provisions of the FD&C Act; not by a need to collect evidence where the agency is contemplating criminal action.

    When criminal investigation is involved, an entirely different FDA Office is the lead rather than FDA's Office running establishment inspections.  Though 483s and Warning Letters may well document evidence that the subject device or firm isn't in compliance with medical device law (i.e., the FD&C Act), it rarely means that such nonconformity involves criminal activity.  Instead, criminal activity is a much higher, and fundamentally different bar, than a failure to fully comply with, for example, the FDA's QS Regulation.

    For example, if one used FDA's regulatory acronym "QSR" (i.e., Quality System Record, i.e., a Quality Manual) to identify FDA's quality system regulation, then that wouldn't generally be a criminal activity; instead, it would just represent a nonconformity with the QS Regulation.

    ------------------------------
    Kevin Randall, ASQ CQA, RAC (U.S., Europe, Canada)
    Principal Consultant
    Ridgway, CO
    United States
    © Copyright 2022 by ComplianceAcuity, Inc. All rights reserved.
    ------------------------------