HI Trina.
I think Dan's point here is key to most of this discussion. Think about it logically. If you have a set-point of 80% confidence your p-value (or the likelihood of getting acceptable results randomly) is so high as to make it almost as though you were throwing a 5 sided die and getting 1 on the roll. It should happen close to 20% of the time, and that is what your confidence is. So that means that 1 in 5 units could fail to meet your criteria for validating your design.
Additionally, in regard to David's comment, complaints are a problem and can be rather time consuming for the company to manage, investigate, report and potentially explain to auditors. However, they are also very hard to explain to juries, especially in light of the fact that an attorney would likely be able to find statisticians lining up to make a few additional dollars by berating the lack of corporate morality by accepting such a low bar for proving the product is acceptable. So I would not worry so much about whether or not the device will receive clearance from FDA on substantial equivalence grounds. I would be far more worried about whether or not I could pass the "red face" test in a deposition or in a trial court before a jury trying to explain why 1 in 5 devices not conforming is considered acceptable.
------------------------------
Victor Mencarelli
Sr. Manager - Regulatory Affairs
Hain Celestial Group
United States
Original Message:
Sent: 04-25-2016 06:23
From: Dan O'Leary
Subject: Confidence & Reliability levels
This idea of confidence & reliability has crept into the literature over the past few years as a method to determine sample size for a hypothesis test. The sample size helps determine the alpha and beta risks. Often it involves a risk classification (major, minor, and critical) that leads to different values and therefor different sample sizes.
To the best of my knowledge there are no regulatory requirements in the US, EU, or Canada that requires this technique. However, ISO 13485:2016 asks for the statistical basis for sample size in design verification and design validation.
Design verification demonstrates that design output matches design input. It is rare that this activity requires a sample. The problem, in my opinion, is that people confuse manufacturability with design validation.
Design validation demonstrates that the device meets intended use and user needs. Here the sample size question is more complex and typically is not amenable to the confidence & reliability technique.
The 510(k) submission demonstrates substantial equivalence to a predicate device. The implication is that your claim of substantial equivalence requires a hypothesis test, and you wish to demonstrate adequate sample size. This depends, in part, on the results. Using a confidence value of 80% means that you set the p-value at 10%. Most practitioners would consider this value too high and would expect 5% or 1%.
------------------------------
Dan O'Leary
Swanzey NH
United States