Regulatory Open Forum

I infer the question comes down to, "How do I know which row in the table to use?" I also infer you are not a statistician.

This is a drug-device combination product, but the drug does not enter into the sample size question. The guidance document is about the device component only. Appendix A(i) sets up the situation.

You have a drug delivery device that you want to compare to a drug delivery device already on the market.

For the one on the market, there is an error rate, use error, across the whole population of users, but the rate is unknown. You would like to show that the use error of your device is not worse than the other device. This is an inferiority study, because you want to show that your device is not inferior to the other device.

Start by conducing the threshold analysis using the three factors: Labeling comparison, Comparative task analysis, and Physical comparison of the delivery device constituent part.

After conducting the analysis, reach one of two conclusions: No design differences or Differences in design.

If the conclusion is "No design differences", then you are done. There is no need to run the statistical test.

If the conclusion is "Differences in design" then reach one of two sub-conclusions: Minor design difference or Other design differences.

If the sub-conclusion is "Minor design difference" then you are done. There is no need to run the statistical test.

If the sub-conclusion is "Other design differences", then you need to show that the design differences don't increase the use error compared to the one the market. You need to run the statistical test.

Using the notation from the draft guidance document, compare the use error rate of the other device, ER_R, with the use error rate of your proposed design, ER_T.

The problem is that you don't know either use error rate. For the test, you will use a sample. By the luck of the draw the use error rates in the sample you use will, likely, not be the same as the actual population of users.

First, segment user population. Are the users doctors, nurses, home health care providers, the patient, etc. Also determine the user location such as a hospital, clinic, or home. These combinations could change the use error rate, so you need to determine which apply. You will need to conduct a separate test for each applicable segment.

By the luck of the draw, the use error rate observed may be different from the actual use error rate. You need to determine how big a difference you would consider as the same. For example, assume the "true" error rate (which is unknown) is 5% and you observe 7%. Would you consider them the same, because of sampling? The draft guidance document calls this d.

Run the test and analyze it using a hypothesis test. A hypothesis test can make two kinds errors, simply put. Saying two things are the same when they are different or saying two things are different when they are the same. Sample size helps control these errors.

The text before the table says, "The desired sample size for each user group population or set of circumstances will be a function of the assumed use error probability, the within subject correlation, and statistical power to rule out the chosen d."

Translating into English:
"Sample size for each user group population" means the number of users in each segment. If the use is a home health care professional in a home setting, that is one user group population. If the use is a lay patient in a home setting, that is one user group population.

"A function of the assumed use error probability" – Each user group population will have a different use error probability. In the home setting we would expect a lower use error probability for home health professional as compared to lay patient.

"Within subject correlation" means that correlation of each person in the sample to make the same error on both devices.

"Statistical power to rule out the chosen d" means the ability of the statistical test to help recognize differences between the two devices.

The footnote to the table includes important information. It says a=0:05. This probably should have been α = 0.05, which is the Type I error. It also says d=0:10 which probably should have been d=0.01. Also notice that the table uses simulation, not direct comparison.

For a worked example, consider the home health professional in a home use setting.

Assume the use error rate is 10.0% and we are willing to say that 11.0% counts as the same. This means d = 1.0% or 0.01.

We expect the design is very similar to the other device to the use errors each subject makes tends to be the same, a within subject correlation of 0.90.

We want larger powers for better discrimination of differences, so let's say 85.5% or 0.85.

This leads to the first row in the table. We want 45 home health care professionals to each use both devices.

Assume another segment is the lay person in a home setting. We assume the error rate is 15.0% and we are willing to say that 16.0% counts as the same; d = 1.0%. For the same correlation and power, the table gives no help since it doesn't have this combination.

I infer the question comes down to, "How do I know which row in the table to use?" I also infer you are not a statistician.

This is a drug-device combination product, but the drug does not enter into the sample size question. The guidance document is about the device component only. Appendix A(i) sets up the situation.

You have a drug delivery device that you want to compare to a drug delivery device already on the market.

For the one on the market, there is an error rate, use error, across the whole population of users, but the rate is unknown. You would like to show that the use error of your device is not worse than the other device. This is an inferiority study, because you want to show that your device is not inferior to the other device.

Start by conducing the threshold analysis using the three factors: Labeling comparison, Comparative task analysis, and Physical comparison of the delivery device constituent part.

After conducting the analysis, reach one of two conclusions: No design differences or Differences in design.

If the conclusion is "No design differences", then you are done. There is no need to run the statistical test.

If the conclusion is "Differences in design" then reach one of two sub-conclusions: Minor design difference or Other design differences.

If the sub-conclusion is "Minor design difference" then you are done. There is no need to run the statistical test.

If the sub-conclusion is "Other design differences", then you need to show that the design differences don't increase the use error compared to the one the market. You need to run the statistical test.

Using the notation from the draft guidance document, compare the use error rate of the other device, ER_R, with the use error rate of your proposed design, ER_T.

The problem is that you don't know either use error rate. For the test, you will use a sample. By the luck of the draw the use error rates in the sample you use will, likely, not be the same as the actual population of users.

First, segment user population. Are the users doctors, nurses, home health care providers, the patient, etc. Also determine the user location such as a hospital, clinic, or home. These combinations could change the use error rate, so you need to determine which apply. You will need to conduct a separate test for each applicable segment.

By the luck of the draw, the use error rate observed may be different from the actual use error rate. You need to determine how big a difference you would consider as the same. For example, assume the "true" error rate (which is unknown) is 5% and you observe 7%. Would you consider them the same, because of sampling? The draft guidance document calls this d.

Run the test and analyze it using a hypothesis test. A hypothesis test can make two kinds errors, simply put. Saying two things are the same when they are different or saying two things are different when they are the same. Sample size helps control these errors.

The text before the table says, "The desired sample size for each user group population or set of circumstances will be a function of the assumed use error probability, the within subject correlation, and statistical power to rule out the chosen d."

Translating into English:
"Sample size for each user group population" means the number of users in each segment. If the use is a home health care professional in a home setting, that is one user group population. If the use is a lay patient in a home setting, that is one user group population.

"A function of the assumed use error probability" – Each user group population will have a different use error probability. In the home setting we would expect a lower use error probability for home health professional as compared to lay patient.

"Within subject correlation" means that correlation of each person in the sample to make the same error on both devices.

"Statistical power to rule out the chosen d" means the ability of the statistical test to help recognize differences between the two devices.

The footnote to the table includes important information. It says a=0:05. This probably should have been α = 0.05, which is the Type I error. It also says d=0:10 which probably should have been d=0.01. Also notice that the table uses simulation, not direct comparison.

For a worked example, consider the home health professional in a home use setting.

Assume the use error rate is 10.0% and we are willing to say that 11.0% counts as the same. This means d = 1.0% or 0.01.

We expect the design is very similar to the other device to the use errors each subject makes tends to be the same, a within subject correlation of 0.90.

We want larger powers for better discrimination of differences, so let's say 85.5% or 0.85.

This leads to the first row in the table. We want 45 home health care professionals to each use both devices.

Assume another segment is the lay person in a home setting. We assume the error rate is 15.0% and we are willing to say that 16.0% counts as the same; d = 1.0%. For the same correlation and power, the table gives no help since it doesn't have this combination.