Regulatory Open Forum

This message was posted by a user wishing to remain anonymous

I am looking to obtain information on how others are trending complaints for medical devices that arise from social media. Our company has a PMA device with an active social media source. Despite requesting that the poster provide contact us privately in order for us to obtain additional information, we are usually unsuccessful at obtaining this information therefore making it impossible to determine if the complaint has been previously reported through either a healthcare provider or through a post-market registry, and cannot obtain any information as to the timing of the event post-implant or details surrounding the event. 

The adverse events reported through the social media source are usually serious in nature and result in MDR reports being filed. My question to others is how do you account for these events when a malfunction of the device is not reported but are impacting your risk thresholds without the ability to determine the relationship to device? We are struggling with sending a signal that may not be an indication of our device performance, as well as us as the manufacturer having a true picture of device performance. 

Thank you in advance for responding.

This problem does not have a great solution. It is the inverse of the underreporting problem. For some devices many complaints are not reported, meaning the manufacturer underestimates the complaint rate. Your problem is that you may have double reporting which overestimates the complaint rate.

I recommend a few approaches. In the first, make a reasonable estimate of the overreporting rate and use it as a deflator. It might be different for different kinds of issues. Compare the deflated rate with the estimated residual risk frequency of occurrence from your ISO 14971:2019 risk management file to detect a signal.

Compare the complaint rate over time, say monthly for a 12 month window, to look for an increase. The easiest way I know is to calculate the linear regression line of the complaint rate and see if the  slope is significantly different from zero. Regression in the Excel Analysis Tool Pack will do this quickly (five minutes to set it up and get the result.)

Use FDA TPLC database to check the rank order of patient problems over time. If a patient problem moves up (more frequent) two or more places, then that generates a signal. You may need to remove your company's MDRs to get a picture of the rest of the industry.

------------------------------
Dan O'Leary CQA, CQE
Swanzey NH
United States
------------------------------

Original Message:
Sent: 13-Oct-2022 21:04
From: Anonymous Member
Subject: Social Media Complaints and Trending

This message was posted by a user wishing to remain anonymous

I am looking to obtain information on how others are trending complaints for medical devices that arise from social media. Our company has a PMA device with an active social media source. Despite requesting that the poster provide contact us privately in order for us to obtain additional information, we are usually unsuccessful at obtaining this information therefore making it impossible to determine if the complaint has been previously reported through either a healthcare provider or through a post-market registry, and cannot obtain any information as to the timing of the event post-implant or details surrounding the event.

The adverse events reported through the social media source are usually serious in nature and result in MDR reports being filed. My question to others is how do you account for these events when a malfunction of the device is not reported but are impacting your risk thresholds without the ability to determine the relationship to device? We are struggling with sending a signal that may not be an indication of our device performance, as well as us as the manufacturer having a true picture of device performance.

Thank you in advance for responding.

Hello Anon,

You are certainly in an interesting, difficult, and evolving situation with social media information as part of your post market system whether they are just feedback or indeed complaints.  Word of advice, remove the social media outlet.  As much as your marketing or management would love to have this, the problems which come with it usually outweigh the benefit.  Or make the social media outlet absolutely two-way - meaning you can provide information to your customers, but in order for them to give feedback, you do need to all of their contact information.  The FDA and other regulatory agencies are well aware of social media and want to see how companies manage these activities.  Just look at some of the Warning Letter or show them to your management, there are a couple really good examples there, where the process under social media needs to be managed very carefully.  Also recommend within your complaint management system having a well established and implemented process for how feedback and complaints are managed, gathered, investigated, and responded within your post market system.

------------------------------
Richard Vincins ASQ-CQA, MTOPRA, RAC
Vice President Global Regulatory Affairs
------------------------------

Original Message:
Sent: 14-Oct-2022 10:33
From: Dan O'Leary
Subject: Social Media Complaints and Trending

This problem does not have a great solution. It is the inverse of the underreporting problem. For some devices many complaints are not reported, meaning the manufacturer underestimates the complaint rate. Your problem is that you may have double reporting which overestimates the complaint rate.

I recommend a few approaches. In the first, make a reasonable estimate of the overreporting rate and use it as a deflator. It might be different for different kinds of issues. Compare the deflated rate with the estimated residual risk frequency of occurrence from your ISO 14971:2019 risk management file to detect a signal.

Compare the complaint rate over time, say monthly for a 12 month window, to look for an increase. The easiest way I know is to calculate the linear regression line of the complaint rate and see if the  slope is significantly different from zero. Regression in the Excel Analysis Tool Pack will do this quickly (five minutes to set it up and get the result.)

Use FDA TPLC database to check the rank order of patient problems over time. If a patient problem moves up (more frequent) two or more places, then that generates a signal. You may need to remove your company's MDRs to get a picture of the rest of the industry.

------------------------------
Dan O'Leary CQA, CQE
Swanzey NH
United States

Original Message:
Sent: 13-Oct-2022 21:04
From: Anonymous Member
Subject: Social Media Complaints and Trending

This message was posted by a user wishing to remain anonymous

I am looking to obtain information on how others are trending complaints for medical devices that arise from social media. Our company has a PMA device with an active social media source. Despite requesting that the poster provide contact us privately in order for us to obtain additional information, we are usually unsuccessful at obtaining this information therefore making it impossible to determine if the complaint has been previously reported through either a healthcare provider or through a post-market registry, and cannot obtain any information as to the timing of the event post-implant or details surrounding the event.

The adverse events reported through the social media source are usually serious in nature and result in MDR reports being filed. My question to others is how do you account for these events when a malfunction of the device is not reported but are impacting your risk thresholds without the ability to determine the relationship to device? We are struggling with sending a signal that may not be an indication of our device performance, as well as us as the manufacturer having a true picture of device performance.

Thank you in advance for responding.

• The FDA unequivocally requires that complaints must be documented when they arise from any source.  So, if the feedback meets the official definition of "complaint", then it must be documented as a complaint and processed based on the firm's "good faith" efforts to obtain the information needed to properly evaluate and, where appropriate, investigate, the complaint.  Refer to the Forum discussion here for an overview of "good faith" efforts to obtain complaint information.

• Accordingly, based on the FDA regulatory requirement in point 1 above, the complaints lodged via social media have just as much regulatory merit and weight as any other complaint.  Therefore, avoid any notion or strategy that would tempt you recategorize or down-classify such complaints.

• Don't make up data gymnastics to skew the data in any way.  The data say what the data say, period.  Any other approach compromises the scientific integrity and merit of the firm and its complaint handling.

  

• If a complaint is lodged via social media that sounds like a complaint lodged via a user report, but where it's not objectively / scientifically possible to confirm that they are the same complaint from the same source, then those must be recorded as separate complaints.  There are no legitimate regulatory or scientific provisions for any other response.

• When such a complaint is reportable, then you can designate in the MDR event codes that you weren't able to definitively establish a causal relationship between the device and the event.  That puts on the record that your firm wasn't able to confirm such a link.

• Don't forget the disclaimer in the MDR report that the firm's report is not an admission that the device caused or contributed to the event.

• Also don't forget that FDA still expects an MDR report even if a causal link hasn't definitively been established.  Much can be said about the longstanding basis and historical precedent for this.  In summary, a report could be required if the manufacturer hasn't yet definitively confirmed that the device caused or contributed to a death or serious injury.  In fact, a report could be required even if it is not known whether a death or serious injury occurred at all.  Accordingly, don't make the mistake of not reporting just because the firm hasn't established a definitive causal link.  I have confirmed this interpretation via many years of MDR experience, including involvement in MDR reporting litigation.  For example, as part of such litigations, FDA informed me that if it cannot be completely ruled out that the device may have caused or contributed to a serious injury or death, then an MDR should be submitted for the event.  FDA's "may have" concept is very liberal.  

• A reason for this very sensitive hair-trigger threshold is because FDA has officially stated that it wants to have the ability to judge the seriousness and causal relationship for itself.  If FDA feels that the reported information warrants further investigation, then it will contact the manufacturer to initiate more in-depth investigation during which the firm will have ample opportunity to further characterize its understanding (or lack thereof) of the issue.

• It is indeed a challenging topic, as firms don't want to send a false signal that might indicate their devices are less safe than they really are.  But FDA's MDR reporting system isn't designed to guard against such false impressions.  Instead, FDA demands reporting even when in doubt.  This allows the agency and the public to judge for themselves.  Accordingly, avoid any smoke and mirrors and diversionary tactics when it comes to MDR reporting.  Trust me when I say that plaintiff lawyers and the FDA will easily discredit any such magic-hat diversions, as they are essentially prohibited by FDA's MDR regulations.  The MDR event codes, the firm's explanatory narratives, and the aforesaid disclaimer are our best (and only legitimate) ways to counterbalance the potentially poor optics of an MDR report.  Litigation attacking the firm for failure to responsibility handle its MDR reporting obligations are far more damaging to the brand and bottom line.

------------------------------
Kevin Randall, ASQ CQA, RAC (Europe, U.S., Canada)
Principal Consultant
Ridgway, CO
United States
© Copyright 2022 by ComplianceAcuity, Inc. All rights reserved.
------------------------------

Original Message:
Sent: 16-Oct-2022 05:03
From: Richard Vincins
Subject: Social Media Complaints and Trending

Hello Anon,

You are certainly in an interesting, difficult, and evolving situation with social media information as part of your post market system whether they are just feedback or indeed complaints.  Word of advice, remove the social media outlet.  As much as your marketing or management would love to have this, the problems which come with it usually outweigh the benefit.  Or make the social media outlet absolutely two-way - meaning you can provide information to your customers, but in order for them to give feedback, you do need to all of their contact information.  The FDA and other regulatory agencies are well aware of social media and want to see how companies manage these activities.  Just look at some of the Warning Letter or show them to your management, there are a couple really good examples there, where the process under social media needs to be managed very carefully.  Also recommend within your complaint management system having a well established and implemented process for how feedback and complaints are managed, gathered, investigated, and responded within your post market system.

------------------------------
Richard Vincins ASQ-CQA, MTOPRA, RAC
Vice President Global Regulatory Affairs

Original Message:
Sent: 14-Oct-2022 10:33
From: Dan O'Leary
Subject: Social Media Complaints and Trending

This problem does not have a great solution. It is the inverse of the underreporting problem. For some devices many complaints are not reported, meaning the manufacturer underestimates the complaint rate. Your problem is that you may have double reporting which overestimates the complaint rate.

I recommend a few approaches. In the first, make a reasonable estimate of the overreporting rate and use it as a deflator. It might be different for different kinds of issues. Compare the deflated rate with the estimated residual risk frequency of occurrence from your ISO 14971:2019 risk management file to detect a signal.

Compare the complaint rate over time, say monthly for a 12 month window, to look for an increase. The easiest way I know is to calculate the linear regression line of the complaint rate and see if the  slope is significantly different from zero. Regression in the Excel Analysis Tool Pack will do this quickly (five minutes to set it up and get the result.)

Use FDA TPLC database to check the rank order of patient problems over time. If a patient problem moves up (more frequent) two or more places, then that generates a signal. You may need to remove your company's MDRs to get a picture of the rest of the industry.

------------------------------
Dan O'Leary CQA, CQE
Swanzey NH
United States

Original Message:
Sent: 13-Oct-2022 21:04
From: Anonymous Member
Subject: Social Media Complaints and Trending

This message was posted by a user wishing to remain anonymous

I am looking to obtain information on how others are trending complaints for medical devices that arise from social media. Our company has a PMA device with an active social media source. Despite requesting that the poster provide contact us privately in order for us to obtain additional information, we are usually unsuccessful at obtaining this information therefore making it impossible to determine if the complaint has been previously reported through either a healthcare provider or through a post-market registry, and cannot obtain any information as to the timing of the event post-implant or details surrounding the event.

The adverse events reported through the social media source are usually serious in nature and result in MDR reports being filed. My question to others is how do you account for these events when a malfunction of the device is not reported but are impacting your risk thresholds without the ability to determine the relationship to device? We are struggling with sending a signal that may not be an indication of our device performance, as well as us as the manufacturer having a true picture of device performance.

Thank you in advance for responding.

This message was posted by a user wishing to remain anonymous

I agree with the feedback from the others. In a prior role, I found FDA was much more understanding of potential over-reporting due to social media than they were to potentially under-reporting by attempting to group these complaints. Make sure your MDRs state that the complaints were obtained from social media and source could not be identified. Trend them all, and keep backup trending about the social media (increase in community feedback in general - not just complaints) in case you get questions from FDA. You'll be able to show FDA you are being transparent. As others have suggested, consider making your social media require info before posting or select only one or two key social media sites to allow commenting and all other sites could block public comments.
Original Message:
Sent: 13-Oct-2022 21:04
From: Anonymous Member
Subject: Social Media Complaints and Trending

This message was posted by a user wishing to remain anonymous

I am looking to obtain information on how others are trending complaints for medical devices that arise from social media. Our company has a PMA device with an active social media source. Despite requesting that the poster provide contact us privately in order for us to obtain additional information, we are usually unsuccessful at obtaining this information therefore making it impossible to determine if the complaint has been previously reported through either a healthcare provider or through a post-market registry, and cannot obtain any information as to the timing of the event post-implant or details surrounding the event.

The adverse events reported through the social media source are usually serious in nature and result in MDR reports being filed. My question to others is how do you account for these events when a malfunction of the device is not reported but are impacting your risk thresholds without the ability to determine the relationship to device? We are struggling with sending a signal that may not be an indication of our device performance, as well as us as the manufacturer having a true picture of device performance.

Thank you in advance for responding.