Regulatory Open Forum

Does anyone have any thoughts on KPIs that could be used in a small company's RA dept? From what I have seen and researched, regulatory KPIs center on # of filings/approvals per year, approval times for the above, # of observations/warning letters per year, or revenue per submission. However, most of these work better for a larger company and as reported on an annual basis. Does anyone have input on KPIs that would be meaningful to measure in a small company (that has fewer products) on a monthly basis?
Thanks!

------------------------------
Hiral Dutia
Advanced Instruments, LLC
Norwood MA
United States
------------------------------

Great question!  I'm just starting to develop mine and am focusing on the value to the business in terms of resource manhours, internal & external submission costs, submission timeline, and # holds/questions received.  However, I get most feedback around the regulatory intelligence I added to beef up my monthly report.  New recognized standards, guidance documents, competitor clearances/recalls/MDRs seem to pique the interest of the SLT more than the boring stats.  However, I expect that the stats will definitely become more meaningful as we grow and globalize. 

Tina

------------------------------
Tina O'Brien RAC, MS
Director of Regulatory Affairs
Aroa Biosurgery
Auckland
New Zealand
------------------------------

Original Message:
Sent: 13-Mar-2017 13:28
From: Hiral Dutia
Subject: Regulatory KPIs

Does anyone have any thoughts on KPIs that could be used in a small company's RA dept? From what I have seen and researched, regulatory KPIs center on # of filings/approvals per year, approval times for the above, # of observations/warning letters per year, or revenue per submission. However, most of these work better for a larger company and as reported on an annual basis. Does anyone have input on KPIs that would be meaningful to measure in a small company (that has fewer products) on a monthly basis?
Thanks!

------------------------------
Hiral Dutia
Advanced Instruments, LLC
Norwood MA
United States
------------------------------

I would consider whether the "regulatory" KPIs are treated as indicators of the performance of the company's regulatory department or the regulatory performance of the company.  If they are treated as measures of departmental performance, then I would consider what indicators reflect the performance of the department itself.  RA has almost no influence on the number of filings or revenue per submission. RA is rarely the cause of a warning letter.  And time to approval depends heavily on the quality of the information other departments provide to RA.

------------------------------
Julie Omohundro, ex-RAC (US, GS), still an MBA
Principal Consultant
Class Three, LLC
Durham, North Carolina, USA
919-544-3366 (T)
434-964-1614 (C)
julie@class3devices.com
------------------------------

Original Message:
Sent: 13-Mar-2017 13:28
From: Hiral Dutia
Subject: Regulatory KPIs

Does anyone have any thoughts on KPIs that could be used in a small company's RA dept? From what I have seen and researched, regulatory KPIs center on # of filings/approvals per year, approval times for the above, # of observations/warning letters per year, or revenue per submission. However, most of these work better for a larger company and as reported on an annual basis. Does anyone have input on KPIs that would be meaningful to measure in a small company (that has fewer products) on a monthly basis?
Thanks!

------------------------------
Hiral Dutia
Advanced Instruments, LLC
Norwood MA
United States
------------------------------

I've done this a number of times, but a lot depends on exactly what RA does in your company and your company culture. As Julie said, in some cultures, where RA doesn't have good support or "feeder" processes, metrics can be tough because they really reflect the broader organization, not RA. That said, a few ideas that may be useful.

- change order review timeliness (note - use time to decision, not generally time to approval)
- ad/promo review timeliness
- % on time RA deliverables to project teams (be they regulatory plans or whatever)
- submission completion time from receipt of last input (setting a target will be challenging as it depends on types of submissions among other thing) as you say, this one may need a longer duration than quarterly in a small organization
- for submission quality you can think of things like
    * no refuse to files
    * days your responses pending
    * % of "repeat" questions (which would imply you didn't update processes after getting the question the first time through)
    * first pass approval/clearance

You will have to adapt to meet the needs to your current organization and your current roles an responsibilities. If you happen to have MDRs and/or internal audit, there are plenty more you can add.

g-

------------------------------
Ginger Glaser RAC
Vice-President, Quality and Regulatory Affairs
Maplewood MN
United States
------------------------------

Original Message:
Sent: 13-Mar-2017 13:28
From: Hiral Dutia
Subject: Regulatory KPIs

Does anyone have any thoughts on KPIs that could be used in a small company's RA dept? From what I have seen and researched, regulatory KPIs center on # of filings/approvals per year, approval times for the above, # of observations/warning letters per year, or revenue per submission. However, most of these work better for a larger company and as reported on an annual basis. Does anyone have input on KPIs that would be meaningful to measure in a small company (that has fewer products) on a monthly basis?
Thanks!

------------------------------
Hiral Dutia
Advanced Instruments, LLC
Norwood MA
United States
------------------------------

In addition to all of the good points in the previous replies, I would include a "policy influence" type section which could include participation in AdvaMed or standards groups related to your product. With the previously mentioned items you'd end up with "Submissions" which is probably the largest chunk; Training or Upkeep which is keeping up with the new standards, guidances, and regulations as well as FDA/Notified Body workshops etc; and Policy issues. 

Best,

------------------------------
Michael Nilo
Network Regulatory Partners
Nilo Medical Consulting Group
Portland OR
United States
------------------------------

Original Message:
Sent: 17-Mar-2017 11:04
From: Ginger Glaser
Subject: Regulatory KPIs

I've done this a number of times, but a lot depends on exactly what RA does in your company and your company culture. As Julie said, in some cultures, where RA doesn't have good support or "feeder" processes, metrics can be tough because they really reflect the broader organization, not RA. That said, a few ideas that may be useful.

- change order review timeliness (note - use time to decision, not generally time to approval)
- ad/promo review timeliness
- % on time RA deliverables to project teams (be they regulatory plans or whatever)
- submission completion time from receipt of last input (setting a target will be challenging as it depends on types of submissions among other thing) as you say, this one may need a longer duration than quarterly in a small organization
- for submission quality you can think of things like
    * no refuse to files
    * days your responses pending
    * % of "repeat" questions (which would imply you didn't update processes after getting the question the first time through)
    * first pass approval/clearance

You will have to adapt to meet the needs to your current organization and your current roles an responsibilities. If you happen to have MDRs and/or internal audit, there are plenty more you can add.

g-

------------------------------
Ginger Glaser RAC
Vice-President, Quality and Regulatory Affairs
Maplewood MN
United States
------------------------------

Original Message:
Sent: 13-Mar-2017 13:28
From: Hiral Dutia
Subject: Regulatory KPIs

Does anyone have any thoughts on KPIs that could be used in a small company's RA dept? From what I have seen and researched, regulatory KPIs center on # of filings/approvals per year, approval times for the above, # of observations/warning letters per year, or revenue per submission. However, most of these work better for a larger company and as reported on an annual basis. Does anyone have input on KPIs that would be meaningful to measure in a small company (that has fewer products) on a monthly basis?
Thanks!

------------------------------
Hiral Dutia
Advanced Instruments, LLC
Norwood MA
United States
------------------------------

Thanks so much for everyone's input! I agree that "regulatory" KPIs are often not just influenced by RA itself, which is why coming up with a list of true regulatory KPIs was difficult.

I think keeping up with regulatory intelligence/training, and percent of on-time RA deliverables to other teams are something I will consider as those could work for a company our size. I will incorporate some of these into our list. If anyone else has more insight, I'd be open to hearing it.

------------------------------
Hiral Dutia
Advanced Instruments, LLC
Norwood MA
United States
------------------------------

Original Message:
Sent: 20-Mar-2017 15:22
From: Michael Nilo
Subject: Regulatory KPIs

In addition to all of the good points in the previous replies, I would include a "policy influence" type section which could include participation in AdvaMed or standards groups related to your product. With the previously mentioned items you'd end up with "Submissions" which is probably the largest chunk; Training or Upkeep which is keeping up with the new standards, guidances, and regulations as well as FDA/Notified Body workshops etc; and Policy issues. 

Best,

------------------------------
Michael Nilo
Network Regulatory Partners
Nilo Medical Consulting Group
Portland OR
United States
------------------------------

Original Message:
Sent: 17-Mar-2017 11:04
From: Ginger Glaser
Subject: Regulatory KPIs

I've done this a number of times, but a lot depends on exactly what RA does in your company and your company culture. As Julie said, in some cultures, where RA doesn't have good support or "feeder" processes, metrics can be tough because they really reflect the broader organization, not RA. That said, a few ideas that may be useful.

- change order review timeliness (note - use time to decision, not generally time to approval)
- ad/promo review timeliness
- % on time RA deliverables to project teams (be they regulatory plans or whatever)
- submission completion time from receipt of last input (setting a target will be challenging as it depends on types of submissions among other thing) as you say, this one may need a longer duration than quarterly in a small organization
- for submission quality you can think of things like
    * no refuse to files
    * days your responses pending
    * % of "repeat" questions (which would imply you didn't update processes after getting the question the first time through)
    * first pass approval/clearance

You will have to adapt to meet the needs to your current organization and your current roles an responsibilities. If you happen to have MDRs and/or internal audit, there are plenty more you can add.

g-

------------------------------
Ginger Glaser RAC
Vice-President, Quality and Regulatory Affairs
Maplewood MN
United States
------------------------------

Original Message:
Sent: 13-Mar-2017 13:28
From: Hiral Dutia
Subject: Regulatory KPIs

Does anyone have any thoughts on KPIs that could be used in a small company's RA dept? From what I have seen and researched, regulatory KPIs center on # of filings/approvals per year, approval times for the above, # of observations/warning letters per year, or revenue per submission. However, most of these work better for a larger company and as reported on an annual basis. Does anyone have input on KPIs that would be meaningful to measure in a small company (that has fewer products) on a monthly basis?
Thanks!

------------------------------
Hiral Dutia
Advanced Instruments, LLC
Norwood MA
United States
------------------------------

Original Message------

Thanks so much for everyone's input! I agree that "regulatory" KPIs are often not just influenced by RA itself, which is why coming up with a list of true regulatory KPIs was difficult.

I think keeping up with regulatory intelligence/training, and percent of on-time RA deliverables to other teams are something I will consider as those could work for a company our size. I will incorporate some of these into our list. If anyone else has more insight, I'd be open to hearing it.

------------------------------
Hiral Dutia
Advanced Instruments, LLC
Norwood MA
United States
------------------------------