Regulatory Open Forum

This message was posted by a user wishing to remain anonymous

Hi All,

I recently joined a startup working in the AI for medical imaging space as the first RA employee. I have some knowledge of RA in software but haven't worked extensively in it before, and am ramping up on the go. While discussing verification and validation requirements with the engineers, some questions popped up and I was hoping I could get some pointers to resources/examples here. 

Software verification -
1. Practically, what does verification outside of software testing include, as in, what is the FDA looking for? Are any examples available to refer?

2. Do unit/integration/system level tests have to be committed separately from the actual code for good documentation practices? Context - my recommendation here was to document V&V in packages of unit testing, integration testing, and system level/validation testing.

Software validation - 
3. Simulated use environment testing - can it be tested in a sandbox environment? Or is prod absolutely necessary?

I apologize if this is asking too much. Thank you so much!

Hi Anon

Not sure if you're looking for good practice recommendations or the minimum requirements for whatever classification this product will fall under.

Some fundamentals - a quality system, a plan, some risk assessment, deciding how the software should be developed before you start pounding out code...

There are FDA guidances on design control, software development, AI/ML, imaging systems, and other related topics, as well as applicable standards

------------------------------
Anne LeBlanc
United States
------------------------------

Original Message:
Sent: 30-Nov-2021 16:46
From: Anonymous Member
Subject: SaMD Verification and Validation

This message was posted by a user wishing to remain anonymous

Hi All,

I recently joined a startup working in the AI for medical imaging space as the first RA employee. I have some knowledge of RA in software but haven't worked extensively in it before, and am ramping up on the go. While discussing verification and validation requirements with the engineers, some questions popped up and I was hoping I could get some pointers to resources/examples here.

Software verification -
1. Practically, what does verification outside of software testing include, as in, what is the FDA looking for? Are any examples available to refer?

2. Do unit/integration/system level tests have to be committed separately from the actual code for good documentation practices? Context - my recommendation here was to document V&V in packages of unit testing, integration testing, and system level/validation testing.

Software validation -
3. Simulated use environment testing - can it be tested in a sandbox environment? Or is prod absolutely necessary?

I apologize if this is asking too much. Thank you so much!

You'll need to follow IEC 62304:2015 and if it's a stand-alone SaMD, IEC 82304.  You can also find good information in Applying Human Factors and Usability Engineering to Medical Devices, BS_PD_ISO_TR_80002-2_2017, FDA Guidance for Industry - Mobile Medical Applications, General Principles of Software Validation; Final Guidance for Industry and FDA Staff, and Guidance-for-the-Content-of-Premarket-Submissions-for-Software-Contained-in-Medical-Devices---Guidance-for-Industry-and-FDA-Staff (although this is being updated) as well as other standards.

------------------------------
Rhonda Johnston
Sr. Mgr Design Quality Assurance
Pittsburgh PA
United States
------------------------------

Original Message:
Sent: 04-Dec-2021 18:03
From: Anne LeBlanc
Subject: SaMD Verification and Validation

Hi Anon

Not sure if you're looking for good practice recommendations or the minimum requirements for whatever classification this product will fall under.

Some fundamentals - a quality system, a plan, some risk assessment, deciding how the software should be developed before you start pounding out code...

There are FDA guidances on design control, software development, AI/ML, imaging systems, and other related topics, as well as applicable standards

------------------------------
Anne LeBlanc
United States

Original Message:
Sent: 30-Nov-2021 16:46
From: Anonymous Member
Subject: SaMD Verification and Validation

This message was posted by a user wishing to remain anonymous

Hi All,

I recently joined a startup working in the AI for medical imaging space as the first RA employee. I have some knowledge of RA in software but haven't worked extensively in it before, and am ramping up on the go. While discussing verification and validation requirements with the engineers, some questions popped up and I was hoping I could get some pointers to resources/examples here.

Software verification -
1. Practically, what does verification outside of software testing include, as in, what is the FDA looking for? Are any examples available to refer?

2. Do unit/integration/system level tests have to be committed separately from the actual code for good documentation practices? Context - my recommendation here was to document V&V in packages of unit testing, integration testing, and system level/validation testing.

Software validation -
3. Simulated use environment testing - can it be tested in a sandbox environment? Or is prod absolutely necessary?

I apologize if this is asking too much. Thank you so much!