Regulatory Open Forum

Hi RAPS,

I am working at a SaMD startup, where our device software is designed to run on a cloud based service in AWS. I am relatively new to SaMD, and had some questions regarding the validation of AWS - 

1. Do we need to validate all resources within AWS that we use (for eg. CloudWatch, IAM, S3, AWS Batch, etc.)? Or is a test run enough to prove that AWS is working as it should?
2.  For tool configuration, again, do we need to list all resources available within AWS, as they can all be configured separately? What is the minimum requirement of configurations that we have to provide for AWS tools? A SW engineer within the team suggested identifying the important resources (AWS Batch, S3 buckets) and providing configurations for these, but I'm not a fan because if these resources are identified as configurations, all resources on the same level should be as well. Adding all the smaller resources and dependencies would be very time-consuming and technical.
Or is saying that AWS is a tool, and any configuration updates within AWS are validated with a test run sufficient?


Any referrals/guidance/opinions on how to validate AWS are appreciated.

Many thanks


------------------------------
Akshay Kulkarni
United States
------------------------------

Hi Akshay

Some tools are more critical than others.

You might begin by defining what services AWS is providing. Then you'll need to consider how those could affect the device software if they don't perform as expected.

There are probably some aspects that would merely inconvenience the company if they failed. If there are any whose failure might lead to the product doing something that could hurt a patient, you would want tighter controls there.

Once you've documented which areas are lower or higher risk (and why), you can focus on the areas that matter. There's no requirement to spend the same amount of effort on areas that don't matter.

------------------------------
Anne LeBlanc
United States
------------------------------

Original Message:
Sent: 10-Mar-2022 16:05
From: Akshay Kulkarni
Subject: AWS Validation as a third-party software

Hi RAPS,

I am working at a SaMD startup, where our device software is designed to run on a cloud based service in AWS. I am relatively new to SaMD, and had some questions regarding the validation of AWS -

1. Do we need to validate all resources within AWS that we use (for eg. CloudWatch, IAM, S3, AWS Batch, etc.)? Or is a test run enough to prove that AWS is working as it should?
2.  For tool configuration, again, do we need to list all resources available within AWS, as they can all be configured separately? What is the minimum requirement of configurations that we have to provide for AWS tools? A SW engineer within the team suggested identifying the important resources (AWS Batch, S3 buckets) and providing configurations for these, but I'm not a fan because if these resources are identified as configurations, all resources on the same level should be as well. Adding all the smaller resources and dependencies would be very time-consuming and technical.
Or is saying that AWS is a tool, and any configuration updates within AWS are validated with a test run sufficient?


Any referrals/guidance/opinions on how to validate AWS are appreciated.

Many thanks


------------------------------
Akshay Kulkarni
United States
------------------------------

This message was posted by a user wishing to remain anonymous

Hi Akshay, 

In addition to Anne's great suggestion, I would also recommend taking into account overall product requirements for cyber security of your device in the cloud environment, do you need to design in additional controls in your software?
Also consider how is data passes to and from your S3 buckets, what controls you have there. 

There is a good MDCG guidance (for IVDR/MDR)https://ec.europa.eu/health/system/files/2022-01/md_cybersecurity_en.pdf
From this guidance, other international standards and guidance are listed for cyber security requirements and testing. 

I am also working for a company using AWS as a cloud provider for an upcoming product, so am keen to hear others feedback. 

Kind Regards, 
AG
Original Message:
Sent: 10-Mar-2022 20:21
From: Anne LeBlanc
Subject: AWS Validation as a third-party software

Hi Akshay

Some tools are more critical than others.

You might begin by defining what services AWS is providing. Then you'll need to consider how those could affect the device software if they don't perform as expected.

There are probably some aspects that would merely inconvenience the company if they failed. If there are any whose failure might lead to the product doing something that could hurt a patient, you would want tighter controls there.

Once you've documented which areas are lower or higher risk (and why), you can focus on the areas that matter. There's no requirement to spend the same amount of effort on areas that don't matter.

------------------------------
Anne LeBlanc
United States

Original Message:
Sent: 10-Mar-2022 16:05
From: Akshay Kulkarni
Subject: AWS Validation as a third-party software

Hi RAPS,

I am working at a SaMD startup, where our device software is designed to run on a cloud based service in AWS. I am relatively new to SaMD, and had some questions regarding the validation of AWS -

1. Do we need to validate all resources within AWS that we use (for eg. CloudWatch, IAM, S3, AWS Batch, etc.)? Or is a test run enough to prove that AWS is working as it should?
2.  For tool configuration, again, do we need to list all resources available within AWS, as they can all be configured separately? What is the minimum requirement of configurations that we have to provide for AWS tools? A SW engineer within the team suggested identifying the important resources (AWS Batch, S3 buckets) and providing configurations for these, but I'm not a fan because if these resources are identified as configurations, all resources on the same level should be as well. Adding all the smaller resources and dependencies would be very time-consuming and technical.
Or is saying that AWS is a tool, and any configuration updates within AWS are validated with a test run sufficient?


Any referrals/guidance/opinions on how to validate AWS are appreciated.

Many thanks


------------------------------
Akshay Kulkarni
United States
------------------------------

when you say validation, do you mean Design Validation (i.e., 820.30(g)) or validation as described in 820.70(i) Automated processes?

For Design Validation you should ensure that your software with any third-party components or platforms (I'll call it a software system) performs as intended, so at minimum you should assess which AWS resources or options could affect the performance of the software system. Some of these activities may be performed as part of Design Verification (e.g., code review or inspection of proper configuration). 

You do not have to perform separate validation of the AWS or validate services which you do not utilize, but your Design Verification and/or Validation activities should capture any resources that you use or may affect performance.
You may have to disable or block unused resources for cybersecurity reasons.

------------------------------
Michael Zagorski RAC
Director of Regulatory Affairs
Pittsburgh PA
------------------------------

Original Message:
Sent: 10-Mar-2022 16:05
From: Akshay Kulkarni
Subject: AWS Validation as a third-party software

Hi RAPS,

I am working at a SaMD startup, where our device software is designed to run on a cloud based service in AWS. I am relatively new to SaMD, and had some questions regarding the validation of AWS -

1. Do we need to validate all resources within AWS that we use (for eg. CloudWatch, IAM, S3, AWS Batch, etc.)? Or is a test run enough to prove that AWS is working as it should?
2.  For tool configuration, again, do we need to list all resources available within AWS, as they can all be configured separately? What is the minimum requirement of configurations that we have to provide for AWS tools? A SW engineer within the team suggested identifying the important resources (AWS Batch, S3 buckets) and providing configurations for these, but I'm not a fan because if these resources are identified as configurations, all resources on the same level should be as well. Adding all the smaller resources and dependencies would be very time-consuming and technical.
Or is saying that AWS is a tool, and any configuration updates within AWS are validated with a test run sufficient?


Any referrals/guidance/opinions on how to validate AWS are appreciated.

Many thanks


------------------------------
Akshay Kulkarni
United States
------------------------------

Hi Akshay & all,

We usually speak about infrastructure qualification and software validation because infrastructure components are generally considered lower risk from a software quality perspective and, because of that, you usually don't do a thorough validation as you would do on the software. As others suggested, list all components that you are using and perform a risk assessment to leverage verification efforts. This can be testing but can also be documentational activities. We usually use impact, complexity and maturity of the product and its supplier as factors in determining the risk. The latter would be challenging especially with big tech companies that don't provide support to industry specific requirements. AWS does have a high level guidance which summarizes the important aspects to consider when using their services in the life science industry. Cybersecurity, often considered separately from software quality but not less important these days, should indeed also be included in a risk assessment.

Kind regards,

------------------------------
Hao Wang
Computer Validation & Compliance Specialist
www.c-realize.com
------------------------------

Original Message:
Sent: 11-Mar-2022 17:29
From: Michael Zagorski
Subject: AWS Validation as a third-party software

when you say validation, do you mean Design Validation (i.e., 820.30(g)) or validation as described in 820.70(i) Automated processes?

For Design Validation you should ensure that your software with any third-party components or platforms (I'll call it a software system) performs as intended, so at minimum you should assess which AWS resources or options could affect the performance of the software system. Some of these activities may be performed as part of Design Verification (e.g., code review or inspection of proper configuration).

You do not have to perform separate validation of the AWS or validate services which you do not utilize, but your Design Verification and/or Validation activities should capture any resources that you use or may affect performance.
You may have to disable or block unused resources for cybersecurity reasons.

------------------------------
Michael Zagorski RAC
Director of Regulatory Affairs
Pittsburgh PA

Original Message:
Sent: 10-Mar-2022 16:05
From: Akshay Kulkarni
Subject: AWS Validation as a third-party software

Hi RAPS,

I am working at a SaMD startup, where our device software is designed to run on a cloud based service in AWS. I am relatively new to SaMD, and had some questions regarding the validation of AWS -

1. Do we need to validate all resources within AWS that we use (for eg. CloudWatch, IAM, S3, AWS Batch, etc.)? Or is a test run enough to prove that AWS is working as it should?
2.  For tool configuration, again, do we need to list all resources available within AWS, as they can all be configured separately? What is the minimum requirement of configurations that we have to provide for AWS tools? A SW engineer within the team suggested identifying the important resources (AWS Batch, S3 buckets) and providing configurations for these, but I'm not a fan because if these resources are identified as configurations, all resources on the same level should be as well. Adding all the smaller resources and dependencies would be very time-consuming and technical.
Or is saying that AWS is a tool, and any configuration updates within AWS are validated with a test run sufficient?


Any referrals/guidance/opinions on how to validate AWS are appreciated.

Many thanks


------------------------------
Akshay Kulkarni
United States
------------------------------

Thank you all for the great suggestions! I will work with the team on identifying important configurations and how best to verify them.

Thanks again!

------------------------------
Akshay Kulkarni
Fremont CA
United States
------------------------------

Original Message:
Sent: 10-Mar-2022 16:05
From: Akshay Kulkarni
Subject: AWS Validation as a third-party software

Hi RAPS,

I am working at a SaMD startup, where our device software is designed to run on a cloud based service in AWS. I am relatively new to SaMD, and had some questions regarding the validation of AWS -

1. Do we need to validate all resources within AWS that we use (for eg. CloudWatch, IAM, S3, AWS Batch, etc.)? Or is a test run enough to prove that AWS is working as it should?
2.  For tool configuration, again, do we need to list all resources available within AWS, as they can all be configured separately? What is the minimum requirement of configurations that we have to provide for AWS tools? A SW engineer within the team suggested identifying the important resources (AWS Batch, S3 buckets) and providing configurations for these, but I'm not a fan because if these resources are identified as configurations, all resources on the same level should be as well. Adding all the smaller resources and dependencies would be very time-consuming and technical.
Or is saying that AWS is a tool, and any configuration updates within AWS are validated with a test run sufficient?


Any referrals/guidance/opinions on how to validate AWS are appreciated.

Many thanks


------------------------------
Akshay Kulkarni
United States
------------------------------