Regulatory Open Forum

This message was posted by a user wishing to remain anonymous

Dear community,

we are developing a new SaMD and formative evaluation was not conducted for the whole software but major parts of it.
Is it required  acc. to 62366 to do a formative evaluation for the remaining parts or could this be covered by the summative evaluation instead.
The only risk I could think of, would be to rework the software in case we identify new hazards during summative evaluation.

Your feedback is highly appreciated.
Best regards.

Formative evaluations are intended to be very similar to other verification activities and (like verification) are generally completed according to an overall verification plan as you complete those various pieces. It is not necessary to do a formative evaluation on the entire design. This is the purpose of the summative evaluation (similar to validation activities). 
As you identify, the other item to pay attention to is your risk analysis (IEC 62366-1 relies on ISO 14971). What are the risks associated from a use case standpoint that cannot be evaluated until the system comes together? You may want to do some limited formative evaluation of those items as well to mitigate the risk of failing the summative evaluation.

------------------------------
Regards,
Mark Swanson, ASQ CBA, CMQ/OE, CQE ASQ, MBA
Becker MN
United States
------------------------------

Original Message:
Sent: 26-Apr-2019 03:05
From: Anonymous Member
Subject: Combine formative and summative evaluation

This message was posted by a user wishing to remain anonymous

Dear community,

we are developing a new SaMD and formative evaluation was not conducted for the whole software but major parts of it.
Is it required  acc. to 62366 to do a formative evaluation for the remaining parts or could this be covered by the summative evaluation instead.
The only risk I could think of, would be to rework the software in case we identify new hazards during summative evaluation.

Your feedback is highly appreciated.
Best regards.

Think of a formative evaluation as a "pre-usability study" where generally it is fairly limited in scope.  In fact, I have worked with a couple products where we did 2 or 3 formative evaluations because of complexity associated with the device and different stages in the design process.  They allow you to go through the usability process that feeds back into your risk management and information supplied to the user.  In the same view, a formative evaluation could theoretically be combined with the summative study if the device was simple or user-interface is straight-forward.  The standard and other guidance documents do not say a formative study is always required, the human factors and usability is a process throughout design controls and follow along into post market to continually give feedback on use of the device.

------------------------------
Richard Vincins RAC
Vice President Global Regulatory Affairs
------------------------------

Original Message:
Sent: 27-Apr-2019 09:03
From: Mark Swanson
Subject: Combine formative and summative evaluation

Formative evaluations are intended to be very similar to other verification activities and (like verification) are generally completed according to an overall verification plan as you complete those various pieces. It is not necessary to do a formative evaluation on the entire design. This is the purpose of the summative evaluation (similar to validation activities).
As you identify, the other item to pay attention to is your risk analysis (IEC 62366-1 relies on ISO 14971). What are the risks associated from a use case standpoint that cannot be evaluated until the system comes together? You may want to do some limited formative evaluation of those items as well to mitigate the risk of failing the summative evaluation.

------------------------------
Regards,
Mark Swanson, ASQ CBA, CMQ/OE, CQE ASQ, MBA
Becker MN
United States

Original Message:
Sent: 26-Apr-2019 03:05
From: Anonymous Member
Subject: Combine formative and summative evaluation

This message was posted by a user wishing to remain anonymous

Dear community,

we are developing a new SaMD and formative evaluation was not conducted for the whole software but major parts of it.
Is it required  acc. to 62366 to do a formative evaluation for the remaining parts or could this be covered by the summative evaluation instead.
The only risk I could think of, would be to rework the software in case we identify new hazards during summative evaluation.

Your feedback is highly appreciated.
Best regards.

Mark and Richard make very good points.

I have one other for you to consider. A formative test has small numbers (5 - 7 per distinct user group). A summative test requires 15 - 20 participants per distinct user group. It is far more expensive to do a summative test than a formative test so it's more cost effective to work out your kinks in formative and know that you'll breeze through your summative.

Regards,

------------------------------
Joely Gardner PhD
Usability Researcher and Professor
San Marcos CA
joely.gardner@humanfr.com
------------------------------

Original Message:
Sent: 26-Apr-2019 03:05
From: Anonymous Member
Subject: Combine formative and summative evaluation

This message was posted by a user wishing to remain anonymous

Dear community,

we are developing a new SaMD and formative evaluation was not conducted for the whole software but major parts of it.
Is it required  acc. to 62366 to do a formative evaluation for the remaining parts or could this be covered by the summative evaluation instead.
The only risk I could think of, would be to rework the software in case we identify new hazards during summative evaluation.

Your feedback is highly appreciated.
Best regards.