Hello Anon,
If you are claiming compliance to IEC 62366-1 the simple answer is yes, always. Otherwise, if you claim compliance to a standard and not follow it, then this raises all kinds of other issues. There are also consideration of a Summative Evaluation or Usability Report or Usability File ... are they the same? For me yes. No matter what is it actually called, if usability testing is needed there should be a document supporting these activities. It also means in other instances you need a usability report for other purposes such as electrical safety testing, 510(k) submissions, etc. I completely agree with Kevin, it does not matter your device, your development activities, how fast your organisation goes, if usability activities are needed for the device, these need to be done. It would not be a good thing for a company when a regulatory agency reviews the usability activities and they are months out of date because usability testing and documentation does not keep up with the organisation. To answer your question, there are ways this can be managed for changes which may be occurring over time, finding creative ways to keep activities current, and utilising methods to review the changes as an on-going basis.
------------------------------
Richard Vincins ASQ-CQA, MTOPRA, RAC
Vice President Global Regulatory Affairs
------------------------------
Original Message:
Sent: 21-Feb-2022 13:27
From: Kevin Randall
Subject: When is summative evaluation compulsory?
As a general rule, there must always be usability engineering summative evaluation to cover the full user interface that is being marketed. Remember that user interface summative evaluation is closely linked with the design/development validation process; neither are optional. Accordingly, the same general design change principles apply when we make user interface design changes.
Specifically, if a design change doesn't affect the user or the user interface, then the existing summative evaluation can still be considered valid and applicable. For example, IEC 62366:2015 / EN 62366:2015 as amended state that, "…For SUMMATIVE EVALUATION, the MANUFACTURER may use data obtained from the SUMMATIVE EVALUATIONS of products with an equivalent USER INTERFACE together with a technical rationale for how this data is applicable. The results shall be stored in the USABILITY ENGINEERING FILE…In the case of a design modification, the SUMMATIVE EVALUATION can be performed by looking at data obtained from previous SUMMATIVE EVALUATIONS for the parts that have not changed. This is the same PROCESS that is used for any POST-PRODUCTION design modification…". [Emphasis added]
But if the user interface has changed, then appropriate summative evaluation is generally required, especially if the user interface changes frequently. Frequent user interface changes would in my opinion require even more care regarding summative evaluation so as to assure that the frequent changes don't increase the frequency of known use errors or introduce new use errors, hazards, hazardous situations, or hazard-related use scenarios.
The needs of public health aren't swayed by the demands of an aggressive business plan. Instead, the business plan needs to sway to the needs of public health.
------------------------------
Kevin Randall, ASQ CQA, RAC (Europe, U.S., Canada)
Principal Consultant
Ridgway, CO
United States
© Copyright 2022 by ComplianceAcuity, Inc. All rights reserved.
Original Message:
Sent: 20-Feb-2022 11:44
From: Anonymous Member
Subject: When is summative evaluation compulsory?
This message was posted by a user wishing to remain anonymous
Hi all,
I am currently updating our usability procedure and I would like to identify some rules that can, in certain circumstances, allow me to discard for a specific change the requirement of summative evaluation. Currently our yes/no decision depends on whether a critical task is involved (associated with a risk with severity 3 or higher).
However, as I work in an "extremely" agile and fast-paced company, it is not feasible to repeat a summative study for the dozen of changes that are proposed by D&D every 2 months. To give you the full picture, a summative study, for our specific device, is almost as burdensome as a clinical study and requires an IRB approval. It took me ~1 year to prepare and implement our first summative evaluation.
For example, for changes that are related to serious risks but don't impact the risks negatively (increase severity or require a new risk control measure), can I conclude that summative is not needed and we can limit our analysis to internal simulated testing (which we call formative)? Is there any other practical advice you can give me concerning on how the evaluate such changes?
Thanks in advance