Regulatory Open Forum

 View Only
Expand all | Collapse all

Risks in software developing as medical device

  • 1.  Risks in software developing as medical device

    Posted 18-Jul-2021 09:38
    Hi
    What could be the existing / potential risks in the development process in medical software, which are medical devices.
    The developers I spoke to argued that there are no risks and the main risks are in information security like software hacking. And in fact, there are no risks associated with the development process. In the risk analysis process, they suggested listing risks associated with the software users.
    I would appreciate yours expertise, ideas and opinion on that issue 
    Mor


    ------------------------------
    Mor Sharon
    Regulatory Affair Specialist
    OROT
    Israel
    ------------------------------


  • 2.  RE: Risks in software developing as medical device

    Posted 18-Jul-2021 16:05
    Hello

    I'm not sure if I completely understand the question, but here are a few thoughts.  I'd begin assessing risks at the top level -- could the software itself cause harm​, or could the use of it lead to harms. This is something like assessing the 62304 safety class A, B, or C. Knowing the overall criticality of it helps you decide the level of control needed in development.

    If someone's life may depend on the software, you may choose to develop it as a high-integrity system component, with very strict controls on environment and processes. At the opposite extreme, you might be content to allow a junior programmer to slap something together any which way, so long as it mostly seems to work. Most software is somewhere in between, and you choose a level of control sufficient to control safety risks, to produce a level of quality that will satisfy customers, and to manage the development costs.

    There's a sweet spot where you spend enough effort in planning the process so you don't end up spending too much effort debugging and re-designing. You have to design a level of quality at the front of the process, because you can't test quality into the software from the verification stage.

    pFMEA on the software development process probably makes sense for a high-integrity component, but not so much for most applications.

    ------------------------------
    Anne LeBlanc
    United States
    ------------------------------



  • 3.  RE: Risks in software developing as medical device

    Posted 19-Jul-2021 01:10
    Edited by Vikram Upadhya 19-Jul-2021 06:32
    Shalom Mor,
        The question really has 2 parts - am trying to help you get some specifics from Development addressed.
    1) Software as a Medical Device needs to have a intended use defined which really lists the possibility if product use, its hazards and possible of misuse. this may include a innocuous tracking software but may unintentionally be misused to track people to track people or use the software as a means to connect to your home network/Mobile
    2) All Products inevitably have some risk -either it could be user error, user error or Misuse - how they would be handled and mitigated from software needs to be handled during product design (also called Design-FMEA) or during Manufacture(Process-FMEA). These risks have a probability of occurrence and may be mitigated by a combination of factors - may be by design, by having controls in place or by Instruction(Instructions for Use/User manual).

    Proper documentation for Software SDLC - per FDA- would try to map the Intended use and misuse possibilities with and without design controls. Thus a Hazard Analysis is needed - if skipped during design, you may want to do root cause analysis with field reported issues- to address that the product is kept up-to-date about possibility of product/software failures.

    Some of the failures may not be apparent or thought of during Design(D-FMEA) and would need to be factored as part of additional risks( as an example the software may have a build dependency that is vulnerable to attack or a processor that does not implement encryption) for continued product security. Do plan to list these with the Development team and with the planned users who may need security, privacy and availability assessments done to avoid liability later.

    regards,
    Vikram U

    ------------------------------
    Vikram Upadhya
    Tech Lead
    MDD-MDR Transition
    HCL Tech
    India
    ------------------------------



  • 4.  RE: Risks in software developing as medical device

    Posted 19-Jul-2021 04:49

    Hi Mor, Anne, Vikram,

    my thought might be obvious nevertheless, I would response with a activity to address definitions and concepts (as SW persons typically are pretty used to work with well defined terms). 

    >> I would ask the team to start by identifying risks (the potential of harm is always existing but may be very unlikely), the hazards and threats leading (by a sequence of events) to hazardous situations or vulnerabilities. Once this separations are clear for the team, I would proceed to the the risk evaluation step. 

    >> It may be, that the team is sound that all risks are controlled to a level of very low occurrence likelihood and are therefor acceptable.

    >> I have seen many teams which started to move  once it was clear that they are asked to list all  risk controls, which they already implemented and which in turn  lead to (team) assessment, that there is no remaining unacceptable risk. 



    ------------------------------
    Uwe Zeller | Regulatory Affairs / Risk Management Consultant
    Biberach an der Riß, Germany
    ------------------------------



  • 5.  RE: Risks in software developing as medical device

    Posted 19-Jul-2021 09:04
    Dear Colleagues,

    I would add to the many useful thoughts mentioned here, that a detailed review of the state-of-the-art of clinical use certainly helps to understand/present what are the foreseen risks and benefits of a software's use.
    Also important, that the proper clinical validation of the software's functionality is needed for acceptance.

    regards

    ------------------------------
    Peter Mikó M.D
    ArtPharm Ltd.
    Gyermely
    Hungary
    ------------------------------



  • 6.  RE: Risks in software developing as medical device

    Posted 19-Jul-2021 12:37
    There are of course supply chain risks, i.e. third party software either Commercial Off The Shelf Software (COTS) or Software of Unknown Provenance (SOUP); take a look at:

    https://www.fda.gov/regulatory-information/search-fda-guidance-documents/shelf-software-use-medical-devices

    There have been some of these open-source software supply chain hacks in the news lately!

    Since the developers are arguing that the main risks are "software hacking", is the development process up to current thinking around secure development processes and procedures:

    https://csrc.nist.gov/publications/detail/white-paper/2020/04/23/mitigating-risk-of-software-vulnerabilities-with-ssdf/final

    As a starter, it sounds like some education needs to occur around standards such as ISO 13485, 14971, 24971 and 62304.

    An excellent reference book is available from RAPS Publications as a starting point "Software as a Medical Device Regulatory and Market Access Implications" by Cobbaert and Bos.

    Best wishes,

    Larry




    ------------------------------
    Larry Gryziak
    VP Quality and Regulatory
    Chicago IL
    United States
    ------------------------------



  • 7.  RE: Risks in software developing as medical device

    Posted 19-Jul-2021 19:13
    Edited by Kevin Randall 19-Jul-2021 19:14
    The most fundamental reason for medical device design and development controls (e.g., ISO 13485 clause 7.3; the U.S. FDA's 21 CFR 820.30, etc.) is to control risks that come from an inadequate development process.  That fact is unequivocally true and stands in stark contrast to the erroneous notion that there are no risks from the software development process.  For example, in a 6-year study, FDA found that approximately 44 percent of medical device quality problems leading to recalls were attributed to design errors that FDA says may have been prevented by adequate design controls.  And regarding design and development of software, FDA did a subsequent study for fiscal year (FY) 1983 through FY 1991 and found that a whopping 90 percent of all software-related device failures were from failure to have a proper software development process.

    When approaching risk management regarding the software development process itself (as distinguished from risk management of the software medical device), the general paradigm needs to start from a process-focused approach rather than a device-focused starting point.

    There are multiple ways to do process-focused risk analysis.  Now, we all know how important it is to realize that FMEA, on its own, is not sufficient to meet ISO 14971's risk management and analysis requirements.  But for temporary discussion purposes, I'm going to dare say something slightly to the contrary in order to help drive home the process-focused concept mentioned above.  Specifically, to really grasp the distinction of the process-focused concept, think in terms of p(process) FMEA vs. d(design) FMEA.  Okay, there I said it.  Now quickly, while my colleagues (and me too of course) are catching their breath and settling down about the mention of FMEA in the context of risk management, I'll reiterate the disclaimer that we need to be sure and remember FMEA's intrinsic limitations and intent.

    So, if you're familiar with doing risk analysis for a process (instead of a device), then those principles are the ones that should be applied in order to do risk management of the software development process itself.  What can be confusing is that the ultimate purpose of medical device risk management is to control the risks to patients and users.  Consequently, it may well be (as the U.S. FDA has indicated), that an inadequate development process can indeed be ultimately correlated to patient and user harms.  But there may also be process-centric hazards and harms (derived e.g., via FMEA) that are focused on the process itself.  When that is the case, then bridge the ISO 14971 risk management gap by being sure to, where appropriate, ultimately extrapolate (by representation as resulting hazardous situations and sequences of events) into the patient or user harms.

    Check out AAMI TIR 36 (Annex B 'Severity' explanation under 'Risk management terminology as it applies to quality and production systems) and ISO 80002-2 (Annex B section B.4) for interesting interpretations of how process risk analysis can differ from product risk analysis.

    ------------------------------
    Kevin Randall, ASQ CQA, RAC (Europe, Canada, U.S.)
    Principal Consultant
    Ridgway, CO
    United States
    © Copyright 2021 by ComplianceAcuity, Inc. All rights reserved.
    ------------------------------



  • 8.  RE: Risks in software developing as medical device

    Posted 19-Jul-2021 20:21
    Edited by Kevin Randall 20-Jul-2021 09:44
    By the way, I should have mentioned that it's only by mere coincidence that AAMI TIR 36 and ISO 80002-2 which I cited happen to involve a software-related context; please note that they aren't intended to address medical device software validation, nor device software development.  Instead, I recommended these resources simply because they coincidentally happen to provide some good illustrations about doing process-focused risk analysis as distinguished from device-focused risk analysis.

    I also forgot to mention that IEC 60812 also contains some good basic process FMEA insights which can be reasonably adapted for software development process risk analysis.


    ------------------------------
    Kevin Randall, ASQ CQA, RAC (Europe, Canada, U.S.)
    Principal Consultant
    Ridgway, CO
    United States
    © Copyright 2021 by ComplianceAcuity, Inc. All rights reserved.
    ------------------------------



  • 9.  RE: Risks in software developing as medical device

    Posted 20-Jul-2021 00:31
    Well, there are all sorts if risks in the process from bad coding to inadequate specification of requirements, using and not qualifying SOUP, poor architecture, not fully testing features, poor timing between different software components, bad buffers, you name it.

    I suggest you refer to the IEC standards for software risk management,  in addition to IEC 62404, & where applicable IEC 82304.. 

    A good RM start is IEC TIR 80002-1, Medical Device Software, Part 1 -Guidance on the application of risk management to medical device software.  The Annexes of IEC TIR 80002,-1 have some great examples for consideration.   


    ------------------------------
    Ginger Cantor, MBA, RAC
    Founder/Principal Consultant
    Centaur Consulting LLC
    River Falls, Wisconsin 54022 USA
    715-307-1850
    centaurconsultingllc@gmail.com
    ------------------------------



  • 10.  RE: Risks in software developing as medical device

    Posted 20-Jul-2021 10:45
    Hi all,
    As an extension to this discussion I'd be keen to hear thoughts / references on is AI/Machine learning in risk assessment when those analytic tools are used in clinical decision support algorithms.
    Cheers,
    Mike


    ------------------------------
    Michael Feehan
    VP Consulting, RWE Regulatory and Safety
    Lancaster NY
    United States
    ------------------------------



  • 11.  RE: Risks in software developing as medical device

    Posted 21-Jul-2021 01:44
    Edited by Vikram Upadhya 21-Jul-2021 01:55
    Hi Mike, 
       Interesting that you have connected the AI/ML as part of Software risk. Only Known intended uses with predictable outcomes which are being supplanted by adequate training/validation data sets may be modelled.
    Known risks could be gauged but unknown risks are another matter which cannot be determined with the classification data at hand. In theory the use of GMLP process should help reduce risks, however risks probability can neither be exhaustively covered nor predicted and the checklist attached lists use of backup mechanisms and possible mechanism that should notify the user incase of failure to classify.

    here are some references:
    https://www.xavierhealth.org/s/AI_GmLP-Checklist-gyxj.xlsx
    https://www.fda.gov/files/medical%20devices/published/US-FDA-Artificial-Intelligence-and-Machine-Learning-Discussion-Paper.pdf

    ------------------------------
    Vikram Upadhya
    Tech Lead
    MDD-MDR Transition
    HCL Tech
    India
    ------------------------------