An ultimate essence and distinction of ISO 14971 risk management compared to traditional FMEA is the extrapolation of reasonably foreseeable sequences of events and hazardous situations into reasonably foreseeable "harms" rather than just product-centric or process-centric failure modes and effects.
ISO 14971 officially defines "harm" as damage or injury to the health of people [paraphrased]. In these terms then, I would suggest that descriptors like "unnecessary treatment", and "no treatment given" are hazardous situations rather than harms. "Progression of disease" is a borderline descriptor containing attributes of a hazardous situation and a harm. Consequently I typically encourage folks to try and press that descriptor further into a projected adverse health consequence(s) (if any).
In the IVD industry, especially in the FDA jurisdiction, it is common and expected that the sequences of events (i.e., the device or manufacturing process failure modes and effects, i.e., FMEA) and consequent hazardous situations (i.e., false negatives, false positives, delayed results, and inconclusives) be extrapolated into reasonably foreseeable harms (i.e., damage to the health of people).
I would agree that from a process analysis perspective (e.g., from a process FMEA perspective, i.e., from the perspective of deriving process-centric sequences of events that could lead to hazardous situations), identifying clinical harms at that step of the analysis doesn't yet provide direct value. ISO 14971 and ISO/TR/DTIR 24971 recognize this by reminding us that FMEA is just one part of the multifaceted risk analysis process. Specifically, I'm suggesting that ISO 14971 for practical intents and purposes calls for traditional FMEA to be embedded within ISO 14971's steps for identifying sequences of events and hazardous situations. Only once we extrapolate these into reasonably foreseeable harms can we say we've met the intent of ISO 14971.
Remember also that ISO 14971 prescribes for risk to be estimated via the combination of probability of harm and severity of harm (i.e., of damage/injury to the health of people), not solely on probability/severity of product/process failure modes (though the probability of a failure mode is certainly a variable to be factored when estimating probability of harm). Consequently, our ISO 14971 risk analysis severity ranking categories need to be articulated in terms of harm, not in terms of failure modes. For example, the minor severity level might be characterized as minor discomfort not requiring clinical intervention; a moderate severity level might be characterized as non-serious injury or injury requiring clinical intervention to prevent serious injury; and a catastrophic severity level would be death. Though this may seem to infringe on the goal of avoiding a one-size-fits-all approach, it really doesn't; don't forget that we are to pair the severity ranking paradigm with organically-estimated reasonably foreseeable harms (e.g., skin rash, headache, dizziness, heart attack, death, etc.) when calculating the risk. It is via the organic derivation of reasonably foreseeable harms for the particular subject device that ISO 14971 alleviates and overcomes perceived concerns related to the generic nature of the severity ranking categories.
For IVDs in particular, ISO/TR/DTIR 24971 states that this requires sufficient specificity to assign appropriate severity. It guides us to answer the question "What harms might occur from a misdiagnosis or inappropriate therapy?". Examples of "harms" given by ISO 14971 include things like heart fibrillation, infarct, brain damage, organ damage, decreased consciousness, death, infection, progression of disease, etc. Such granularity needs to be preserved a) to prove that we've actually organically thought about the actual risks associated with the IVD instead of slighting it; b) so that the Quality/Regulatory Team can more effectively manage the product's market feedback; and c) objectively rather than subjectively meet the intent of ISO 14971.
I find it refreshing to know that this stuff isn't just a bunch of theoretical gibberish. For example, I worked on two IVD FDA premarket submissions in the last couple years where in one of those cases, the Sponsor properly articulated harms by naming specific disease / injury conditions that were germane for their product and its intended use. In the other case, the Sponsor instead attempted a more generalized approach where specific clinical harms were not clearly articulated. That case resulted in a submission hold. Here is a redacted excerpt of the objections FDA lodged in that case, which shows the importance of being very specific regarding the harms:
"…Under your Risk Management Plan document you identify the criteria used to evaluate risk severity. After review of these criteria, it is apparent that your risk severity criteria were not constructed in consideration of the possible severity of harm to the patient. For example, you list for a severity of critical as 'loss of function – no results' and severity of catastrophic as 'False Positive – Unnecessary observation'. These criteria for severity do not specifically address the severity of harm to the patient and are incongruent with the intended use of the [redacted] test…You should revise your Risk Management Plan by constructing risk evaluation criteria for severity which might more appropriately be applied when assessing risk of harm to a patient. When constructing the risk evaluation criteria for severity of harm, you should consider as an example of high severity the possible harm to a patient, i.e. [redacted…and tested as false negative]…Please provide for our review, your revised Risk Management Plan which incorporates severity criteria pertinent for assessing risk of harm to the patient…"
Remember that because of precedents like this, as well as the fundamental standardization of these principles in ISO 14971, it is expected to be commonplace that firms will articulate the reasonably foreseeable injuries/damage to the health of people that could come from false, delayed, or inconclusive diagnostic test results (or, for non-IVD devices, other various hazardous situations). I can name multiple IVD companies who are in alignment with these precedents and I've been the primary architect and author of the remediation that was needed to resolve FDA objections for firms who weren't.
Note also that these principles are generally uniform regardless of device technology (i.e., IVD or non-IVD), so they are applicable to any medical device company who is genuine about performing useful, organic risk analysis that will help maintain public health as well as the Sponsor's long term profits in the medical device marketplace.
Hope this helps.
------------------------------
Kevin Randall, ASQ CQA, RAC (Europe, U.S., Canada)
Principal Consultant
ComplianceAcuity, Inc.
Golden, CO
United States
www.complianceacuity.comNote that I'm now far older and even uglier than in the photo above. Brace yourselves for my updated photo coming soon.
© Copyright 2020 by ComplianceAcuity, Inc. All rights reserved.
------------------------------
Original Message:
Sent: 19-May-2020 15:43
From: Anonymous Member
Subject: Risk Management Severity Matrix - Feedback Requested
This message was posted by a user wishing to remain anonymous
We are trying to move away from a one-size fits all risk severity/rating and description approach to allow for a better risk scoring outcome for manufacturing. Ultimately, we're trying to tie processing risks (i.e. production delays, reprocessing, scrap) to patient risks without trying to break down these risks into potential clinical harms since they may be unknown or difficult to determine.
For example, a patient sample mix-up during processing could lead to a patient receiving a false positive, false negative, or physiologically impossible result depending on the symptoms they're exhibiting. Each of these hazards could present different clinical harms (i.e. unnecessary treatment, no treatment given, progression of disease). From a process analysis perspective, trying to identify these potential clinical outcomes does not provide much value since ultimately we know the risk is that the test result would be impacted, and our mitigation activities would be focused on controls to prevent mix-ups. Using our proposal, we would identify the hazard (incorrect sample processed) potentially leading to an incorrect test result being obtained, which would be our highest severity ranking since from a processing perspective since our concern wouldn't necessarily be whether or not the patient would not receive necessary treatment but rather they would receive an incorrect report.
Ultimately we're trying to use that same methodology throughout our analysis of processes and identifying where failures can occur that would impact the test result, not necessarily trying to break down the possible outcomes of an incorrect test results since this is done in our Hazard Analysis and User/Design FMEA. Overall, this process is similar to our software risk management activities where they define software severities on its own scale (i.e. annoyance to user, no result generated) and do not identify clinical harms. That process allows them to conduct their risk analysis within the scope of their own work, but still be within the framework of our risk management process.
As reference, here is a draft of what we're hoping to implement. As you'll note the process severities align clinical risks, so there shouldn't be an issue of low patient risks correlating to high manufacturing risks and vice versa.
Really, we're just thinking it's a more appropriate scaling method for the end users and the activity being assessed. In addition, I would think an auditor would appreciate us making use of an approach that is more suited for the activity being assessed and which results in more meaningful and actionable risk outcomes.
As always, I welcome comments from those interested in offering their perspetive. Regardles, wishing everyone healthy vibes!