Determine if updating or decommissioning is necessary

Why does this matter?

Updating and decommissioning AI products helps ensure that healthcare providers always have the best tools to care for their patients. By staying on top of updates and knowing when to retire AI tools, healthcare delivery organizations can create a safe, efficient, and modern environment for patients and staff.

• Regular updates can make AI products work better. It allows products to adjust in response to changes in data, stay in line with regulatory rules, and offer new features. 

• Sometimes, it’s necessary to retire an AI product. This can happen when the product is outdated, doesn’t meet new rules, no longer addresses current patient needs, or if there is a termination of contract with an outside vendor or partnership. Decommissioning these products allows healthcare providers to focus on using the most effective tools. 

“If it’s not moving anything positive for our patients, and we get negative feedback from our clinical teams, then those would be the kind of cases where we’d say, Okay, this is not actually relevant.”

Technical Expert

How to do this?

Step 1: Continuously monitor AI products and analyze outcomes to decide when to take action

• Establish a change-management process informed by a cross-functional team that monitors performance metrics, user feedback, and any regulatory requirements. This team should include clinicians who are familiar with care delivery within the implementation context, data scientists familiar with the AI product, IT leaders familiar with data source systems used in the implementation context, and operational leaders responsible for quality of care in the implementation context.
• Analyze performance metrics of AI outputs, clinical use, and health outcomes for warning signs that do not meet pre-specified thresholds and require action. 
• Monitor changes in regulations, the emergence of new technologies, and complaints from users and patients that could require the current product to be decommissioned. Continue to identify potential risks.

“Let’s say the algorithm does seem to be getting good performance [metrics], clinicians aren’t complaining that much about it, but empirically, you see, their usage drops off a bit. Then that’s [an indication] let’s do a user study. We can keep using the algorithm, but we need to talk to a half dozen people about whether they’re feeling alarm fatigue, or how we can make the interface more user friendly.”

Bias Expert

For internally developed AI products

Step 2: If a decision is made to update the product, implement a design and development plan 

• Draft a design and development plan and consider:
  • Assessing the impact of the change
  • Retraining staff
  • Updating documentation
  • Conducting testing to ensure the update will function as expected
• Consider returning to earlier lifecycle stages, such as defining the role of AI to confirm the scope of use of the updated AI product.

Step 3: If a decision is made to decommission the tool, develop a plan to transition to an alternative product or eliminate the product altogether

• Communicate the plan to decommission a product with clinical staff, patients, and the impacted community.

For externally procured AI tools

Step 2: Follow the vendor’s process for updating or decommissioning the product. 

• Establish early in the procurement process how the vendor updates and decommissions the AI product. 
• Maintain a continuous relationship with the vendor to ensure successful product management throughout the lifecycle. 
• Work with the vendor’s development teams to review any updates or new versions before clinical integration. 
• If the vendor decommissions the product, work with the same vendor or alternative vendor to transition to an alternative product. 
• Communicate any changes to clinical staff and patients and provide training on any impacts from the new update.