Determine suitability of technical approaches

4 minutes Author: Suresh Balu and Mark Sendak

Why does this matter?

To maximize chances of success. By investing in internally prioritized problems well suited for AI. Be aware of the types of problems AI can and cannot address.
The ground is shifting. AI product developers are validating new capabilities at an impressive rate. Verify the validity and reproducibility of performance claims in your organizational context.
Ensure appropriate investment in AI products. To avoid wasting time and resources on AI products, it’s crucial to ensure they are appropriate for addressing the root cause of the problem in their context of use.

How to do this?

Step 1: Ensure understanding of AIThe ability of a machine or computer to perform cognitive functions typically associated with human minds, such as learning and problem-solving. and machine learningA subfield of artificial intelligence that uses algorithms to analyze data, learn from it, and make informed decisions.

Review available resources even if the concept seems clear:
- What is AI? We drew you a flowchart to work it out.
- What is machine learning?

Step 2: Identify the root causes of the problem that are NOT amenable to technology

Clearly identify the root causes of the prioritized problem. Refer to the guide on determining the dimensions of the problem for instructions.
For each root cause, consider the questions below to identify problems that are not amenable to technology.

Problem Root Cause Category	Questions that Confirm Root Cause Category (if Response is Yes)	Is the Root Cause Amenable to Technology?
Behavioral root cause	Do people need to treat each other differently to address the problem? Do different parts of the organization need to collaborate in new ways to address the problem?	No
Incentive misalignment root cause	Are people not incentivized to work together to address the problem? Are there strong entrenched interests that benefit from the status quo? If relevant stakeholders had complete and accurate information, would they not be empowered to take action to address the problem?	No
Knowledge gaps root cause	Do people not agree on the best thing to do to address the problem? If relevant stakeholders had complete and accurate information, would they disagree on the optimal decision?	No
Organizational barriers root cause	Are there compliance, security, or privacy concerns that must be addressed? Does the organization lack the relevant staffing or resources to address the problem? Are there policies that shape how healthcare is provided that need to change to address the problem?	No

Technologies will not fix the above challenges. Only consider technology as a potential solution for root causes that do not fall within any of the categories above.

Step 3: Identify the root causes of the problem that are amenable to technology

Identify the root causes that were not eliminated in the prior step. Review the categories of problems below to determine if the problem is amenable to technology:

Problem Root Cause Category	Questions that Confirm Root Cause Category (if Response is Yes)	Is the Root Cause Amenable to Technology?	Most Relevant Technology to Address the Root Cause
Communication root cause	Is information that already exists within the organization poorly transmitted and shared with relevant stakeholders? Is information presented to relevant stakeholders in a way that increases cognitive load on frontline clinicians?	Yes	Communication tools, visualization software
Notification root cause	Are there delays in how important information is made available to relevant stakeholders?	Yes	Alerting systems, notification engines
Coordination root cause	Do groups of individuals have difficulty rapidly convening and coordinating action in response to new information? Do groups of individuals have difficulty efficiently delegating and completing tasks?	Yes	Workflow software, task management tools
Insight generation root cause	Do individuals need additional information that is not currently available to make optimal decisions? Does historical data need to be rapidly synthesized to inform a current decision?	Yes	AI / ML

“AI is good in areas where we need efficiency or repeatability. So, first is where we can use AI or ML to gain some efficiencies, where there are repeatable sequences of events or a simple decision logic that [the AI product] can go through [instead of] a human being and let them do more integrative thinking.”
Technical Expert

“Where there’s greatest clinical uncertainty, we’re using a data set to make better predictions. That’s where I see the greatest utility.”
Technical Expert

Step 4: Take a pause, and make sure you’re not missing anything

Stop for a moment- and go through the scenario below:¹

Imagine that an AI product that addresses the root cause of the problem exists and is freely available, simple to install, and easily integrated into clinical workflows. The healthcare delivery organization takes the AI product and implements it. After 12 months of implementation, the integration fails spectacularly without any clinical or operational benefit. The AI product is then decommissioned (see the guide on determining if updating or decommissioning is necessary) and a final impact assessment is conducted.

Take 15 minutes to write down every reason why the AI product implementation failed spectacularly.
This will likely include non-technical threats not related to AI. Don’t lose track of those non-technical threats, and work on addressing them. Remember that well-performing technologies often fail at solving problems.

References

More information about project premortem on Harvard Business Review.

contribute to the topic guide

take the survey

adopt health ai

join us