Determine suitability of technical approaches
Why does this matter?
- To maximize chances of success. By investing in internally prioritized problems well suited for AI. Be aware of the types of problems AI can and cannot address.
- The ground is shifting. AI product developers are validating new capabilities at an impressive rate. Verify the validity and reproducibility of performance claims in your organizational context.
- Ensure appropriate investment in AI products. To avoid wasting time and resources on AI products, it’s crucial to ensure they are appropriate for addressing the root cause of the problem in their context of use.
How to do this?
Step 1: Ensure understanding of AIThe ability of a machine or computer to perform cognitive functions typically associated with human minds, such as learning and problem-solving. and machine learningA subfield of artificial intelligence that uses algorithms to analyze data, learn from it, and make informed decisions.
- Review available resources even if the concept seems clear:
Step 2: Identify the root causes of the problem that are NOT amenable to technology
- Clearly identify the root causes of the prioritized problem. Refer to the guide on determining the dimensions of the problem for instructions.
- For each root cause, consider the questions below to identify problems that are not amenable to technology.
|Problem Root Cause Category||Questions that Confirm Root Cause Category (if Response is Yes)||Is the Root Cause Amenable to Technology?|
|Behavioral root cause||No|
|Incentive misalignment root cause||No|
|Knowledge gaps root cause||No|
|Organizational barriers root cause||No|
- Technologies will not fix the above challenges. Only consider technology as a potential solution for root causes that do not fall within any of the categories above.
Step 3: Identify the root causes of the problem that are amenable to technology
- Identify the root causes that were not eliminated in the prior step. Review the categories of problems below to determine if the problem is amenable to technology:
|Problem Root Cause Category||Questions that Confirm Root Cause Category (if Response is Yes)||Is the Root Cause Amenable to Technology?||Most Relevant Technology to Address the Root Cause|
|Communication root cause||Yes||Communication tools, visualization software|
|Notification root cause||Yes||Alerting systems, notification engines|
|Coordination root cause||Yes||Workflow software, task management tools|
|Insight generation root cause||Yes||AI / ML|
“AI is good in areas where we need efficiency or repeatability. So, first is where we can use AI or ML to gain some efficiencies, where there are repeatable sequences of events or a simple decision logic that [the AI product] can go through [instead of] a human being and let them do more integrative thinking.”Technical Expert
“Where there’s greatest clinical uncertainty, we’re using a data set to make better predictions. That’s where I see the greatest utility.”Technical Expert
Step 5: Take a pause, and make sure you’re not missing anything
- Stop for a moment- and go through the scenario below:1
Imagine that an AI product that addresses the root cause of the problem exists and is freely available, simple to install, and easily integrated into clinical workflows. The healthcare delivery organization takes the AI product and implements it. After 12 months of implementation, the integration fails spectacularly without any clinical or operational benefit. The AI product is then decommissioned (see the guide on determining if updating or decommissioning is necessary) and a final impact assessment is conducted.
- Take 15 minutes to write down every reason why the AI product implementation failed spectacularly.
- This will likely include non-technical threats not related to AI. Don’t lose track of those non-technical threats, and work on addressing them. Remember that well-performing technologies often fail at solving problems.
- More information about project premortem on Harvard Business Review.