Design and test workflow for clinicians

Why does this matter?

Effective use of an AI product requires optimizing the workflow and user experience. Focusing only on technical aspects of the product without considering its end users can diminish the impact of the AI product. Poor workflow and user experience can add a burden to end users, prompt resistance to the product, and undermine product adoption. In the worst case, they can exacerbate the problem that the AI product was meant to address.  

Beware that vendors are incentivized to promote the use of their products rather than ease the burden on frontline clinicians. Proactively resist the EHR trap. Only considering workflows that are easily configurable and implementable within the EHR will overlook approaches that could further ease the burden on front-line clinicians. Consider workflows beyond the EHR and if an EHR vendor strongly resists making data accessible to third-party AI product systems, consider submitting an information-blocking claim through the office of the national coordinator for health IT. If the EHR is ONC-certified, the vendor must comply with the 21st Century Cures Act’s final rule. Learn more about ONC Certification Criteria here.

How to do this?

Step 1: Interview and observe care delivery within the proposed context of the use

• Understand the current care delivery process within the proposed context of use defined earlier.
• Have team members spend time observing patient care in the context of use and interview end users. 

Step 2: Build a patient journey map for the condition and care pathway targeted by the AI product

• Build a patient journey map to understand the patient experience. 
• Map out actions, motivations, questions, and barriers in the journey map. 
  • Actions: What does a patient do at each step?
  • Motivations: Why does a patient go from one step to the next? 
  • Questions: What uncertainties, jargons, or issues prevent a patient from progressing from one step to the next? 
  • Barriers: What structural process, cost, inconvenience, or other barriers stand in the way of going from one step to the next? 
• Include steps that are out of the healthcare delivery organization’s control in the journey map. For example, a patient’s insurance may not cover something required to go to the next step. Even if it’s outside the healthcare delivery organization’s control, include it in the journey map because the challenge will continue to exist even after the AI product rollout.
• Validate the journey map with various stakeholders, including frontline clinicians and managers from different professions and business units. Also make sure to validate the journey map with patients who are familiar with the care pathway. 

Step 3: Identify the precise moment and context to implement the AI product

• Identify information related to the 5 Rights of clinical decision support.
  • The right person: Who in the healthcare delivery setting is best equipped to act on the information generated by the AI product?
  • The right information: What information, beyond the output of the AI product, does the user need to take appropriate action?
  • The right time/point in the workflow: When is the best time to present new information generated by the AI product to improve decision making and the care pathway?
  • The right context: What should the user be doing when they are presented with the new information generated by the AI product?
  • The right channel: How should the user receive the new information generated by an AI product?
• Ensure that the point of intervention (person, context, communication channel) is within the healthcare delivery organization’s control. If the healthcare delivery organization does not control the person, context, or communication channel, external partnerships may be needed to successfully integrate the AI product.
• Ensure that clinical end users feel confident that information generated by an AI product would remove friction in the care pathway. Friction is anything that gets in the way of a patient or clinician completing a task.

“Our lab has done some work on an intelligent autocomplete for filling out triage notes in the emergency department. When I first heard about that, I thought this is a no brainer, clinicians are going to love having autocomplete. It has not been as popular as you would expect. It’s not because the algorithm is wrong…but it doesn’t fit in their workflow. They have to train it…this wasn’t what they wanted.”

AI Bias and Fairness Expert

Step 4: Build a minimal viable product

• Mock up a user interface for different affected clinician stakeholders. 
• Present end users with a prototype interface and iterate with them until they are pleased with the way information generated by the AI product is presented. 
• Build out storyboards that visualize different user interactions the AI product is trying to promote. For example, visualize the different tasks that users should be able to complete using the AI product.
• Ensure that end users are satisfied with the interactions and that they are able to effectively use the AI product.

Step 5: Develop a swimlane diagram visualizing workflow steps, technology use, and social interactions

• Build out a swimlane diagram that visualizes the effective use of the AI product, including details about clinical decision-making and actions. 
• Present this workflow diagram to all clinical stakeholders who are actively involved in or affected by the workflow and ensure that stakeholders are satisfied. 
• Identify the decision-makers who are best positioned to approve changes to workflows across clinical groups. For example, nursing leaders may need to be engaged in addition to nurse end users. 

“We [work] iteratively with our pilot sites. We come up with what we think that design should look like, get their input, build it, [then] go through user acceptance testing”

Technical Leader

Additional resources

To learn more about different types of friction and how to identify and remove friction in customer experience: Amazon’s Friction-Killing Tactics To Make Products More Seamless

To learn more about information blocking, here is content provided by ONC:

• Anyone who believes they may have experienced or observed information blocking by any health care provider, health IT developer of certified health IT, or health information network or health information exchange is encouraged to share their concerns with us through the Information Blocking Portal on ONC’s website, HealthIT.gov.
• ONC has the authority to review claims of potential information blocking against health IT developers of certified health IT that may constitute a non-conformity under the ONC Health IT Certification Program. Separately, OIG has the authority to investigate claims of potential information blocking across all types of actors: healthcare providers, health information networks and health information exchanges, and health IT developers of certified health IT. Therefore, upon receiving a claim of potential information blocking, ONC shares the claim with OIG. ONC makes every effort to share these claims of information blocking within two business days of receipt.
• See the following FAQ for more information on how information-blocking actors’ acts or omissions (“practices”) would be evaluated to determine whether the unique facts and circumstances constitute information blocking: Information Blocking
• The following fact sheet illustrates what happens when a claim is submitted to the information blocking portal:  Information Blocking Portal Process.