Guides
Define the role of AI
Why does this matter?
Using the exact same AI product differently can significantly alter obligations and the impact of the product. Departure from an intended scope of use can erode the careful work prioritizing problems and matching AI solutions through decision points to identify and prioritize a problem and define AI product specifications. Whether the AI product is developed internally or externally, a clear understanding of the intended scope of use helps establish expectations for the AI product.
How to do this?
Step 1: Define the scope of the AI product
- Consider the clinical, operational, technical, and/or financial goals and constraints that surfaced in key decision points on identifying and prioritizing a problem and defining AI product specifications.
- Write a clear and concise intended use statement.
- What does the AI product do?
- Who is the intended user?
- Who does the AI product impact?
- Where will the AI product be used?
- What hardware or other systems will the AI product interact with (if any)?
- What limitations should be applied (if any)?
Step 2: Evaluate the scope
- Iterate on the scope with multi-disciplinary stakeholders. Stakeholders involved in this process should include: clinicians familiar with the target condition and context of use; data scientists and statisticians familiar with the modeling approach; software engineers or IT personnel familiar with the data source systems and legacy technologies used in the implementation context; regulatory or legal experts; and patient or community representatives with lived experience with the target condition and context of use.
- Assess the consequences and obligations for fulfilling the intended use statement.
- Be sure to consider factors that could influence patient access to care and the potential to minimize the negative impacts of inequitable care.
- Once there is agreement on the scope of use, formalize the use statement and distribute the document to clinical champions, business unit leaders who oversee the quality of care in the implementation context, and technical leaders involved in the integration.
“We need to have a clinician [as dedicated champions of the technology]… they need to believe that such a tool actually has a place in their workflow and [they can] name the specific things that they would do in response to that alert.”
Technical Expert
Step 3: Define constraints
- Anticipate that frontline clinicians, operational leaders, and technical leaders will intentionally or unintentionally seek to broaden the use of the AI product. Individuals within the organization will perceive extensions of product use as efficient opportunities to scale impact, as completing the procurement process requires significant time, effort, and resources.
- Ask clinical champions and business unit leaders to identify adjacent use cases for the AI product up front. For each adjacent use case, decide whether to broaden the intended use statement or to constrain the use of the AI product.
- For all adjacent use cases determined to sit outside the intended use, provide concrete warnings and contraindications for inappropriate use of the AI product. Provide a rationale for why the product should not be used. For example, if there is no data to determine the safety or effectiveness of an AI product outside a specific population, explicitly state that the product is not validated outside the target population.
Step 4: Identify dependencies
- AI products put into clinical practice become highly intertwined with clinical workflows, data source dependencies, and changes in clinical practice.
- Review the table below to identify different types of dependencies and take appropriate action.
| Dependency Type | How to Identify the Dependency | Actions to Take |
|---|---|---|
| Data source | Review all data elements that go into the AI product. Identify all technologies involved in the generation, transformation, and capture of all data elements. |
|
| Clinical workflow | Review all data elements that go into the AI product. Identify all workflow steps involved in the generation, transformation, and capture of all data elements. |
|
Additional resources
- To learn more about how AI accumulates hidden technical debt:
- To learn more about the dynamic nature of AI product lifecycle management:
- Guides for monitoring AI products and monitoring affected work environments
contribute to the topic guide
