Audit investment decisions

Why does it matter?

Procurement audits are typically conducted to improve efficiency, and compliance, reduce risk, and prevent fraud. When investing in new and rapidly changing technologies, such as AI-based solutions, it is essential to document the decision process accurately, evaluate all identified risks, develop a risk mitigationThe process of reducing or minimizing the potential harm or adverse effects associated with a particular risk or hazard. plan, and ensure effective operating controls. 

To generate objective insight. Auditing AI procurement ensures:

• Compliance with all laws and regulations, 
• Identifies and documents risks, 
• Helps appropriately prioritizes through a risk assessment framework
• Develops a mitigation plan
• Verifies effective operating controls
• Refines processes to manage risks to achieve planned outcomes. 

This helps generate objective insights to surface gaps in the procurement process and document changes.

To develop trust. AI procurement audit assures stakeholders (especially front-line clinicians) that investment decisions are reliable and risks are evaluated. They also ensure investment decisions meet legal requirements and provide legitimacy for the procurement process. 

To understand the return on investment. Document the scope and range of potential return on investment for AI solutionsThe combination of the AI product and its use in healthcare delivery setting (including user experience and workflow of use) , including quality and financial metrics. Ensures accurate representation of economics.

How to do this?

To audit your organization’s decision to invest in an AI-based solution, follow these steps:

Step 1: Gather preparatory information

• Collect and review past contracts and transactions for internal and external suppliers. 
• Understand how the procurement team verifies that all parties are upholding the agreed-upon terms and conditions. 
• Ascertain that the procured solutions are adding value to the organization.

Step 2: Define audit scope

• Determine one or two specific recent AI solution investment decisions that will be audited. 
• Clarify the objectives, stakeholders, and boundaries of the audit.

Step 3: Review documentation

• Gather and review all relevant documents, such as investment policies, guidelines, and procedures. 
• Look for formal documentation of the decision-making process, including reports, analysis, and risk assessments.

Step 4: Identify key stakeholders 

• Identify all stakeholders involved in the decision-making process. 
• Determine their roles and responsibilities and assess their level of involvement in the process.

Step 5: Evaluate the decision-making criteria

• Assess the decision-making process based on the criteria established in the policy and procedure documentation. 
• Determine whether the criteria are capable of supporting AI-based products and services. 
• Evaluate whether the process was followed correctly and whether the criteria for making the decision were precise, reasonable, and appropriate for AI-based solutions.

Step 6: Review the decision outcome

• Evaluate the outcome of the decision, including the investment decision itself and the subsequent results. This will include results of AI solution monitoring, work environment monitoring, and post-implementation audits.
• Determine whether the decision achieved the desired measures of success and value creation. 
• Ensure that unintended consequences are tracked and that learnings are incorporated into the procurement process.

Step 7: Assess risk management 

• Evaluate the methods used to identify, assess, prioritize, and manage risks. 
• Determine whether risk mitigation efforts were effective.

Step 8: Verify the accuracy and quality of the solution

• Evaluate the consistency of the methods used to collect, evaluate, and report the performance of the solution.
• Ensure that solution-related data used in the decision-making process are accurate and complete and that results are applicable to the organization.

Step 9: Determine compliance 

• Determine whether the decision-making process complied with all relevant regulatory requirements, laws, industry standards, and internal policies and procedures.

Step 10: Identify areas for improvement

• Recommend appropriate actions to address any deficiencies in the decision-making process and identify areas for improvement.

Step 11: Report findings

• Prepare a report summarizing the findings of the audit and recommendations for improvement. 
• Present the report to the relevant stakeholders and follow up to ensure that the recommendations are implemented.

“We are doing due diligence. We’ll ask the developers and principal investigators, read through the literature, look for the FDA processes, which are very nascent right now, but to see where things are in the regulatory process in terms of filings, etc. So especially on the business side, there’s a fair number of folks who will work, or at least are empowered, to do some due diligence about the programs.”

Clinician

If your organization does not have a specific process for the procurement of AI tools, you can adapt your existing software procurement process and extend it for AI-based products. Here are the steps to consider:

Step 1: Assess needs 

• Identify the problem or opportunity the AI solution is expected to address. 
• Document the desired outcomes, goals, and objectives for the AI solution.

Step 2: Define technical specifications

• Define the technical specifications for the AI solution, considering factors such as data elements, cadence, and integration with existing infrastructure and workflow.

Step 3: Develop a procurement strategy 

• Develop a procurement strategy that outlines the procurement process for AI tools. This should include criteria for vendor selection, evaluation of proposals, and contract negotiation.

Step 4: Identify vendors 

• Research and identify potential suppliers of AI solutions that meet the organization’s technical and business requirements.

Step 5: Tendering process: 

• Create an RFP template for AI solutions that incorporates your organization’s requirements. 
• The RFP should include full technical specifications, AI technology and solution evaluation criteria, and a timeline for selection.

Step 6: Evaluate proposals

• Evaluate proposals based on requirements, specifications, and established evaluation criteria for AI solutions. 
• Consider factors such as retrospective and silent performance evaluation, vendor experience, technical capabilities, and pricing. 
• Fully specify metrics to determine the suitability of the AI model and solution for clinical use. Examples include : AUROCArea Under the Receiver Operating Characteristic Curve(AUROC) is a metric used to evaluate the performance of a binary classification model by measuring the ability of the model to distinguish between positive and negative classes. targets, AUPRCArea Under the Precision-Recall Curve(AUPRC) is a metric used to evaluate the performance of binary classification models, particularly in cases where there is a class imbalance. targets, interpretabilityThe ability to explain or to present an ML model’s reasoning in understandable terms to a human The ability to understand the value and accuracy of system output. Interpretability refers to the extent to which a cause and effect can be observed within a system or to which what is going to happen given a change in input or algorithmic parameters can be predicted.requirements, performance by sub-cohorts of interest, etc.

Step 7: Conduct supplier due diligence

• Conduct due diligence on the financial, operational, and strategic stability of the supplier. 
• Prior expertise and experience in delivering AI solutions should also be considered.

Step 8: Develop contracting terms 

• Ensure the contract includes terms and conditions such as liabilities, outcome confidentiality, data privacy and security, secondary use of data, data ownership, intellectual property rights, and service line agreements (SLAs).

Step 9: Develop a process for integration and monitoring of the AI solution

• Develop an appropriate process for AI integration into the organization’s workflow. 
• Establish infrastructure and a process for monitoring the performance of the AI tool and monitoring the affected work environment.
• Also, consider measuring and monitoring the effectiveness of the solution in achieving the planned outcomes.