What makes AI risk distinct
AI risk is the risk that a third party's use of AI affects your data, your decisions, or your obligations in ways you didn't approve or can't see. It isn't a standalone risk domain. Its risk shows up inside the domains you already assess, like security, privacy, operational resilience, and compliance. To assess it, your team needs answers to five questions about each relevant vendor.
Three traits make AI risk distinct. First, vendors embed it inside products you already use, so a tool you approved for one purpose can start drafting content, scoring records, or routing decisions without a new procurement event. Second, AI changes often: a vendor can swap the underlying model, add a data source, or move to a different AI provider within a single release cycle. Third, AI increasingly shapes decisions that matter, like which transaction gets flagged or which claim gets escalated.
Why a separate AI questionnaire falls short
Many teams respond to AI by adding a separate AI questionnaire to their vendor assessments. That is a contrived way to look at AI, because it treats AI as its own category when its influence runs across your entire technology and third-party stack.
A standalone questionnaire also inherits the usual limits: the answers are self-reported and describe a single moment. A statement like "a human reviews all outputs" is easy to write and hard to validate, and the AI feature you assessed on March 1, 2026 may not be the one processing your data on September 30, 2026. A better approach is to build the five questions below into the security, privacy, and resilience reviews you already run.
What to assess: five questions
1. What does the AI do?
Ask which functions use AI and which decisions each one informs, including features the vendor turns on by default. A feature that suggests email subject lines carries different risk from one that scores loan applications. Useful evidence is a current inventory of AI features, with the purpose of each and who relies on its output.
2. What data does it use?
Find out whether your data trains or fine-tunes any model, where it is stored, and who else processes it. Many vendors send data to a foundation model provider, which makes that provider your fourth party. Useful evidence includes data flow documentation, a list of AI subprocessors, and contract terms on training use and retention.
3. Where does a human review the output?
Ask where review happens, who does it, and what happens when the reviewer disagrees with the system. "Human in the loop" can mean anything from approving every output to sampling a small share. Ask for a description of the review step for each high-impact function, including how overrides are recorded.
4. What evidence supports the claims?
Ask for testing results, evaluation methods, model documentation, and control descriptions. A vendor that can't show how it tests its AI is telling you something.
5. How will you hear about changes and incidents?
Agree on what counts as a material change, like a new model provider, a new AI feature, or a new use of your data, and how quickly the vendor will notify you. AI incidents include more than breaches: a model producing wrong outputs at scale is also an incident. Look for notification terms in the contract and a named contact for AI-related issues.
Scale the depth to the use case: a system that touches regulated decisions deserves all five, plus evidence your team reviews directly.
What good evidence looks like
The evidence question deserves the most attention, because AI claims are easy to make and hard to check. You learn more about an AI system by seeing how it performs on realistic work than by reading its description. This is how the AI research community evaluates models: benchmarks like SWE-bench measure whether a model can resolve real software issues, not whether its provider says it can.
At Coverbase, we built TPM-Bench to apply that idea to third-party management, testing how AI performs on tasks like security due diligence and contract review, with and without supporting documents. TPM-Bench is not an AI risk certification, and it doesn't measure any vendor's AI risk. It illustrates the kind of evidence worth asking any vendor for: results on realistic tasks, a clear explanation of how they were measured, and where performance falls short.
Where AI can do more for risk teams
The same standard applies when you evaluate AI for your own risk program. Many AI use cases in TPRM are superficial: they analyze documents, questionnaires, and other text faster, but they leave the underlying problems in place. The bigger opportunities usually sit in work your team doesn't do today because it lacks the bandwidth, access, or data.
AI makes that work possible. Three examples show the difference.
Take continuous monitoring. Most programs watch vendors from the outside, through security ratings, news, and breach alerts, because looking inside every application would take more people than any team has. AI can review how your third-party applications are configured and used from the inside, like who has access, which integrations are connected, and where data flows, and flag changes as they happen.
Intake is the second. Today, a business owner fills out a form, and the risk team spends days chasing the context the form missed. AI can ask follow-up questions in the moment, based on what the person has already said, so the assessment starts with a clear picture of the use case, the data involved, and the decisions the vendor will touch.
Scenario planning is the third. Few teams have time to model how an incident at one vendor would spread to the vendors and fourth parties connected to it. AI can map those connections and draft tabletop exercise scenarios that test dependencies a questionnaire would never surface.
When you assess an AI tool for this kind of work, ask it the same five questions you ask any vendor. The answer to the evidence question matters most, because the value of this work depends on how well the AI performs it.
Where to start
You don't need a new AI domain or a new AI questionnaire. Build these questions into the reviews you already run, starting with the vendors whose AI touches your most important data and decisions. Then set reassessment triggers for model changes, new AI providers, and new uses of your data, so you don't wait for the annual review to learn what changed.