Don't stop here
Hand-picked guides our readers explore right after this one.
Prompts for operations including process optimization, KPI tracking, vendor management, and cost reduction
Read the guideExpert guide to Claude prompts with XML tags, artifacts, and complex reasoning
Read the guidePrompts for OpenAI Codex CLI coding agent for code generation, debugging, and multi-file projects
Read the guideProcurement field guide · Checked August 13, 2026
A capable model is only one part of an AI purchase. The decision also includes data use, security, output quality, human review, billing, support, contract promises, integrations, and the cost of leaving. This checklist turns those questions into evidence you can compare.
Michael Okeje
AI procurement research and vendor-risk analysis · Last updated August 13, 2026
A vendor can be excellent for one workflow and a poor fit for another. Before opening a comparison spreadsheet, write what the system will do, what it will not do, what information it will see, what action it may take, and what a human will review. A drafting assistant that uses public information deserves a different assessment from an agent that changes customer records or influences employment, credit, health, or access decisions.
This order also prevents a common procurement mistake: allowing a compelling demo to define the problem. The demo usually shows the happy path with curated inputs. Your evaluation should include your actual cases, awkward inputs, permissions, review burden, volume, and failure recovery. If a vendor cannot let you test the workflow honestly, that is evidence about the fit.
Use the questions below in the same order for every vendor. Ask for a written answer and record the product tier, region, model, date, and person who answered.
A sales answer can clarify a product, but it may not bind the provider. The FTC's vendor-security guidance recommends spelling expectations out in writing and verifying compliance. Treat material promises about training, retention, security, uptime, incident notice, subprocessors, deletion, and support as contract questions. If a promise matters to the risk decision, it should not live only in a call recording or a sales email.
“Enterprise security” should become a list of controls and evidence. “Accurate” should become representative cases and a rubric. “Your data is private” should become a data-flow and contract review.
If a requirement is essential, state who must do what, by when, with what remedy if it fails. Confirm which documents govern when the marketing page, help center, order form, and contract differ.
If the product cannot meet a requirement, decide whether the use case can change, a compensating control is sufficient, or the vendor should be rejected. Do not bury an unresolved exception in a risk register nobody owns.
Vendor terms, models, subprocessors, and prices change. Store the date and product tier beside each answer, and define what change triggers a re-review.
Use weights that reflect the consequence of failure. For a low-risk writing tool, usability and cost may carry more weight. For a workflow handling customer records, security, privacy, evaluation, and exit should be hard gates rather than points that a slick interface can compensate for.
| Area | Suggested weight | Score 1-5 when evidence is sufficient | Automatic concern |
|---|---|---|---|
| Workflow fit and quality | 25% | Representative test cases pass the required rubric. | Vendor only permits a scripted demo. |
| Data and privacy | 20% | Data use, retention, access, and deletion are clear for the tier. | Answer depends on an undocumented setting. |
| Security and controls | 20% | Access, logging, incident, and evidence match the risk. | No usable access or incident path. |
| Total cost and support | 15% | Three usage scenarios are affordable and support is defined. | No cap or visibility on variable usage. |
| Integration and operations | 10% | The team can implement, monitor, and own it. | Critical knowledge remains vendor-only. |
| Exit and resilience | 10% | Data, fallback, and migration are tested. | No export or service-change plan. |
A score is a decision aid, not a substitute for judgment. Set hard gates for data misuse, unauthorized access, unacceptable quality, and legal or policy constraints. A vendor should not pass because its strengths elsewhere mathematically outweigh a non-negotiable failure.
What happens when the vendor silently changes the model or retrieval behavior?
Can we prove what the system saw, produced, and did after a customer or regulator asks?
What is the cost of a difficult customer, long document, repeated retry, or failed integration?
Who is responsible for reviewing an output that is plausible but wrong?
How do we continue the workflow during an outage, account suspension, price increase, or vendor exit?
These questions move the decision beyond feature comparison. They also connect directly to the site's AI governance guide and AI evaluation guide, which cover ownership, evidence, monitoring, and incident response after the purchase.
Export a sample of the records you would need to preserve: source files, prompts, configurations, outputs, feedback, metadata, and audit events. Ask a second person to use the export without the vendor's dashboard. If the workflow cannot be reconstructed, the business has already accepted lock-in.
Then write the fallback process. It may be slower, manual, or less elegant, but it should preserve the essential customer or operational outcome. Identify the credentials to revoke, integrations to disable, data to delete, customers to notify, and staff who can resume the work. An exit plan is not a prediction that the vendor will fail; it is an acknowledgement that a critical process should have a recovery path.
If the decision is to run a controlled trial, use the 30-day AI pilot plan rather than opening the tool to the whole organization. If the company is building its own agent, see how to build an AI agent and AI agent business models.
Ask what data the vendor collects, stores, uses for training, and shares; where it is processed; who can access it; how outputs are evaluated; what happens when the model or subprocessor changes; how costs are calculated; what support and incident commitments exist; and how you can export and delete your data if you leave. Ask for evidence and contract language, not only marketing answers.
Start with the data and actions the tool will handle, then assess access controls, encryption, authentication, logging, incident response, subprocessors, retention, vulnerability management, and relevant independent reports or attestations. The right level of diligence depends on the use case. A tool handling public brainstorming material does not need the same approval as one processing employee, customer, health, or financial data.
The answer depends on the vendor, product tier, account settings, contract, and jurisdiction. Do not infer the policy from a product slogan. Read the current terms, privacy documentation, data-processing terms, and enterprise agreement, and ask what happens to prompts, files, feedback, logs, and outputs. Record the checked date because these terms can change.
Exit risk is the difficulty and cost of leaving a vendor without losing data, workflow continuity, quality, or customer commitments. It includes proprietary formats, undocumented prompts, embedded integrations, model-specific behavior, long export times, minimum commitments, price changes, and lack of a fallback. Evaluate the exit path before the tool becomes part of a critical process.
Yes, but the assessment can be proportionate. A small business can begin with a one-page record of the use case, data, permissions, vendor claims, cost, quality test, owner, contract terms, incident contact, and exit plan. More sensitive or consequential uses deserve deeper review. The goal is to make an informed decision, not to recreate a large enterprise procurement department.
Give both vendors the same representative tasks, data boundary, success rubric, volume assumptions, and support questions. Compare accepted outcomes, errors, review time, latency, total cost, contract terms, controls, and exit options. Do not compare a polished demo from one vendor with an unconfigured trial from another or treat a benchmark score as proof of fit for your workflow.