The red flags in an AI vendor show up in three places: the demo, the contract, and the bill. Here is what to look for in each, from someone who picks AI vendors for a living and has to answer for them.
What are the red flags in an AI vendor?
By where they show up:
- In the demo. It runs on their sample data, not yours. It never shows a wrong answer. Nobody can say what it does when it is unsure.
- In the contract. They can use your data to train their models. There is no clause about where the data is stored or who can see it. You cannot export your data in a format you can use elsewhere. Auto-renewal for a year or more.
- In the price. Billed by usage with no cap and no alert. A low first-year price with the real price in the renewal. Required add-ons that only show up on the order form.
- In the company. The tool is a thin layer on someone else's model, and the only thing they own is the login screen. The roadmap slide promises the feature you need "next quarter."
How do I test the demo?
Bring your own data. A handful of real examples from your business, including a couple of ugly ones. Ask them to run those live. Then ask what happens when it gets one wrong: who sees the mistake, and how does it get fixed?
A vendor who will not run your examples is selling you their best day.
What should be in the contract?
At minimum:
- Your data is not used to train their models without your written permission.
- You can export everything you put in, in a common format, at any time.
- Where the data is stored and who at the vendor can see it.
- What happens to your data when you cancel.
- A price cap, or an alert before usage passes a number you set.
If they will not put those in writing, they will not honor them either.
Is it a red flag if the vendor uses ChatGPT or Claude underneath?
No. Most AI tools run on one of a few big models. The question is what they add on top. If the answer is your data, your workflow, and support when it breaks, that can be worth paying for. If the answer is a nicer screen, you may be paying a markup on something you already have.
Who you can trust with AI advice covers the same question for consultants instead of vendors.
What should I not do?
Do not sign at the end of the demo. Do not let one excited department buy a tool with company data in it before someone reads the contract. And do not pick a vendor without a written list of what you need. Turning a strategy into vendor RFPs shows how to write one.
Reading AI vendor contracts and running the selection is part of a fractional seat. The fractional executive page has the levels, or bring one contract to a one-hour second opinion for $150. Start on the start form.
Answered by Jared Lukes, Minneapolis. Twenty-five years in brand, UX, and technology, currently holding fractional AI and technology seats at three companies. About Jared.