The red flags in an AI vendor show up in the demo, the contract, and the bill. I pick AI vendors for clients and have to answer for them, and these are what I look for.
What are the red flags in an AI vendor?
By where they show up:
- In the demo. It runs on their sample data, not yours. It never shows a wrong answer. Nobody can say what it does when it is unsure.
- In the contract. They can use your data to train their models. Nothing says where the data is stored or who can see it. You cannot export your data in a format you can use elsewhere. It auto-renews for a year or more.
- In the price. Billed by usage with no cap and no alert. A low first-year price with the real price in the renewal. Required add-ons that first appear on the order form.
- In the company. The tool is a thin layer on someone else's model, and all they own is the login screen. The roadmap slide promises the feature you need "next quarter."
How do I test the demo?
Bring your own data: a handful of real examples from your business, including a couple of ugly ones. Ask them to run those live. Then ask what happens when it gets one wrong. Who sees the mistake, and how does it get fixed?
A vendor who will not run your examples is selling you their best day.
What should be in the contract?
At minimum:
- Your data is not used to train their models without your written permission.
- You can export everything you put in, in a common format, at any time.
- Where the data is stored and who at the vendor can see it.
- What happens to your data when you cancel.
- A price cap, or an alert before usage passes a number you set.
If they will not put those in writing, do not expect them to honor them.
Is it a red flag if the vendor uses ChatGPT or Claude underneath?
No. Most AI tools run on one of a few big models. Ask what they add on top. If it is your data, your workflow, and support when it breaks, that can be worth paying for. If it is a nicer screen, you may be paying a markup on something you already have.
Who you can trust with AI advice covers the same question for consultants.
What should I not do?
Do not sign at the end of the demo. Do not let one excited department buy a tool with company data in it before someone reads the contract. And do not pick a vendor without a written list of what you need. Turning a strategy into vendor RFPs shows how to write one.
Reading AI vendor contracts and running the selection is part of a fractional seat. The fractional Chief AI Officer page has the levels, or bring one contract to a one-hour second opinion for $150. Send it through the start form.
Answered by Jared Lukes, Minneapolis. Twenty-five years in brand, UX, and technology, currently holding fractional AI and technology seats at three companies. About Jared.