Buying AI Was the Easy Part.You didn’t consider how you’ll get your money’s worth.
You approved the AI. Your team uses it every day. But is it actually doing work, or did you just buy another subscription? Here are the eight things to check before you renew.
You did the hard part on paper. You picked a tool, approved the budget, told the team to use it. For most companies that is where it ends. And a tool nobody set up right does not save anyone time. It just lands on the invoice every month.
I call the part you skipped the harness. It is everything around the AI that turns it into work that actually gets done. Who owns it. Who checks it. Whether anyone can prove it is working. The tool is the easy part. The harness is the job.
What follows is the harness, broken into eight checks. You do not need to be technical to run them. You just need to run them before you sign the next renewal.
A license isn’t a strategy.
The AI itself is a commodity now. Everyone buys the same one. Your competitor can sign your exact contract tomorrow morning. So the tool was never going to be your edge. The harness around it is.
Watch where the real money goes. In June, Salesforce spent $3.6 billion to buy an AI customer-service company. They did not pay for a smarter chatbot. They paid for the system around it, the one that closes three out of four customer questions with no human involved. A week later they changed how they charge for it: $2 when the AI actually resolves a case, nothing when it doesn’t. That is the whole shift. The market stopped paying for access to AI and started paying for results. Your budget should work the same way.
The harness, in eight checks.
Run these on every AI tool you pay for. Each one is a question you can ask your team out loud, today.
Who owns this AI, by name?
Every tool you pay for needs one person who owns whether it works. Not the team. A name. If you can’t say it out loud, nobody owns it. And a tool nobody owns is a tool nobody is making better.
“Who owns this tool, by name, and who do we call if it breaks on a Friday?”
What stops it from doing something that costs you a customer?
Your AI is writing emails, booking appointments, answering the phone, updating records. Before it does something that embarrasses you in front of a customer, something should catch it. Most setups have nothing. They just hope it behaves.
“What’s the check before this touches a customer or a dollar?”
Can you prove what it did, and whether it worked?
Say the AI handled a hundred conversations last week. Can your team pull up which ones went fine and which went sideways? If it’s a black box, you can’t improve it, and you can’t defend it when a partner or an auditor asks.
“Show me last week. What did it do, and did it actually work?”
What work did it actually take off someone’s plate?
This is the only number that matters, and almost nobody can give it to you. Not “we use AI.” What percent of a real job now happens without a person touching it? The best support AIs handle three of four contacts alone. If your vendor charges per seat or per message no matter what happens, you’re paying for their business, not yours.
“What share of this job does it do without us, and are we paying for results or just for access?”
Are you paying for twelve tools doing the work of three?
AI stacks bloat fast. Every team wants one more. More tools almost never means more done. Usually it means more invoices and more confusion. One engineering team cut their setup from sixteen tools down to one and got faster, cheaper, better results. Cutting is usually the upgrade.
“What can we cut this quarter without losing anything that matters?”
Does this make you different, or just like everyone who bought the same thing?
Generic AI learns from the public internet, so it gives you the same answers your competitors get. Your edge is what only you have: your customers, your call recordings, your history, the way your shop actually runs. If none of that feeds your AI, you bought a commodity and you’re using it like one.
“What do we feed it that a competitor never could?”
Who checks the AI’s work, and can they keep up?
AI produces fast. If the person checking it can’t move at the same speed, mistakes go straight to customers and you find out from a complaint. The rare skill now isn’t making more. It’s catching what’s wrong as fast as it’s made. If your team can’t keep up, you have a hire to make.
“Who’s checking the output, and can they keep up with the volume?”
Can your team hand off a whole job, or are they still fiddling with prompts?
The teams pulling ahead stopped trading clever prompts. They hand the AI a whole job: the goal, the steps, what done looks like, what to do when it gets something wrong. Then they let it run. If your people are still swapping prompt tricks, they’re training for a year that’s already over.
“Can we hand off a whole task start to finish, or are we still babysitting it?”
When to run the check.
What the harness won’t do for you.
It won’t tell you which AI to buy, where to start, or what to pay. It tells you whether you built the thing that makes AI pay back. Most companies didn’t. That’s why most AI spend quietly disappears.
Credit where it’s due.
This framework stands on other people’s ideas and reporting.
- The idea of “the harness” — and that the real skill now is handing off a whole job, not writing a sharper prompt — comes from Nate B. Jones.
- Salesforce’s $3.6B acquisition of Fin (formerly Intercom), and Fin’s 76% autonomous resolution: Salesforce, TechCrunch.
- Agentforce Help Agent and the $2 pay-per-resolution pricing: Salesforce, SalesforceDevops.
- The “fewer tools, better agent” result: Vercel’s engineering team, “We removed 80% of our agent’s tools.”
- On verification becoming the scarce skill: Fiona Fung, Anthropic — via Lenny’s Newsletter.
Figures are reported by the sources above; specifics (e.g., exact tool counts) are summarized for a general audience.
Find out where your AI spend is actually paying off.
The Aventary AI Diagnostic is short and fixed-scope. I run the harness check across what you’ve already bought, show you where the money is working and where it’s leaking, and hand you a plan you can act on. You get it in weeks.
Book Your AI Diagnostic