You have had the calls, wondering whether an AI automation trial project would separate the builders from the presenters. Everyone demos well. Everyone shows you a workflow diagram with your logo on it and a number at the bottom that assumes nothing goes wrong. And you still have no idea which of these people can actually build something that survives contact with your business.
The answer is one narrow piece of real work, done for real money, in a fixed window, with a result you can measure. Not a discovery call. Not a proof of concept that runs on sample data and dies in a folder. A small live build that tells you what a partner is like to work with before you hand them anything that matters.
This post covers how to scope one, what to pay for it, what to watch during it, and how to read the outcome honestly.
Key Takeaways
- An AI automation trial project is a small paid build on real data with a fixed deadline, not a free pilot or a demo running on sample records.
- Scope one task that already runs in your business today, has a number attached, and can be finished inside three weeks.
- Pay for the trial. Free work gets junior staff, no deadline pressure, and a vendor who is selling rather than building.
- The trial tests the partner, not the technology. Communication, honesty about limits, and handover quality matter more than the automation itself.
- Write the pass mark before the work starts. “It seems to be working” is not a result you can act on.
- Ask who owns the accounts, the data, and the workflow files at the end. If the answer is the vendor, that is a finding.
- A good trial ends with a working automation and a clear picture of the next three things worth building, in that order.
Why the Trial Project Beats the Discovery Call
I have sat on both sides of this. When I am the one being evaluated, I can make a sixty-minute call go extremely well. I know your industry; I have seen your problem forty times; I can describe the solution fluently. None of that is evidence. It is pattern matching plus a decent speaking voice.
The MIT NANDA research on enterprise AI found that about 95% of AI pilots deliver no measurable return. The projects that worked started with a specific process, not with a technology choice. That is the whole finding, and it is also the argument for a trial. You are not testing whether AI works. You are testing whether this particular partner can point it at one of your processes and get a result.
The discovery call tells you how someone sells. The trial tells you how they build, how they communicate when something breaks at 4 pm on a Thursday, and whether they tell you the truth when a piece of the plan turns out to be harder than they quoted.
What a trial is not
It is not a free pilot. Free work is worth what you pay for it, and I will come back to why.
It is not a proof of concept on dummy data. Your business is not dummy data. Half of what makes automation hard is that your contact records are messy, your team named the pipeline stages something odd in 2021, and one person still does a critical step in a spreadsheet nobody mentions in meetings. A trial on clean sample data skips all of that, which means it skips the actual job.
It is not a strategy document. Plenty of consultants will sell you a roadmap. A roadmap is not a build. If the trial produces a PDF instead of a working thing, you learned nothing about their delivery.
How to Scope an AI Automation Trial Project
The single biggest failure mode is scoping too big. Founders pick their most painful problem, which is usually also the most tangled one, and then everyone spends three weeks arguing about edge cases. Pick something smaller than feels satisfying.
Four rules for choosing the task.
It already happens. Automate something the business does today, not something you have been meaning to start. If nobody currently follows up dormant leads, automating dormant-lead follow-up is a new process and a new automation at once. Two variables, no clean read.
It has a number. Response time in minutes. Calls answered versus missed. Quotes sent per week. Hours someone spends on it. If you cannot put a number on the task before the build, you will not be able to say whether the build worked.
It runs often. Daily or weekly, not quarterly. A quarterly process gives you one data point inside the trial window, which is not a result; it is an anecdote.
It finishes in three weeks. Two is better. Long enough to hit real problems, short enough that momentum survives.
Tasks that make good trials
Lead response is the usual first pick because the infrastructure is already there. You are getting enquiries now, and something happens to them now, and both of those things are measurable. Speed matters here more than most founders realise: research on lead response consistently shows the first business to reply wins the large majority of the deals, and most businesses take hours. A trial that takes first response from four hours to under two minutes gives you a number on day one.
Missed calls is another. Count what the phone does for a week before the build. Count it again after. Dr Claire ran a dental practice with two receptionists and was still missing close to half her inbound calls, which is a systems problem wearing a staffing costume. After the voice build, missed calls went to zero and booked appointments rose 44%. That was not a trial project; it was a full build, but the shape of the measurement is exactly what a trial should copy: count the thing, change the thing, count it again.
Quote follow-up, appointment reminders, and inbound enquiry routing all work for the same reason. Small surface area, obvious metric, high frequency.
Tasks that make bad trials
Anything that needs six people to agree on the process before you can automate it. Anything that touches your finance system on the write side. Anything where the current process is undocumented and contested, because you will spend the trial doing process design and calling it automation. And anything described to you as “our whole operation, but AI.” That is not a trial; that is the actual project with a smaller invoice attached.
Pay for It, and Here Is Why
Founders ask me for free pilots. I understand the logic. You have been burned, you want proof before money changes hands, and a vendor confident in their work should be happy to prove it.
The logic is wrong, and it is wrong in a way that costs you.
Free work goes to the bottom of the queue. Not out of malice. Every agency and every partner has paying clients with deadlines, and when Thursday gets tight, the unpaid trial is what slips. You will get the junior, you will get the leftovers of someone’s week, and you will get a result that tells you nothing about how they perform when they are properly engaged.
Free work also changes the relationship. An unpaid trial is a sales activity, so it gets staffed by the person who is good at selling. A paid trial is a delivery activity, so it gets staffed by the person who is good at delivering. Those are frequently not the same person, and the second one is the one you are actually trying to evaluate.
And the price itself is information. A partner who cannot put a number on a two-week build has not done many two-week builds. A partner who wants a five-figure commitment before touching anything is telling you their model does not do small. Neither is disqualifying. Both are worth knowing.
What a trial should cost
Low four figures is the range where this works for most small builds, roughly the cost of a week of a decent contractor. Enough that it goes on a real project plan with a real person assigned. Small enough that if the partner turns out to be wrong for you, the lesson costs less than one bad month of a twelve-month engagement.
Ask whether the trial fee credits against the full build if you proceed. Most partners worth hiring will say yes, because they would rather have the engagement than the trial fee. If they say no, ask why. The answer will tell you something.
Ask about payment timing too. Half up front and half on the agreed result is normal and fair. All up front on a first engagement with someone you have never worked with is worth a conversation, and I have written more about that in AI automation payment terms.
What to Measure While It Runs
Write the pass mark before anyone starts building. Not after. Once you have seen the output, your brain will helpfully redefine success to match whatever you got.
A pass mark is one sentence with a number in it. “Every web enquiry gets a first response within three minutes, seven days a week, for two consecutive weeks.” “Zero unanswered calls between 8 am and 6 pm across a fortnight.” “The Tuesday report arrives at 7 am without anyone touching it, four weeks running.”
Then measure three things, and only one of them is about the automation.
Did it hit the number? Binary. It did, or it did not. Partial credit is how eighteen-month engagements start on a foundation nobody actually checked.
How they behaved when it broke. It will break. Something will hit an edge case, an API will change, a phone number will be formatted in a way nobody anticipated. That is not the test. The test is what happened next. Did you find out from them or from a customer? Did they explain what went wrong in language you could follow? Did they fix the cause or patch the symptom? This is the most valuable signal in the whole exercise, and it only shows up because you insisted on real data.
What they did with the parts they could not do. Every trial hits something that is harder than expected or genuinely out of reach. A good partner tells you early, explains the constraint, and offers the nearest workable alternative. A bad one goes quiet, then delivers something adjacent to what you asked for and hopes you have forgotten the brief. You want the person who says “that one is not worth automating and here is why” in week one, not the person who says yes to everything and delivers 60% of it.
The questions to ask at handover
Who owns the accounts? If the automation runs inside a platform, whose login is the master, and what happens to it if you part ways? Vendor-owned accounts are a lock-in mechanism whether or not anyone frames it that way.
Where do the workflow files live? Can you export them? Can another developer read them, or is the logic buried somewhere only that partner can reach?
What is the running cost, separate from their fee? Tool subscriptions, API usage, phone numbers. You want the full picture, not the labour line.
What breaks it. A partner who has actually built the thing can list three failure modes without pausing. One who can has not thought about it, which means you will discover them yourself, live, in front of a customer.
I have put a longer version of this list in the AI automation contract checklist, which is worth reading before you sign anything larger.
Reading the Result Without Fooling Yourself
The trial finishes. Now the hard part, which is being honest about what you saw.
There are three outcomes, and only one of them is ambiguous.
Clear pass. The number was hit, the communication was good, the handover was clean, and you understand what you now own. Move forward. The natural next step is not “do everything”; it is “do the next two things on the list they gave you,” which is the point of a trial that ends with a prioritised list rather than a sales pitch.
Clear fail. The number was missed, or it was hit but the process to get there was miserable. Missing the number is not automatically disqualifying if the reason was something genuinely unforeseeable and they handled it well. Missing the number and going quiet about it is disqualifying. Walk. You spent low four figures instead of finding out ten months and forty thousand dollars into a build.
The ambiguous middle. This is where most trials land and where most founders make the mistake. It sort of works. It hit the number some days. The communication was fine, not great. There is a temptation to proceed because you have invested effort in the relationship and starting over feels expensive.
Resist it. Ask one question: if the next build is five times the size, does this partner get five times better, or do these problems get five times worse? Small-scale friction does not resolve at scale. It compounds. The vendor who was slightly slow to respond on a two-week trial will be genuinely unreachable on a six-month one.
The thing the trial actually reveals
Here is what surprises people. The trial rarely tells you much about the technology. The tools are good now and mostly work. What the trial tells you is whether this partner understands your business well enough to know what is worth automating in the first place.
The best signal is what they push back on. If they took your brief, built exactly what you asked for, and never once said “the thing you actually want is slightly different from what you described,” they are order-takers. Order-takers are fine for well-specified work and dangerous for anything else, because you will get precisely what you asked for and you were not entirely right.
The partner you want is the one who, three days in, comes back and says the bottleneck is not where you thought it was. That is a person who has looked at the actual work rather than the brief about the work. It is also, quietly, the whole difference between buying automation and getting an AI partner.
Where a Trial Fits in the Bigger Picture
One automated task is not a transformation. I want to be direct about that, because the trial can create a false sense of completion.
What you have after a good trial is one process that runs without you and a partner you have tested under real conditions. That is genuinely valuable, and it is also about 2% of the available ground. The reason most businesses stall after their first automation is that they treated it as a finished project rather than the first module of something larger.
The pattern that actually compounds looks like this. The business gets an AI brain first, meaning the context, the numbers, and the decisions live somewhere the system can read rather than only inside the founder’s head. Then it gets an AI workforce, meaning specific tasks handled end to end. Each automated task is easier than the last, because the context is already there. The first one takes three weeks. The fifth takes three days.
Without that foundation, every automation is a standalone build starting from zero, and you end up with six disconnected tools nobody has the appetite to maintain. That is the state most founders are already in when they call me, and adding a seventh does not help.
So the trial is not the destination. It is how you find out whether the person you are about to build the foundation with is any good. There is more on what a longer engagement should look like in the 90-day AI automation checkpoint, and on how to run the earlier conversation in the AI automation discovery call checklist.
Conclusion
The reason founders get burned on AI automation is almost never that they picked the wrong technology. It is that they committed to a large build with someone they had only ever seen present. Presenting and building are different skills, and the gap between them is where the money goes.
An AI automation trial project closes that gap for the cost of about a week’s work. One real task, on real data, with a number attached and a deadline that is close enough to create pressure. You find out how they build, how they communicate under strain, whether they tell you the truth about limits, and whether they understand your business well enough to disagree with your brief when your brief is wrong.
And if it goes well, you have not just bought a working automation. You have found the person you can build the rest with, which is the thing you were actually shopping for.
Start smaller than feels satisfying. Pay for it. Write the pass mark down first.
Ready to Scope One?
If you want a second opinion on what your trial project should be, book a 30-minute Discovery Call. No pitch, no deck. We look at what is actually eating your week, pick the one task worth testing first, and I will tell you straight if the honest answer is that you do not need a partner for it yet.
Book a 30-minute Discovery Call
Frequently Asked Questions
How long should an AI automation trial project take?
Two to three weeks is the right window. Long enough for real problems to surface in real data, short enough that the momentum holds and nobody loses interest. Anything under a week usually means the scope was too small to test anything meaningful. Anything past a month has stopped being a trial and become a project you have not properly contracted for.
Should I pay for a trial project or ask for a free pilot?
Pay for it. Free work slips down the queue behind paying clients, gets staffed by whoever is available rather than whoever is best, and turns the exercise into a sales activity instead of a delivery one. Paying also gets you a real project plan and a real deadline. Ask whether the fee credits against a full build if you proceed, because most partners will agree to that.
What is a reasonable price for an AI automation trial?
Low four figures for a small build is the usual range, roughly what a week of a good contractor costs. The exact number matters less than what it signals. A partner who cannot price a two-week build has not done many. A partner who insists on a five-figure minimum before touching anything is telling you their model does not accommodate small first steps.
What should I automate first as a test?
Pick something that already happens in your business today, runs at least weekly, and has a number attached to it. Lead response, missed calls, and quote follow-up are the common picks because the measurement is obvious and the infrastructure already exists. Avoid anything that requires several people to agree on the process first, because you will spend the trial doing process design.
How do I know if the trial actually succeeded?
Write the pass mark as one sentence with a number in it before any building starts, then check it honestly at the end. u0022Every enquiry gets a first response within three minutes for two consecutive weeksu0022 is a pass mark. u0022It seems to be working wellu0022 is not. Judge the communication and the handover separately from the automation itself.
What questions should I ask at the end of a trial?
Ask who owns the platform accounts, where the workflow files live and whether you can export them, what the ongoing running cost is separate from the partner’s fee, and what three things are most likely to break it. A partner who can list the failure modes without pausing has genuinely built the thing. One who can has not thought it through.
Can a trial project tell me if AI automation is right for my business?
Partly. It tells you whether one specific process can be automated and whether this specific partner can do it well. It does not tell you whether a wider programme makes sense, because a single automated task in isolation rarely changes how the business runs. Treat the trial as a test of the partner and the process, then decide on scale separately.
About Octavius
Titus Mulquiney is the founder of Octavius AI, where he builds AI brains and AI workforces for founder-led businesses stuck running everything out of their own head. Twenty years in marketing, ex-Sony product manager, ex-GM Zeal NZ. Based in Auckland, working with operators across NZ, Australia, and the US. Connect on LinkedIn.