A lawyer in Auckland asked me last month whether the on-premises vs cloud AI system answering her phones was sending client files to America. Fair question. She had no idea, and the vendor she asked before me could not tell her either. If you run a medical practice or a law firm here, that decision is not an IT preference. It is the difference between a system you can defend to the Privacy Commissioner and one you cannot.
This post covers what the two options actually mean, what New Zealand law requires of you, where the real trade-offs sit once you look past the marketing, and the specific questions to put to any vendor before you sign anything.
Key Takeaways
- The on-premises vs cloud AI question in healthcare and legal comes down to one thing: who can read the data, and under what contract.
- New Zealand’s Privacy Act 2020 does not ban offshore AI. It makes you responsible for comparable protection wherever that information ends up.
- A server in your own building is not automatically safer. Unpatched hardware in a locked cupboard fails audits that cloud providers pass.
- Most practices settle on a tiered setup: cloud AI for front-of-house work, tighter controls or local processing for clinical notes and matter files.
- Ask every vendor three things: do you train on my data, how long do you keep it, and which sub-processors touch it.
- Zero-retention API access plus a no-training contract closes most of the gap with an on-premises build, for a fraction of the cost.
- Running local models means you own patching, backups, uptime and GPU capacity. That is a staffing decision as much as a technical one.
On-Premises vs Cloud AI: What the Two Options Actually Mean
On-premises AI means the model runs on hardware you control. A machine in your server room, or a private box you rent in a New Zealand data centre. The model weights sit on your disk. The prompts and the responses never leave your network. In practice, this means open-weight models such as Llama, Qwen or Mistral, running on a GPU server you bought or lease.
Cloud AI means you send the text to someone else’s servers over an API; they run the model, and they send the answer back. Every major model people actually want to use sits here: Claude, GPT, Gemini. The data crosses the wire, gets processed offshore in most cases, and comes back.
There is a third option that gets skipped in nearly every conversation I have, and it is the one most practices end up choosing. Call it contracted cloud. You use a frontier model through a business or enterprise agreement that excludes your data from training, sets retention to zero or near zero, names every sub-processor, and pins the processing region. Some vendors will commit to Australian or Singaporean regions. Microsoft, Google and Amazon all offer the major models through their own cloud regions with those controls attached.
The reason this matters: the security question is rarely about geography. It is about who has the legal right to read, retain and reuse the information. A file sitting in Sydney under a no-training, zero-retention contract with audit logging is often better protected than the same file sitting on a Windows box in your storeroom that has not been patched since installation.
What New Zealand Law Actually Requires of Health and Legal Practices
I am not a lawyer and this is not legal advice. Take the specifics to your own advisor. But you should know the shape of the obligations before a vendor tells you what they are.
Under the Privacy Act 2020, information privacy principle 11 governs when you may disclose personal information, and principle 12 governs sending it outside New Zealand. IPP 12 does not forbid offshore processing. It requires you to believe on reasonable grounds that the recipient is subject to comparable safeguards, usually through contractual terms or because they operate under a privacy regime the law recognises. The full set of principles is published by the Office of the Privacy Commissioner.
Health practices carry more. The Health Information Privacy Code 2020 replaces the general principles with health-specific rules covering collection, storage, retention and disclosure of health information. Rule 5 puts the security obligation squarely on you as the health agency, regardless of who you outsourced the processing to. You cannot contract your way out of accountability. You can only contract your way into evidence that you took reasonable steps.
Law firms sit under the confidentiality duty in the Lawyers and Conveyancers Act rules, plus the Privacy Act on top. Confidentiality is broader than privacy. It covers commercially sensitive information about a client that is not personal information at all. Feeding a draft agreement into a consumer chatbot that reserves the right to train on your conversations is a confidentiality problem before it is ever a privacy one.
The practical takeaway is that the law asks a question about reasonable steps and comparable protection. It does not ask where the server is. That gives you more options than most vendors admit, and less cover than most practices assume.
On-Premises vs Cloud AI in Healthcare and Legal: The Honest Trade-Offs
Here is what I tell practice owners when they ask me to settle the argument.
The case for on-premises is real but narrower than it sounds
If you handle information where any offshore disclosure is genuinely unacceptable- mental health records, protected witness material, matters involving government security classifications- then local processing is the answer, and the cost is justified. Same if a professional body or an insurer has told you in writing that offshore processing is off the table.
The cost is not the hardware
A GPU server capable of running a decent open-weight model is a capital line item, not a catastrophe. The cost that surprises people is everything around it. Someone has to patch it, back it up, monitor it, rotate keys, restore it when a disk fails, and upgrade the model when a better one is released three months later. In a fifteen-person practice, that person is either an outside contractor on a call-out rate or the office manager who already has a full job.
The model quality gap is closing but has not closed
Open-weight models are genuinely good now. They are not equal to the frontier models on long documents, careful reasoning, or following a complicated set of instructions without drifting. For summarising a call, they are fine. For reviewing a lease against a checklist, the gap still shows.
On-premises does not remove the audit obligation
Whichever way you go, you need a record of what the system did, what it was asked, and what it produced. That obligation does not disappear because the box is in your building. If anything it gets harder, because the logging is now yours to build. I wrote about why that record matters more than the model choice in the case for an AI audit trail in your business.
Cloud gets better while your server ages
The uncomfortable truth is that a cloud provider ships security improvements, model upgrades and compliance certifications continuously. Your on-premises install is at its best on day one and decays from there unless someone owns it.
The Questions to Ask Any AI Vendor Before You Sign
Print this list. Ask it verbatim. A vendor who cannot answer these in writing does not understand their own supply chain, which is your answer.
- Do you or any sub-processor use my data to train, fine-tune or evaluate models? Get this in the contract, not in an email.
- How long is my data retained, and where? Zero retention is available from the major providers in business agreements. Ask for it.
- Which region processes the request, and which region stores it? These can be different.
- Name every sub-processor. If a vendor is built on top of a model provider, a hosting provider and a telephony provider, that is three more companies touching the data.
- Is the data encrypted in transit and at rest, and who holds the keys?
- What audit logs do I get, how long are they kept, and can I export them?
- What is your breach notification commitment, in hours, and who does the notifying?
- Can I delete a client’s data on request, including from logs and backups, within a defined window?
- Where does a human review the output, and who are those humans?
- If I leave, what happens to my data and how do I get it back?
The tenth one filters more vendors than the other nine combined.
Two more things worth knowing. Consumer chatbot plans and business plans behave differently on training by default, so a staff member using their personal account is a different risk from the same model accessed through your contracted API. And any AI that touches your phone system, your booking calendar or your practice management software is holding health information, whether or not anyone calls it clinical.
The Pattern Most Practices Actually Land On
After enough of these conversations, the same shape keeps appearing, and it is not all-or-nothing.
Sort your information into tiers. Front-of-house sits in tier one: who called, when, what they wanted, when they can come in. Tier two is operational: quotes, invoices, staff rosters, referral tracking. Tier three is the clinical file or the matter file.
Tier one and tier two run perfectly well on contracted cloud AI with the controls above. This is where the return sits anyway, because this is where the phone goes unanswered and the follow-up never happens. One dental practice I worked with had two receptionists and was still missing close to half its inbound calls at peak. Once an AI receptionist picked up the overflow and the after-hours calls, missed calls went to zero and booked appointments rose 44%. Nothing clinical ever went near the system. It took a name, a reason for calling and a preferred time, then wrote to the booking calendar. If missed calls are your bleed too, I covered the mechanics in after-hours call answering and AI appointment booking.
Tier three gets treated differently. Either it stays inside your practice management system with an AI that has been contracted to zero retention and no training, or it runs against a local model, or a human de-identifies before anything is sent. Which of those three you pick depends on your professional obligations and your appetite, not on a general rule.
The mistake I see most often is a practice freezing the whole decision because tier three is hard. Eighteen months pass, the phones still go unanswered, and nothing has improved. Start where the information is least sensitive, and the payoff is clearest, then work inward as your controls mature. That sequencing logic is the same one I use with any business getting started, which I set out in AI adoption for SMEs.
Where This Leaves You
The on-premises vs cloud AI question is worth asking. It is just rarely the first question. The first question is which information you are actually putting through the system, and what the contract says about who may read it, keep it, and reuse it. Answer that honestly and the hosting decision usually answers itself, because most of the work you want automated does not touch the sensitive tier at all.
The bigger picture is that a health or legal practice is not short of AI tools. It is short of a system that knows how the practice runs and does the repetitive work without a person driving it. An AI brain that holds your context and an AI workforce that acts on it. Data handling is the first design decision in that build, not an afterthought bolted on once something breaks.
If you want a straight read on which parts of your practice can be automated safely, and which parts should stay exactly where they are, book a 30-minute discovery call. No pitch deck. Bring your list of concerns, and I will tell you where the line sits.
Frequently Asked Questions
Is it legal to use cloud AI with patient data in New Zealand?
Yes, with conditions. The Privacy Act 2020 and the Health Information Privacy Code 2020 do not prohibit offshore processing. They require you to take reasonable security steps and, under IPP 12, to have reasonable grounds to believe the overseas recipient protects the information comparably. That belief usually rests on contractual terms. Check the specifics with your own privacy advisor before you rely on a vendor’s assurance.
Does my data leave New Zealand when I use ChatGPT or Claude?
In most default configurations, yes. The major models are processed in overseas data centres. Business and enterprise agreements let you pin a processing region, and some providers offer Australian or Singaporean regions through Microsoft, Google or Amazon cloud platforms. Consumer accounts give you far less control, which is why a staff member using a personal login carries a different risk from your contracted access.
Is on-premises AI more secure than cloud AI?
Not automatically. On-premises removes the offshore disclosure question, which matters. It also hands you every other security responsibility: patching, key rotation, backups, physical access, monitoring and incident response. A well-run cloud agreement with zero retention, no training and audit logging often beats an unmaintained local server. Security comes from who maintains the system, not from which building it sits in.
What is the cheapest compliant option for a small practice?
Usually a contracted cloud setup, not a local build. You pay per use rather than for hardware, and you avoid the ongoing maintenance load that sinks most on-premises projects in small practices. Spend the money instead on getting the contract right, restricting what information reaches the system, and keeping proper logs of what it did.
Can I use AI on legal documents without breaching client confidentiality?
You can, if the terms allow it. Confidentiality is broader than privacy, so a vendor reserving the right to train on your conversations is a problem even when no personal information is involved. Use an account contracted for zero retention and no training, keep an access log, and check whether your client engagement terms already cover the use of third-party processing.
What should an AI vendor contract include for a health or legal practice?
At minimum: no training on your data, a defined retention period, a named list of sub-processors, the processing and storage regions, encryption in transit and at rest, exportable audit logs, a breach notification window in hours, a deletion process that covers logs and backups, and clear terms for getting your data back if you leave.
Do I need patient consent to use an AI receptionist?
Usually you need transparency more than fresh consent, because you are already collecting that information to book appointments. Tell callers they are speaking with an AI assistant, disclose it in your privacy statement, and record what the system collects. Keep the AI to scheduling and general enquiries rather than clinical detail, and the obligations stay light.
About Octavius
Titus Mulquiney is the founder of Octavius AI, where he builds AI brains and AI workforces for founder-led businesses stuck running everything out of their own head. Twenty years in marketing, ex-Sony product manager, ex-GM Zeal NZ. Based in Auckland, working with operators across NZ, Australia, and the US. Connect on LinkedIn.