Your Staff Are Already Using AI. The Question Is Whether Client Data Is Going With It.

Somebody at your firm pasted a client document into an AI chatbot this week. Probably several people, probably more than once, and almost certainly without mentioning it.

They were not being reckless. They had a forty-page agreement to summarize before a four o'clock call, a tool that does it in nine seconds, and no policy telling them otherwise. From where they sit, that is good judgment about time.

The question is not whether this is happening at your firm. It is whether you know what leaves the building when it does.

Why Banning It Does Not Work

The instinctive response is a prohibition. It fails for a straightforward reason: the productivity gain is real and large, the tools are reachable from any personal phone, and enforcement is essentially impossible.

What a ban reliably produces is not less AI use. It is less visible AI use. Staff stop asking, stop mentioning it in meetings, and shift to personal accounts on personal devices, where you have no logging, no policy, and no idea what was shared. You have traded a manageable risk for an invisible one.

The firms handling this well are not the ones with the strictest policy. They are the ones that gave people something equally useful that runs somewhere safe.

What Actually Happens to a Document You Paste In

The honest answer is that it varies enormously by provider and plan tier, which is precisely the problem.

Consumer and free tiers frequently reserve the right to use submitted content to improve their models, and typically retain conversation history by default. Business and enterprise tiers generally contract not to train on customer data and offer shorter retention, administrative controls, and sometimes a signed data processing agreement.

The gap between those two situations is significant, and almost nobody at a small firm knows which one their staff are using. An employee who signed up personally in 2024 is probably on consumer terms. The terms have also changed several times since.

For a business with no confidentiality obligation, this is a manageable ambiguity. For one holding privileged material, client financials, or protected health information, "we are not certain what the terms were" is not a position you want to explain to a client, a board, or a regulator.

The Obligation Question Comes Before the Technology Question

Professional firms do not get to treat this purely as an IT decision, because the duty is not primarily about data security. It is about confidentiality.

Attorneys have duties of confidentiality and competence that extend to the technology they use, and bar associations across the country have issued guidance on generative AI addressing client confidentiality, supervision of the work product, and in some circumstances client disclosure. Accountants handling client financial records, medical and dental practices under HIPAA, and any firm operating under client non-disclosure agreements sit in comparable positions.

The specifics differ by profession and jurisdiction, and the applicable guidance is worth reading directly rather than taking secondhand. But the shared question is the same one: can you state, clearly and accurately, where client material goes and who can access it? For most firms today the honest answer is no, and that is the actual exposure.

What Private AI Means in Practice

Private deployment means the model runs on hardware you control, indexing documents stored on your own systems, answering questions locally. Nothing is transmitted to an external AI provider. If your requirements demand it, the system can operate with no outbound internet access at all.

This is more achievable than it sounds. Open-weight models have improved substantially, and the tasks professional firms actually want are well within their reach:

• Searching your own documents. Ask a question in plain language and get an answer drawn from your files, with citations pointing back to the source. For most firms this is the highest-value starting point by a wide margin, because it solves the problem of institutional knowledge nobody can find.
• Summarizing long material. Contracts, deposition transcripts, policy documents, correspondence threads.
• Drafting from your own templates. An assistant that has read your precedents produces something closer to your house style than a generic tool ever will.
• Answering questions about internal procedure. Onboarding a new hire who can query five years of accumulated practice on their first day.

Where frontier cloud models still hold a clear advantage is the hardest novel reasoning. That is worth knowing, and it is worth being honest that most day-to-day firm work does not need it.

The Hardware Is Smaller Than People Expect

The common assumption is a server room and a six-figure budget. For a small firm's document search and question-answering workload, a single workstation-class GPU is usually sufficient. Heavier concurrent use across a larger team pushes toward a dedicated GPU server, but that is a step taken after the value is proven, not before.

The economics also differ from subscriptions in a way worth noting. Per-seat AI licensing is a permanent operating cost that scales with headcount. A machine is a capital purchase you own, depreciate, and can repurpose for other internal workloads when your needs change. For a firm with steady, predictable usage, ownership often becomes the cheaper option within two to three years.

A Realistic Path Forward

The firms that get this right tend to move in roughly this order:

• Find out what is already happening. Ask without penalty. You cannot write a sensible policy for behavior you have not measured, and staff will tell you if the question is not a trap.
• Write the policy before buying anything. Even one page. What may be used, what must never leave, which tools are approved, and who to ask. Most of the immediate risk is closed by clarity, not by hardware.
• Assess feasibility honestly. What documents exist, in what formats, what questions people actually want answered, and what it would take to serve the number of people who would use it.
• Start with document search. It is the narrowest useful deployment, delivers value quickly, and teaches you what your firm actually wants before you commit to more.
• Expand only where it earns its place. A small system in daily use beats an ambitious one nobody adopted.

When You Should Not Do This

Private AI is oversold, and a fair amount of what gets pitched does not justify the hardware.

If your document volume is small, your material is not confidential, and your team is three people, a commercial business-tier subscription with proper terms is very likely the better and cheaper answer. The case for a private deployment rests on having both a genuine confidentiality obligation and enough document volume that searching it is a real problem. Without both, you are buying infrastructure to solve something a policy would have handled.

Anyone recommending a GPU server before asking what is in your file room is selling hardware, not solving a problem.

Where to Start

The immediate risk at most firms is not the absence of a private AI system. It is the absence of an answer to a simple question: if a client asked today whether their documents have been processed by an outside AI service, could you answer with confidence?

Archway Point helps law firms, accounting practices, insurance agencies, and professional businesses in St. Charles County and the greater St. Louis area work through exactly this. That starts with a written AI usage policy and a feasibility assessment, and only moves to private AI infrastructure and dedicated compute if the assessment says it is worth doing. Sometimes it says the opposite, and we would rather tell you that in week one than after you have bought a server. If your staff are already using these tools and nobody has written down the rules, start there.