Private LLM vs Public LLM: Data, Cost and Control Compared for Enterprise Buyers

Four places your company’s LLM can run

The question is usually framed as two options. In practice there are four, and most buying confusion comes from treating the middle two as if they were the ends.

1. Consumer public LLM. A personal ChatGPT, Gemini or Claude account, often paid on an employee’s card. On ChatGPT Free, Plus and Pro, the “Improve the model for everyone” setting is on by default. Claude Free, Pro and Max users choose whether chats may be used for training; if they allow it, Anthropic may keep that data for up to five years (both verified October 2026). The company has no admin console, no retention control and no record of what was shared.
2. Business or enterprise tier of a public LLM. ChatGPT Business and Enterprise, Claude Team and Enterprise, Gemini in Google Workspace or Gemini Enterprise, and Microsoft 365 Copilot. The model still runs on the vendor’s shared infrastructure, under a business contract: no training on business data by default, admin controls, single sign-on and, on higher tiers, retention settings and regional storage.
3. Dedicated cloud. A model served inside your own cloud account, through services such as Amazon Bedrock, or an open-weight model on cloud GPUs you rent. The provider operates the hardware; you control the account, network rules, logging and region. AWS says, for example, that model providers have no access to Bedrock logs or to customer prompts and completions.
4. Self-hosted or on-premises private LLM. An open-weight model on servers you own or lease, in your data centre, a private or sovereign cloud, or an air-gapped network. Nothing leaves unless you route it out, and you take on the hardware and operations.

Options 3 and 4 are both private in the sense most auditors care about: the customer controls the environment. For the components of a full platform, see our overview of what a private AI platform includes. This article is about choosing between the options.

Private LLM vs public LLM: the comparison buyers need

The table compares the business tier of a public LLM (option 2), since that is the realistic alternative for a company, with a self-hosted private LLM (option 4). A dedicated cloud deployment usually lands between the two columns.

What business tiers of public LLMs promise in October 2026

All four large vendors now separate business data from consumer data. The details differ, so read them in the vendor’s own words before a security review.

OpenAI. OpenAI says it does not train its models on business data by default, covering ChatGPT Business, Enterprise and Edu and the API platform. Qualifying organisations can configure how long OpenAI keeps business data, and API customers can apply for zero data retention. Enterprise includes data residency in a list of regions. Our ChatGPT seat-cost breakdown lists which controls sit on which plan.

Anthropic. Anthropic does not use inputs or outputs from its commercial products, including Claude for Work and the API, to train models by default. Feedback a user submits is kept for up to five years, and Team and Enterprise owners can turn feedback off. Custom retention is an Enterprise feature; the Claude plan comparison covers it alongside the usage billing.

Google. Google’s Workspace privacy hub states that Workspace does not use customer data to train models without the customer’s prior permission or instruction, and that Gemini app chats on work accounts are not reviewed by human reviewers. Admins set how long Gemini app conversations are kept, up to 36 months. Gemini Enterprise, sold through Google Cloud, adds residency and customer-managed keys on its Standard and Plus editions, as set out in our Gemini edition comparison.

Microsoft. Microsoft’s Copilot privacy documentation (updated July 2026) says prompts, responses and data accessed through Microsoft Graph are not used to train foundation LLMs, and admins can set Copilot retention through Microsoft Purview. For EU customers, Copilot is an EU Data Boundary service. Two exceptions matter for regulated buyers: Anthropic models offered as a subprocessor inside Copilot are currently excluded from the EU Data Boundary, and customers outside the EU may have queries processed in the US, the EU or other regions. Licence costs are in our Copilot licence breakdown.

Retention promises sit below legal obligations

The clearest lesson of the last two years came from a lawsuit. In May 2025, in the copyright case The New York Times brought against OpenAI and Microsoft, a US magistrate judge ordered OpenAI to preserve ChatGPT output logs that would otherwise have been deleted. OpenAI said at the time that the order covered ChatGPT Free, Plus, Pro and Team, and API customers without a zero data retention agreement; ChatGPT Enterprise, Edu and zero-retention API traffic were outside it. A later order, filed on 9 October 2025, ended the broad obligation from 26 September 2025. Logs already preserved stayed preserved, and OpenAI still had to keep data linked to accounts the Times flagged.

Nothing in that episode was a breach. It shows that data held by a vendor is subject to the vendor’s legal obligations, in the vendor’s jurisdiction, and your deletion setting cannot override a court. If your risk assessment depends on deletion on a fixed schedule, only a private deployment puts that schedule under your own legal team. If it does not, an enterprise tier with clear retention terms is usually adequate.

Private LLM vs ChatGPT for a company

Most buyers asking about a private LLM are really asking whether employees can keep using ChatGPT. The answer has three parts.

Personal accounts are the real risk. When staff paste client contracts or source code into personal accounts, training is on unless each person turned it off, and the company has no logs and no way to delete anything. A sanctioned alternative usually stops this more reliably than a ban.

ChatGPT Business or Enterprise closes most of that gap. A company workspace gives admins control, removes training on business data by default and, on Enterprise, adds retention settings, residency and audit tooling. For a marketing team drafting copy, that is a reasonable end state.

A private LLM covers what a ChatGPT workspace cannot. That means data your contracts or regulator keep away from outside AI processors, answers grounded in internal systems with each user’s permissions enforced, a model version you can freeze for an audit, and operation without internet access. You can also mix models: a local model for confidential work and an approved cloud model for the rest.

Many companies run both, with an acceptable-use policy that says which data classes go where. An April 2025 report written for the European Data Protection Board by an outside expert is a useful input here: it sets out a risk management method for LLM systems, practical mitigations and examples of the parties’ roles under the GDPR and the AI Act, though it does not state the Board’s official position. If you are replacing ChatGPT with a self-hosted assistant, our private ChatGPT alternative page walks through the migration steps.

How good are open-weight models now?

Model quality used to settle this debate in favour of public services. The gap has narrowed, and the open-weight field now includes models from several large labs under licences that allow commercial self-hosting. As checked on Hugging Face in October 2026:

• gpt-oss-120b from OpenAI, released in August 2025 under Apache 2.0. Its model card says it runs on a single 80 GB GPU.
• Qwen3.8-27B from Alibaba’s Qwen team, released in August 2026 under Apache 2.0: a 27-billion-parameter dense model with a native context of 262,144 tokens.
• Gemma 4 from Google, released from March 2026 in sizes up to 31B, under Apache 2.0.
• DeepSeek-V4.1-Flash, released in September 2026 under the MIT licence.
• Mistral Small 4 under Apache 2.0, and Mistral Large 4, which carries Mistral’s own licence terms.

Licence terms differ between models, so legal review belongs in model selection. Leaderboards change monthly and rarely reflect your workload. A more reliable method: take 100 to 200 real prompts from the work you plan to move, run them through the open-weight candidates and your current public service, and have the people who do that work grade the answers blind. For summarisation, drafting, extraction and questions over internal documents, a well-chosen open-weight model with good retrieval often holds up well in that test. Frontier hosted models may still lead on the hardest multi-step reasoning, which is one reason to keep an approved route to them for non-sensitive work.

Hardware follows from the model choice. The VRAM calculator estimates GPU memory for a given model, precision, context length and number of concurrent users.

What each option costs, in plain terms

The three cost models behave very differently as usage grows.

Per seat. ChatGPT Business, Claude Team, Gemini in Workspace and Microsoft 365 Copilot are sold per user per month, sometimes with usage limits or credits on top. Cost grows with headcount, however lightly people use the tool. Enterprise tiers are often quote-only or add usage billing. The four vendor analyses linked above give annual figures for 250 and 1,000 users.

Per token. APIs and some enterprise plans bill by the volume of text processed. This suits light or uneven use and gets expensive for document-heavy retrieval or agents that call the model many times per task. Our explainer on flat versus token pricing shows where the curves cross.

Owned capacity. A private LLM is mainly GPU servers, power and cooling, staff time and a platform licence if you buy one. Hardware costs the same busy or idle, so cost per user falls as adoption rises, and capacity is sized for peak concurrent use. The on-prem LLM cost comparison works through a multi-year total cost of ownership.

A rough rule: a few dozen occasional users almost always cost less on seats. Hundreds or thousands of daily users, or heavy document and agent workloads, often cost less on owned capacity over three years. The AI savings calculator lets you test your own numbers, and VDF AI pricing shows how our Cloud plans and on-premises Seat Bundles are structured.

When a public enterprise tier is the right answer

A public LLM is often the right call. A business or enterprise tier is usually the better choice when:

• The data you plan to use is public, internal-but-low-risk, or already stored with the same vendor (for example, a company that keeps its email and documents in Microsoft 365 adding Copilot).
• Your contracts, clients and regulators accept a cloud AI processor under a data processing agreement.
• You have no team to run GPUs and do not want one.
• Usage is light, uneven or still unknown, so you would be guessing at hardware size.
• You need the newest frontier model on release day for research or coding work.

In those cases, spend the effort on an acceptable-use policy, single sign-on, a data classification that tells people what not to paste, and a review of the vendor’s retention and residency terms.

When you need a private LLM

A private LLM earns its cost and effort when one or more of these is true:

• Data may not leave. Client confidentiality terms, health data rules, public-sector classifications or banking secrecy forbid an outside AI processor.
• Auditors want your records. You must show who asked what, with which documents and model version, from logs you hold. EU AI Act duties for high-risk uses are easier to evidence on infrastructure you control.
• The system must run offline. Plants, labs, ships and secure networks often have no internet route.
• You need deep customisation. Fine-tuning on internal data, retrieval over systems with complex permissions, or agents acting on internal tools.
• Usage is heavy and steady. Owned capacity beats seats or tokens over three years.

The private AI overview sets out the deployment modes, the on-premises LLM guide covers sizing and rollout, and our private LLM page describes a company-wide setup.

Decision table: requirement to deployment

Buyer checklist: ten questions to answer before you sign

1. Which data classes will employees put into the tool, and which of them may go to an outside AI processor under your contracts and regulators?
2. Does the vendor train on your data by default, and what exactly is excluded (feedback, flagged content, connected apps)?
3. How long are prompts, files and outputs kept, who sets that period, and what happens under a legal hold or court order?
4. Where are prompts processed and stored, and which features or models fall outside the residency promise?
5. Can you export complete audit logs (user, prompt, documents used, model, time) to your own SIEM?
6. Can you pin a model version, and how much notice do you get before it changes?
7. Does retrieval over internal documents respect each user’s existing permissions in the source system?
8. What does year three cost at your expected number of active users, including usage above any included limits?
9. If you leave, can you take your prompts, agents, workflows and logs with you in a usable format?
10. Who operates the service day to day, and what does it cost in staff time or support fees?

For public services, questions 3, 4 and 5 are answered by the vendor’s terms. For a private LLM, the answers are your own policy, which is the main thing you are paying for. The Trust Center shows how we answer these questions for VDF AI.

Where VDF AI fits

VDF AI runs assistants, retrieval and agents on infrastructure you control: your own servers, a private or sovereign cloud, or an air-gapped network. VDF AI Cloud is a hosted option for teams that do not need self-hosting. Self-hosted deployments sit at options 3 and 4 above, with prompts, documents, embeddings, outputs and audit logs kept inside your environment.

VDF AI Router sends each request to a model you have approved, so confidential work can stay on a local open-weight model while other requests use an approved cloud model; in air-gapped mode it routes to local models only. Retrieval keeps each user’s existing permissions, and role-based access control comes with every plan. On premises, Microsoft Entra ID single sign-on is built in; other identity providers connect through an SSO-aware reverse proxy. Audit events stream to your SIEM, and encryption keys can be customer-managed.

On premises, VDF AI is licensed by seat in fixed annual Seat Bundles. Every seat includes the full platform, the licence does not meter model tokens, and any bundle can start with a three-month free platform trial. Model hardware is a separate cost. Cloud plans and bundle structure are on the pricing page.