HIPAA Compliant LLM: Which APIs Sign a BAA and When Self-Hosted Models Are the Better Route (October 2026)

People who search for a HIPAA compliant LLM usually want one of three things: a model they can safely send patient notes to, an API they can build a clinical app on, or a vendor that will take the compliance question off their hands. The last one does not exist. HIPAA places duties on covered entities, meaning health plans, clearinghouses and providers that bill electronically, and on the business associates that handle protected health information (PHI) for them.

This guide covers the other two. We checked each major provider’s BAA page on 22 October 2026, compared the hosted route with self-hosted open-weight models, and mapped the Security Rule safeguards to the parts of an LLM application. If your question is about ChatGPT specifically, our plan-by-plan review of ChatGPT and HIPAA covers every tier.

Is there a HIPAA compliant LLM?

A language model is a file of weights. It does not store records, grant access or report breaches, so HIPAA has nothing to say about the model itself. What HIPAA governs is the path PHI takes: who receives it, where it is stored, who can read it and what happens when something goes wrong.

That turns the question into two practical ones. First, does any outside party create, receive, maintain or transmit PHI on your behalf? If so, 45 CFR 164.502(e) requires a written business associate agreement with it before the data flows. Second, does the application around the model meet the Security Rule’s safeguards? That part is always yours, whichever route you choose.

Vendors that call a model “HIPAA compliant” mean that they will sign a BAA and that their service supports the controls you need. HHS’s Office for Civil Rights says it does not endorse or certify technology products, and Microsoft’s compliance documentation notes that HHS approves no certification standard through which a business associate could demonstrate HIPAA compliance.

Routes for putting PHI into a language model

The second row is the quickest to build. The last row is the only one in which no outside company receives patient data at all. Many health systems run both: a hosted model for work without PHI and a local model for anything that touches the record.

HIPAA compliant LLM APIs: which providers sign a BAA

Every provider below will sign a BAA, but each one covers specific services and features. Features outside the list are not covered even when they sit in the same console.

Two points from that table deserve emphasis. Retention settings are part of the deal: OpenAI ties PHI processing to specific retention configurations, and Anthropic’s page sets retention conditions for covered models. And agent features are where coverage most often stops. Web fetch, code execution, file stores and third-party connectors are exactly the tools an agent needs, and several are excluded.

Google renamed Vertex AI in April 2026. The platform now appears as Gemini Enterprise Agent Platform, and its BAA listing uses that name along with “Generative AI on Agent Platform”. Contracts and security reviews written in 2025 may still say Vertex AI, which is the same service.

Self-hosted open-weight models keep the model vendor out of the PHI path

When you download an open-weight model, the publisher sends you a file and never sees your prompts. Google does not receive the notes you summarise with MedGemma, and OpenAI does not receive what you ask gpt-oss. With no PHI flowing to the publisher, there is no business associate relationship to cover.

The remaining agreements depend on who runs the hardware:

• Your own data centre. No outside AI party is involved. Hardware maintenance and remote support vendors may still need a BAA if they can reach systems holding ePHI.
• Your account at a cloud provider. The cloud provider is a business associate, because it stores and processes ePHI on your behalf even when it cannot read it. Its standard BAA usually covers GPU instances, but check the service list.
• A managed AI vendor. If a third party operates the model servers for you, it is a business associate and needs its own BAA, with its subcontractors bound the same way.

Self-hosting moves work onto your team. You choose and test models, size GPUs for peak demand, patch the serving stack and keep model versions stable. Our guide to open medical models and their GPU needs covers model choice, and the guide for healthcare CIOs and CISOs compares deployment patterns.

Security Rule safeguards for an LLM application

The technical safeguards in 45 CFR 164.312 were written long before language models, but each maps to a specific part of an LLM stack:

The administrative safeguards in 164.308 sit on top. The risk analysis must include the AI application, its connectors and its data flows, and workforce training must tell staff which tool is approved for PHI and which tools are not.

Retrieval needs particular care. If a clinician’s assistant can search every document in a shared index, it can surface records the clinician has no reason to see. Permission-aware retrieval, where each search is filtered by the user’s own access rights, is how access control survives the move to AI.

Prompts, outputs and logs are ePHI too

An LLM application copies PHI into more places than most teams expect:

• the prompt and the context window, including retrieved passages;
• conversation history kept for the user;
• the vector index and embeddings built from clinical documents;
• the model server’s request logs, caches and traces;
• monitoring tools that capture payloads for debugging.

Each copy is electronic PHI once it is stored. The HHS proposal to update the Security Rule says so directly for AI: its preamble states that ePHI in AI training data, prediction models and algorithm data maintained for covered functions is protected by the HIPAA Rules, and that a risk analysis should consider the type and amount of ePHI an AI tool accesses, who receives it and who receives the output.

In practice, list every store in your asset inventory, give each one an owner and a retention period, and switch off payload logging in tools that do not need it. Hosted APIs have their own logs as well. OpenAI, for example, keeps abuse monitoring logs for up to 30 days by default, which is why its BAA depends on specific retention settings.

Minimum necessary and de-identification before the prompt

The minimum necessary standard in 164.502(b) and 164.514(d) applies to AI just as it applies to email. An assistant drafting a prior authorization letter needs the diagnosis, the requested service and the supporting history, not the whole chart. Build prompts from the fields a task needs, and limit which roles can use features that pull full records.

De-identification removes data from HIPAA altogether, but only if it is done properly. Section 164.514(b) offers two methods:

1. Safe Harbor. Remove 18 kinds of identifiers, including names, geographic units smaller than a state, all date elements except the year, phone numbers, medical record numbers and full-face photographs, and have no actual knowledge that the rest could identify someone.
2. Expert Determination. A qualified expert applies statistical methods, finds the risk of re-identification very small and documents the method.

Free-text clinical notes are hard to de-identify with Safe Harbor, because dates, places and rare details are scattered through the prose. A redaction model can help, but treat its output as PHI until an expert method or a documented review says otherwise. For most clinical use, it is simpler to keep the data identified and keep it inside your boundary.

The proposed Security Rule update in October 2026

HHS published a proposed rule to strengthen the Security Rule on 6 January 2025 (90 FR 898), with comments closing on 7 March 2025. Among other changes, it would remove the distinction between required and addressable implementation specifications, and require a written technology asset inventory and a network map of systems that affect ePHI.

It has not been finalised. The most recent Unified Agenda entry for RIN 0945-AA22 lists the rule among long-term actions, with final action dated July 2027. The current Security Rule applies until then.

For AI projects the direction is clear enough to plan for. An asset inventory and network map that already include model servers, vector stores and AI logs will be needed under the current risk analysis duty anyway, and would be required in writing if the proposal is adopted.

Choosing a HIPAA-ready model for each healthcare workload

Model quality is the smaller part of this decision. Start from the data path:

• Administrative drafting with PHI, such as prior authorization letters and referral packets: a hosted model under a BAA works if the features you need are covered; a local model works if they are not. A person reviews each output before it is sent, as in the prior authorization use case.
• Clinical documentation: notes are dense with identifiers and go into the legal record. A local model inside your network keeps the data path short, and the clinician signs the final note. See the clinical documentation support use case.
• Agents that use tools: check each tool against the provider’s covered list. Web fetch, code execution and third-party connectors are often excluded from BAAs, so self-hosting is frequently the only way to run them with PHI.
• Research on de-identified data: either route works once the de-identification method is documented.

Before go-live, record the decision: which model, which version, which endpoints, which agreement and which retention setting. That record is what an auditor will ask for.

How VDF AI fits a HIPAA program

VDF AI is a self-hosted platform for private assistants, retrieval over your own documents and governed agents. It installs on your servers, in your private cloud or in an air-gapped network, so prompts, retrieval indexes, conversation history and audit logs sit in storage you operate, and PHI reaches only the models you host. VDF AI Router registers local endpoints served by Ollama or vLLM next to any cloud models your policy approves, and air-gap mode keeps every request on local models.

On-premises installations sign users in through Microsoft Entra ID, with Entra security groups mapped to roles; other identity providers connect through an SSO-aware reverse proxy. Role-based access control is on every plan, retrieval respects each user’s permissions, agents can pause for human approval before they act, and each prompt, retrieved source and tool call is logged. These are building blocks for the safeguards in 164.312, not a compliance status: VDF AI holds no HIPAA certification, because none exists, and your own risk analysis decides whether a deployment meets the rule.

The clinical documentation agent is one starting point. The healthcare AI brief, the healthcare and life sciences solution and the private GPT guide for healthcare cover the wider rollout. This article is general information rather than legal advice.