Private RAG Data Architecture for Multi-Tenant Environments

Single-tenant private RAG is a solved shape. Point retrieval at the organisation’s documents, scope it by department, ground the model’s answers in the results. The department isolation problem is real but bounded, and a mistake is usually an internal embarrassment.

Multi-tenant is a different problem, and the difference is consequence. When one platform serves several genuinely separate parties — a law firm’s clients, a group’s legally distinct subsidiaries, the customers of a hosted deployment — a retrieval result crossing a boundary is not an embarrassment. It is a contractual breach, a regulatory notification, and in some sectors the end of the engagement. This piece is about the data architecture that keeps that from happening, and about which of those decisions are expensive to revisit later.

Define the tenant boundary first

Before choosing a mechanism, be precise about where the boundaries are. Common tenant boundaries in enterprise deployments:

• Client boundaries in professional services — law, audit, consulting, insurance broking — where confidentiality obligations run to each client individually and conflict-of-interest rules may prohibit certain material being reachable at all.
• Legal entity boundaries in a group structure, where subsidiaries in different jurisdictions have data residency and disclosure obligations that do not travel across the group.
• Hosted customer boundaries, where a partner or managed service provider runs the platform on behalf of several customers.
• Contractual boundaries within one organisation — programme data covered by a customer agreement or a government contract that restricts access more tightly than internal policy does.

The useful test is consequence, not structure. If a document appearing in the wrong result set would trigger a notification obligation rather than a shrug, it is a tenant boundary and needs a structural control.

Three isolation patterns

Multi-tenant retrieval architectures fall into three broad shapes, and the choice is mostly a trade between isolation strength and operational cost:

Siloed. A separate retrieval stack per tenant — separate index, separate storage, in the strongest variant separate compute. Isolation is structural: a query in one tenant’s stack has no path to another’s data. The cost is duplication, and it grows linearly with tenant count.

Pooled. One shared index with tenant identity carried in metadata and enforced at query time. Cheapest to operate and simplest to scale, but the isolation is only as strong as the correctness of every query path. One path that forgets the filter is a leak with nothing behind it.

Bridged. A shared platform and shared models, with a separate index or namespace per tenant underneath. Structural separation of the data, shared operation of everything else. For most enterprise and hosted deployments this is the right default: it puts a hard boundary where the consequence is highest and shares infrastructure where the consequence is low.

A pragmatic refinement is to tier it — bridged by default, siloed for the tenants whose contracts demand demonstrable physical separation. That lets you offer a stronger isolation commitment to the customers who require it without duplicating everything for everyone.

Enforce at retrieval, never in the prompt

The single most important rule in this architecture: tenant scope must be enforced by the retrieval layer, from an identity the caller cannot choose.

The failure mode worth naming explicitly is the system that resolves tenant scope from something the request carries — a parameter, a session field, or worse, an instruction in the prompt telling the model which tenant’s data to consider. Prompt-level scoping is not a control. It is a request, and models do not enforce requests. Anything reachable by the retrieval call is reachable, whatever the prompt says.

The workable pattern is that tenant identity is derived at authentication, propagated as a claim through the request path, and applied by the retrieval layer as a non-optional predicate — with the underlying index or namespace selected from that claim rather than from anything in the request body. Then a malformed or malicious query returns nothing instead of returning someone else’s documents.

That principle extends to the agent layer. When a tool-using agent reaches a database rather than a document store, the same rule holds: the connection’s scope is bound to the tenant claim, not chosen by the agent.

Tag at ingestion, not at query time

Tenant attribution has to be established when a document enters the system, because it cannot be reliably reconstructed afterwards. A secure ingestion pipeline for a multi-tenant deployment needs a few properties:

• Every chunk carries its tenant, source, and version. Attribution is per chunk, not per document, because chunks are what retrieval actually returns.
• Untagged content fails closed. A document that arrives without resolvable tenant attribution is rejected, not indexed with a default. Defaults are how content ends up in the wrong place.
• The source connector is bound to a tenant. A connector to a shared drive or mail store should be configured for one tenant, so a mis-scoped folder cannot silently pull another tenant’s material into the wrong index.
• Deletion is a first-class operation. Offboarding a tenant, or honouring a deletion request, has to remove their content from the index, the embeddings, and any derived caches. This is dramatically easier when the tenant has its own namespace and dramatically harder in a pooled index — which is one of the strongest practical arguments for the bridged pattern.

What can safely be shared

Not everything needs separating, and over-separating has real costs in memory, GPU capacity, and operational complexity.

Models can be shared. One embedding model and one set of rerankers across tenants does not mix anyone’s data: embeddings are computed per document and written to that tenant’s own index. Sharing keeps the memory footprint predictable and means retrieval quality improvements land everywhere at once. Per-tenant models are justified by vocabulary and domain, not by isolation.

Generation capacity can be shared, with the usual caveats: no cross-request caching of retrieved content, and no shared conversational memory across tenant boundaries.

Audit records must not be shared. Each tenant’s retrieval and agent activity should be separately attributable and separately exportable — because sooner or later one tenant will ask for their audit trail, and the answer cannot involve filtering it out of everyone else’s.

Where multi-tenancy meets sovereignty

Multi-tenant designs collide with data residency more often than teams expect. Two tenants in different jurisdictions may not be lawfully served by one index in one location, however well isolated it is logically — because the obligation is about where the data physically rests, not about who can query it. That pushes some deployments toward regional silos regardless of what the isolation model would otherwise allow.

This is one of the places where the on-premises posture earns its keep. When the retrieval layer, the models, and the logs all run in infrastructure you control, tenant placement is a deployment decision you make rather than a vendor region you select — and demonstrating to a customer or regulator exactly where their content lives and who can reach it becomes a matter of showing your own architecture rather than relaying a third party’s assurances.

How VDF AI supports multi-tenant private RAG

VDF AI enforces retrieval scope at the data layer rather than in the prompt, so an agent’s reach is bounded by policy it cannot talk its way around. Data connections and document sources are scoped explicitly, which lets a deployment place structural boundaries between tenants while sharing models and platform infrastructure underneath — the bridged pattern, without building the plumbing yourself. Retrieval and agent activity are logged per scope, so a tenant’s audit trail can be produced without disentangling it from anyone else’s. And because the whole platform runs inside your own environment, tenant placement, residency, and deletion stay decisions you make and can evidence — which is what customers under contractual confidentiality obligations actually need to see.