Permission-Aware Private RAG: Preserve Document Access Controls

Moving retrieval-augmented generation behind the firewall solves an important problem: documents, embeddings, prompts, and model outputs no longer have to pass through a public AI service. It does not solve a different problem: whether the person asking a question was entitled to see every passage the retriever found.

That distinction is easy to miss. A company can deploy a vector database, embedding model, reranker, and local LLM entirely in its own data center and still create a powerful internal disclosure channel. If the index mixes board papers, HR files, legal matters, operating procedures, and ordinary intranet pages without preserving source permissions, the model can summarize material a user could never open directly.

Permission-aware private RAG closes that gap. It treats identity and authorization as part of retrieval itself, not as a login check around the chat interface. The design principle is simple: the AI may retrieve no more than the requesting identity could retrieve from the source. Implementing that principle requires decisions at every stage of the pipeline.

Data locality and data entitlement are separate controls

On-premises deployment establishes a processing boundary. It helps an enterprise determine where prompts, document fragments, embeddings, logs, and outputs are stored and processed. Permission-aware retrieval establishes an entitlement boundary. It determines which subset of that locally held information may be used for a particular request.

Regulated organizations need both. Residency without authorization can contain a breach geographically while still exposing confidential material internally. Authorization without locality can restrict users correctly while sending approved content to an infrastructure location that policy forbids. A defensible private AI architecture evaluates the two controls independently and requires both to pass.

This follows the zero-trust principle that network location should not create implicit trust. Being connected to the corporate network, or reaching an application hosted in the company data center, is not evidence that an identity may read every indexed resource. The same reasoning underpins the zero-trust architecture for on-premises AI: access is granted per request, for a specific resource and action.

The permission-aware retrieval path

A production query should pass through a visible sequence of controls:

1. Authenticate the caller. Establish the human, agent, or service identity and validate session context. Do not reduce every request to a shared application account.
2. Resolve current entitlements. Obtain groups, roles, attributes, case assignments, geographic restrictions, and any purpose-based rules relevant to the request.
3. Classify the request. Determine the workflow, data class, permitted repositories, and model-routing constraints before search begins.
4. Security-trim retrieval. Search only chunks whose access policy allows that identity and purpose. Restricted candidates should be excluded before reranking.
5. Rerank the allowed set. The reranker receives only authorized passages. It must not become a side door around the retrieval filter.
6. Generate from approved context. The prompt builder uses the trimmed results and the model router selects an eligible local model for the data class.
7. Validate citations and output. Confirm that citations point to documents the caller can still open and that the answer does not combine permitted facts into a prohibited inference.
8. Record the decision. Preserve identity, policy version, allowed repositories, retrieved chunk identifiers, model version, output, and any approval step in the audit trace.

The order matters. If authorization occurs only at the user interface, every component behind it operates with excessive privilege. If it occurs only after generation, the restricted data has already reached the model and may already be present in logs, caches, traces, or output filters.

Carry source permissions into the index

Permission-aware RAG begins during ingestion. Every document needs a stable source identifier and an authorization envelope that can be evaluated later. Depending on the source, that envelope may contain user and group access lists, classification labels, matter or case identifiers, business-unit ownership, residency tags, legal holds, and explicit deny rules.

Chunks should inherit the document’s restrictions by default. If a source format contains sections with tighter access — an annex, hidden worksheet, redacted page, or row-level restriction — the chunk metadata must reflect the narrower rule. A child chunk must never become more accessible than its parent because an ingestion pipeline dropped security metadata during parsing.

Two implementation choices deserve particular attention:

• Pre-filtered vector search applies authorization constraints inside the retrieval query. This is generally the safer and more efficient pattern because restricted chunks never enter the candidate list.
• Physically separated indexes place high-sensitivity or strongly isolated domains in different stores. They simplify some policies and reduce blast radius, but increase operational overhead and make authorized cross-domain questions harder.

Many enterprises use both: separate indexes for hard boundaries such as legal privilege or national-security zones, and metadata-based pre-filtering within each zone. Post-filtering can provide defense in depth, but it should not be the primary permission mechanism.

Embedding vectors also require protection. They are derived from source content, can reveal similarity relationships, and may be linked back to sensitive records through metadata. Treat the vector store as part of the AI data plane, with encryption, backup controls, administrative separation, and access logging appropriate to the source material.

Keep permissions current after ingestion

A perfectly permissioned index becomes unsafe when its authorization data goes stale. People change teams, investigation rooms close, cases are reassigned, contracts expire, and documents are reclassified. A source revocation should not wait for the next weekly embedding job.

Design permission synchronization as a separate path from content synchronization. Access-control changes are usually small and should propagate faster than full document reprocessing. Practical controls include event-driven updates from the source system, short-lived entitlement caches, cache invalidation on high-risk revocations, reconciliation jobs to detect drift, and a deny-by-default response when the policy service is unavailable.

Deletion needs equal rigor. Removing the visible document is not enough if old chunks remain in the vector index, retrieval cache, prompt cache, evaluation dataset, or trace store. The secure private RAG ingestion lifecycle should include a deletion manifest that proves where derived artifacts were removed or retained under an explicit policy.

Set a measurable target for permission freshness. For example, a legal repository might require revocations to affect retrieval within minutes, while a low-sensitivity knowledge base may tolerate a longer window. The correct number comes from the data risk and source-system behavior, not from the convenience of the indexing schedule.

Do not lose the user’s identity at the agent boundary

Agentic workflows complicate authorization because an agent may call retrieval as one step in a longer plan. The dangerous shortcut is to let every agent use a powerful service credential. That credential can reach more data than the user, and the orchestration layer may quietly turn a narrow question into a broad search.

Separate three identities in the trace:

• the requesting identity whose entitlements bound the data search;
• the agent identity that identifies the governed workflow and its approved purpose;
• the service identity used to authenticate the technical call.

Access should be the intersection of those scopes, not the union. A user cannot gain access because an agent has a broad connector, and an agent cannot exceed its registered purpose because a user happens to hold a powerful role. High-impact exceptions should route to an explicit human approval rather than silently switching to a privileged service account.

This pattern is also essential for auditability. A log that records only rag-service as the caller cannot answer who initiated the retrieval, which workflow justified it, or which entitlement was evaluated.

Test for cross-boundary leakage, not only answer accuracy

Standard RAG evaluation asks whether the system retrieved the right passage. Security evaluation must also ask whether it refused to retrieve the wrong passage for the wrong identity.

Build a test matrix that pairs realistic questions with users from different groups. Include direct requests for restricted documents, indirect questions whose answers exist only in restricted sources, renamed groups, revoked users, conflicting allow and deny rules, stale caches, deleted content, and multi-hop questions that combine facts across repositories. Verify the candidate set, reranker input, prompt context, citations, answer, and trace — not just the final response.

Track at least four measures: unauthorized retrieval rate, authorized retrieval recall, permission-change propagation time, and audit-record completeness. A system can achieve zero observed leakage by denying every query, so the security measure must be evaluated alongside usefulness. The private RAG accuracy framework provides the quality side of that paired assessment.

How VDF AI supports the architecture

VDF AI Agents provides role-based access over tools and knowledge sources and records the retrieval hits, tool calls, model response, and output for each execution. VDF Data Suite keeps RAG data preparation, embeddings, evaluation material, and fine-tuning workflows inside the customer’s environment. VDF AI Networks adds governed multi-step orchestration, per-agent tool allowlists, model routing, approval points, and full execution traces.

For a permission-aware deployment, those platform controls should be connected to the enterprise identity provider and the source repositories’ authorization model. The implementation goal is not a second, manually maintained copy of every permission. It is a retrieval path that can prove which identity and policy were evaluated, which chunks were allowed, and why the model received them.

Private RAG earns trust when it preserves the security semantics users already rely on. Keeping data on-premises is the foundation. Ensuring the AI never retrieves beyond the caller’s rights is what makes that foundation operationally safe.