How to Use Metadata Filters in Private RAG (and Why They Are a Security Control)

In most enterprise RAG discussions, metadata filtering shows up as a tuning technique: add a date filter to stop the system citing a policy from 2019, add a document-type filter so a question about contracts does not retrieve marketing collateral. Useful, unglamorous, tuning.

In a private RAG deployment serving multiple departments from one corpus, metadata filters are doing something else entirely. They are the mechanism deciding whether a user in one business unit can retrieve a document belonging to another. That is not a relevance knob. That is an access control boundary — and it is often the only one standing between a shared index and a disclosure incident.

Treating it as tuning is how organisations end up with a system that answers HR questions correctly and occasionally quotes a confidential compensation review to whoever asked nicely.

The structural problem with a shared index

The appeal of one corpus is obvious. Ingest everything once, embed it once, and let any authorised user query across the organisation’s knowledge. It is cheaper to run, simpler to operate, and far better at answering cross-functional questions than a set of isolated silos.

The problem is that a vector index has no native concept of who is asking. It contains vectors and their nearest neighbours. Similarity search over an unfiltered index will happily return the most semantically relevant chunk regardless of whether the requester should ever see it — and the generation model downstream will faithfully summarise it, with a citation, because that is precisely what it was built to do.

The permissions that governed those documents in SharePoint, in the file system, in the DMS — none of them survive ingestion unless you deliberately carry them across. Embedding a document strips it of everything except its text and whatever metadata you chose to attach. This is why department-level data isolation in RAG has to be designed at ingestion rather than bolted on at query time.

Pre-filter, always — the ordering is the control

The single most consequential decision in filtered retrieval is when the filter runs relative to the search.

Post-filtering searches the whole index, then removes results the user is not allowed to see. It is easier to implement, and it is indefensible as a security control. Restricted content was retrieved. It existed in the process. It was in a variable. Every layer between that retrieval and the final response — logging, tracing, caching, error handlers, the debug output someone enabled during an incident — is now a potential disclosure path. The filter is a curtain, not a wall.

Pre-filtering constrains the search itself, so restricted vectors are never candidates. Nothing the user cannot see is ever retrieved, so no downstream component can leak it.

For relevance tuning the distinction barely matters. For access control it is the whole thing. The rule worth writing down: filters that express permissions must be pre-filters, and they must be applied server-side from the authenticated session — never passed in from the client and never assembled by the model.

That last clause matters more than it first appears. An agent that constructs its own retrieval filters is an agent that can construct the wrong ones. Permission filters belong in the retrieval layer, applied unconditionally from the caller’s identity, in a code path the model cannot influence. The model may add filters that narrow the search further; it must never be able to widen it. This is the same containment principle described in securing the AI data plane.

Designing the metadata schema

Metadata is only as good as the ingestion that produces it, and the failure mode is subtle: filters that work perfectly against metadata that is quietly wrong.

A workable schema covers four distinct jobs:

Access dimensions — the fields that express who may see the chunk. Department, business unit, tenant, clearance level, project. These should be derived from the source system’s access control at ingestion, not assigned by hand. Hand-assigned classification drifts within weeks, and drift here means either over-restriction (users cannot find things, so they stop using the system) or under-restriction (they find things they should not).

Provenance — source system, document ID, version, ingestion timestamp. This is what lets you answer “where did this answer come from and is that document still current?” It is also what makes the audit trail meaningful rather than decorative.

Lifecycle — status and effective dates. A superseded policy is arguably worse than no policy, because it is confidently wrong and looks authoritative. Any corpus containing versioned documents needs a status field that excludes drafts and superseded versions by default.

Content type — the ordinary relevance dimension, and the one that gets the most attention despite being the least critical of the four.

Capture fields you will filter on. Speculative metadata — “we might want to filter by author one day” — adds ingestion fragility and buys nothing until that day arrives.

Keeping metadata synchronised with reality

Here is the failure that reliably surfaces six months after go-live, when the system has been declared a success and nobody is watching it closely.

Someone changes a document’s permissions in the source system. A contract moves to a restricted folder. An employee changes department. A project team is disbanded. The source system reflects this immediately. The vector index does not — it holds metadata captured at ingestion, which is now a snapshot of a permission model that no longer exists.

The index is now enforcing last quarter’s access rules with complete confidence. Nothing errors. Nothing alerts. Retrieval works exactly as designed, against stale premises.

Handling this properly requires deciding, per source, how permission changes propagate:

• Re-ingest on change — a webhook or change feed from the source system triggers metadata refresh. Cleanest, and requires the source system to emit change events.
• Scheduled reconciliation — periodically re-read ACLs from source and update index metadata. Simpler, with a window of staleness you must be able to state and defend.
• Resolve at query time — store only a stable identifier in the index and resolve current permissions from the source system during retrieval. Always correct, at the cost of a dependency on the source system’s availability and latency on every query.

Most deployments end up with a mix, chosen per source according to how volatile its permissions are and how sensitive its content is. The one option that is not available is having no answer, which is nonetheless the most common state of affairs. The database and document connector layer is where this synchronisation logic belongs, and it should be specified before the first document is ingested rather than discovered afterwards.

Filters, rerankers, and where relevance actually comes from

Filtering and ranking are frequently conflated, and they solve different problems.

A filter is binary and about eligibility: this chunk is permitted and in scope, or it is not. A reranker is continuous and about ordering: given eligible candidates, which best answers this question?

Trying to make filters do ranking work produces brittle retrieval — over-narrow filters that exclude the answer, or filter sets so complex nobody can predict their behaviour. Trying to make ranking do filtering work produces the post-filtering failure described above. Filter for eligibility, rerank for relevance, and keep the two concerns separate in the code as well as in the design.

One practical note on interaction: a highly selective filter can degrade approximate nearest-neighbour recall, because the index was built over the whole corpus and the filter may leave too few candidates in the regions the search explores. Behaviour varies considerably between vector stores. Test with filter selectivity that reflects production — a filter matching two percent of the corpus behaves nothing like one matching sixty.

Testing what you cannot see

Access-control filters have an unpleasant property: when they fail, nothing looks broken. The system returns an answer. The answer is well-formed and correct. It was simply assembled from a document the user should never have been shown, and nobody notices until someone recognises content they were not meant to have.

This has to be tested adversarially, not incidentally:

• Query as a user in one department for content you know exists only in another, and confirm the system reports it cannot find it — rather than answering.
• Confirm that the absence of a result does not itself leak. “I cannot answer that” and “I am not permitted to show you the Q3 restructuring plan” are very different responses, and only the first is safe.
• Change a permission in the source system and verify the index reflects it within your stated window.
• Test that a user cannot influence their own filters through the prompt. Ask the system directly to search other departments and confirm it cannot comply.

That last test is the one most often skipped and most often revealing, particularly where an agent has any hand in constructing retrieval parameters. It sits alongside the broader controls in preventing data leakage in enterprise AI systems.

Metadata filtering is not the exciting part of a private RAG build. It is, in a shared corpus serving a regulated organisation, the part that determines whether the system can be deployed at all.