Data Deletion in Private RAG: Retention, Erasure, and Proving a Document Is Gone

Every private RAG deployment eventually meets the same request: remove this document, and everything derived from it.

It arrives from an employee exercising a data subject right, from legal hold expiry, from a contract that ended and took its data-sharing basis with it, or from a security incident where a file was indexed that never should have been. It is a simple request and, in most retrieval architectures, it touches six or seven stores.

Deleting the source is the easy part. Proving the content is no longer reachable is the part that gets designed after the incident rather than before it.

Count the copies before you promise deletion

A document that entered a retrieval pipeline exists in more places than the ingestion diagram suggests:

The first design decision is not how to delete. It is to require, at ingestion time, that every derived artifact carries a stable reference back to its source document so that deletion can be propagated deterministically. Systems that cannot answer “which chunks came from this file” cannot honour an erasure request reliably, and retrofitting that lineage after a large index is built is expensive.

Soft delete is a query filter, not erasure

This is the part most often misunderstood. Vector indexes commonly implement deletion by marking entries as removed and excluding them from results, deferring physical removal until a compaction or rebuild. From the application’s point of view the content disappears. From the storage layer’s point of view it is still there.

Work published in 2026 on HNSW-based vector stores demonstrated that soft-deleted embeddings remained recoverable by reading the underlying storage files directly, bypassing the API. The authors reported recovery of roughly 25-46% of personally identifiable information from text embeddings in their tests, and far higher rates on facial embeddings, and proposed per-epoch key rotation — encrypting vectors and destroying the key on deletion — as a mitigation.

The practical implications:

• Distinguish suppression from erasure in your own policy. Suppression (excluded from results) can be immediate. Erasure (no longer present in storage) requires compaction, segment rebuild, or cryptographic key destruction.
• Give erasure a bounded, documented interval. “Removed from results immediately, physically erased at the next compaction, which runs within N days” is a defensible statement. “Deleted” without qualification is not.
• Consider crypto-shredding where per-tenant or per-epoch encryption keys make key destruction the erasure mechanism, which is often more tractable than forcing full index rebuilds on demand.

The stores people forget

Caches. A semantic answer cache can return a synthesised answer containing text from a document that has already been removed from the index. Cache invalidation must be keyed to source document identity, not just to the query string.

Conversation history. If users can reopen prior chats, the retrieved passage lives in the transcript. Decide explicitly whether erasure rewrites history, deletes the affected turns, or is scoped to the retrieval layer only — and say so in the policy.

Execution logs. Observability that stores full prompts and retrieved context is a copy of the content. Sound practice is to log identifiers and hashes plus a redacted excerpt, keeping the audit trail useful without turning it into a shadow index that outlives its source.

Backups. Retention design must state what happens when a pre-deletion snapshot is restored. The usual answer is a persistent deletion queue that is replayed against any restored system as a mandatory recovery step.

Fine-tuned models. If enterprise content was used to fine-tune a model, deleting the index does not affect the weights. Removing a specific example’s influence from trained parameters is an open research problem, which is one of several reasons retrieval is a better default than fine-tuning for content with an uncertain retention future.

What regulators are actually asking

Under the GDPR, erasure rights apply to personal data wherever it is processed, and the EDPB has been explicit that AI systems do not sit outside that regime by default. Opinion 28/2024, adopted in December 2024, holds that models trained with personal data cannot be assumed anonymous and requires case-by-case assessment of whether individuals can be identified or their data extracted; the EDPB’s supporting work on data subject rights addresses how rectification and erasure can be implemented in AI systems. The EU AI Act adds record-keeping and traceability expectations on top of that, not instead of it.

The question a supervisory authority or internal auditor will ask is procedural: show the request, show what was deleted, show where, show when, show who approved it, and show the verification. That is answerable with a deletion register and a repeatable runbook. It is not answerable with a claim that the platform “supports deletion”.

A workable deletion design

1. Lineage at ingestion. Every chunk, embedding, index entry and cache entry references its source document identifier and version — the natural extension of a secure ingestion pipeline.
2. A single deletion API. One call fans out to every derived store rather than each team writing its own cleanup script.
3. Two-phase semantics. Immediate suppression from retrieval and cache; scheduled physical erasure with a stated maximum interval.
4. Access-aware scope. Deletion inherits the same partition rules as retrieval, so a tenant-scoped erasure cannot reach across boundaries — see permission-aware private RAG.
5. Verification. Post-deletion retrieval probes and cache checks, recorded against the request.
6. A deletion register. Request, scope, artifacts, stores, timestamps, approver, verification result — retained as evidence.
7. Backup replay. The deletion queue is re-applied after any restore, as a named step in the recovery runbook.

VDF AI runs private retrieval, local models and agent orchestration inside the customer’s environment, so the index, the caches and the execution records stay within one governed boundary — which is what makes a deletion claim verifiable rather than contractual. VDF AI Data governs the ingestion and retrieval layer; VDF AI Networks records what was retrieved and by whom, so a deletion request can be scoped against evidence rather than assumption.

Design the deletion path with the ingestion path. Retrofitting it is where the cost lands.