Securing the AI Data Plane: Protecting Prompts, Embeddings, Logs, and Agent Outputs On-Premises

When a security team reviews a conventional application, it knows where the sensitive data lives: databases, object stores, backups, a handful of integrations. An AI platform breaks that mental model. The moment agents start retrieving documents, embedding content, calling tools, and generating outputs, the system produces new classes of sensitive data that did not exist before — and most of them never appear on a data inventory.

A prompt can contain personal data. A retrieved document carries the classification of its source. An embedding is derived from confidential text. A model output can restate anything the agent saw. Tool traffic moves data between systems. Logs and traces capture all of it. Audit trails are, by design, a permanent record of sensitive activity.

Keeping the platform on-premises is the right foundation — it keeps this data inside infrastructure you control. But location alone does not secure it. Each artifact is a distinct attack surface with its own handling requirements. This piece walks the seven artifacts of the AI data plane and the controls each one needs.

Why “we run it on-prem” is a starting point, not an answer

On-premises and air-gapped deployment removes the largest category of AI data risk: sending sensitive content to infrastructure you do not control. That matters, and it is why regulated organisations choose private deployment in the first place, as covered in What Is an On-Premise AI Agent Platform.

But “on-prem” describes where the data is, not how it is protected once it is there. A vector index of confidential documents sitting unencrypted on a shared volume is a breach waiting to happen, regardless of the building it lives in. An internal actor with broad log access can read every prompt any employee ever sent. The perimeter moved inside your walls; the controls have to move with it.

The useful frame is to treat the AI platform as producing a data plane — a set of artifacts that flow through the system — and to secure each artifact class deliberately. Below are the seven that matter most.

1. Prompts

Prompts are the most underestimated artifact. Users paste contracts, patient details, source code, and credentials into them without thinking of the prompt as a record. Yet every prompt is captured, often logged, and sometimes retained indefinitely.

Controls: classify prompts by the sensitivity of what they may contain, apply the same access rules as the underlying data, redact or tokenise obvious secrets before storage, and set explicit retention. A prompt store is a sensitive data store — inventory it as one.

2. Retrieved documents and context

In a private RAG system, the retrieval step pulls source content into the model’s context. That content arrives carrying its own classification — a restricted document does not become less restricted because an agent retrieved a passage of it.

Controls: enforce the source system’s access permissions at retrieval time, so an agent can only surface what the requesting user is entitled to see. Propagate the source classification onto the retrieved context, and carry it forward into logs and outputs. Retrieval that ignores source permissions turns a governed repository into an open one.

3. Embeddings and vector indexes

Embeddings feel abstract — they are just numbers — and so they are routinely stored with weaker controls than the documents they came from. That is a mistake. Embeddings are derived from source content, and research has demonstrated that under certain conditions meaningful information about the original text can be reconstructed from its vectors.

Controls: treat a vector index built from confidential material as confidential itself. Encrypt it at rest, access-control it, bind it to the same residency rules as its source, and include it in deletion workflows — when a source document is deleted for legal or privacy reasons, its embeddings must go too. An index that outlives its source is a quiet compliance failure.

4. Model outputs

A model output can restate, summarise, or combine anything the agent was exposed to. An output generated from restricted inputs inherits their sensitivity, even though it is freshly written text with no source label attached.

Controls: classify outputs based on the inputs that produced them, not on their surface content. Apply retention and access rules accordingly, and validate outputs before they are released into downstream systems — a governance step that also catches quality failures. The output is not a clean slate; it is a derivative of everything upstream.

5. Agent tool inputs and outputs

Agentic systems move data. An agent calls a ticketing API, queries a database, writes to a repository, hits an internal service. Every one of those calls carries data out of the model’s context and into another system — and back. This traffic is where an AI platform most resembles a set of machine identities acting across the estate.

Controls: scope each tool’s credentials to least privilege, gate high-impact actions behind approval, log every tool call with its inputs and outputs, and treat the tool boundary as a trust boundary. The security questions vendors tend to skip here are covered in The Enterprise AI Agent Security Questions Vendors Ignore. An agent with broad tool access and no logging is an unmonitored automated user.

6. Logs and traces

Logs are where every other artifact ends up. Prompts, retrieved context, outputs, and tool traffic are all captured in operational logging — which means the log store often contains a concentrated copy of the most sensitive data in the platform, held under looser controls because it is labelled diagnostic.

Controls: apply the same classification, encryption, access control, and retention to logs as to the underlying data. Redact secrets and, where appropriate, personal data at capture time. Restrict who can read raw prompt-and-output logs, and log that access too. The observability practices that make this workable are covered in AI Agent Observability: Logs, Traces, and Audit Trails. Do not let the diagnostic label become a governance loophole.

7. Audit trails

Audit trails are deliberately permanent — that is their value. They are also, by construction, a durable record of who did what with sensitive systems, which makes them both a control and a target. An audit trail that can be altered is worthless; one that can be read by anyone is itself an exposure.

Controls: make audit records tamper-evident and append-only, separate the ability to act from the ability to edit the record of acting, restrict read access to those with a genuine oversight need, and retain according to regulatory requirement. The audit trail is the evidence base for everything else — protect it accordingly.

Making it operational

Seven artifacts is a lot to secure piecemeal, so the controls that matter most are the ones that apply across all of them:

• Classify once, propagate everywhere. A classification assigned at the source should travel with the data through retrieval, embedding, output, and log. Consistent classification is what makes every other control enforceable.
• Least privilege at every boundary. Users, agents, tools, and services should each see only what they need. Broad standing access is the common root cause behind most of the failure modes above.
• Encrypt at rest and in transit, everywhere. Including the artifacts that feel abstract — embeddings, indexes, log stores.
• Retention and deletion as first-class workflows. When a source is deleted, its derivatives — embeddings, cached outputs, logs — must be deletable too. Orphaned derivatives are how deleted data quietly persists.
• Audit the access to the audit. Reading sensitive logs and records is itself an event worth recording.

None of this is a legal guarantee, and no single control makes a platform secure. What securing the data plane does is close the gap between “we run AI on-premises” and “we can account for every sensitive artifact our AI creates” — which is the standard regulated organisations are actually held to.

How VDF AI approaches the data plane

VDF AI is built for private, on-premises, and air-gapped deployment, so the entire data plane stays inside infrastructure the organisation controls. VDF AI Networks and VDF AI Agents enforce source permissions at retrieval, scope tool access to least privilege, and log tool calls, routing decisions, and outputs as governed records rather than loose diagnostics. Classification propagates from source through embeddings and outputs, and access to sensitive logs and audit trails is itself controlled and recorded. The goal is not just to keep AI data in the building — it is to account for every artifact the platform creates.