How to Deploy AI Agents in Production on Your Own Infrastructure: A 12-Step Checklist

To deploy AI agents in production on your own infrastructure, fix the data boundary, build a runtime environment with local model serving and block outbound traffic by default. Then wire identity and short-lived secrets, register every tool with least-privilege scopes, connect knowledge through permission-aware retrieval and define how multi-agent workflows fail. Put approval gates on consequential actions, trace and audit every run, release only after evaluation and in stages from shadow mode, and have service levels, a kill switch and a rehearsed rollback ready before any agent acts on a live system.

This checklist covers the platform and runtime side of the work. Choosing the workflow, setting autonomy per action and building the evaluation set are programme decisions, covered in the ten-step implementation roadmap; the steps below assume those decisions exist and turn them into infrastructure, configuration and evidence.

National cyber agencies have now described what careful deployment looks like. On 1 May 2026, the Australian Signals Directorate’s ACSC, CISA and the NSA in the United States, and the national cyber centres of Canada, New Zealand and the United Kingdom published joint guidance on adopting agentic AI. Its key actions: take a security-first approach across design, deployment and operation; limit initial use to low-risk, well-defined tasks; apply strong oversight, continuous monitoring and human control; and keep assessing risk and threat models. Several steps below draw on its recommendations.

The checklist at a glance

Before step 1, agree three inputs: a named owner and workflow for each agent, the data classification scheme your security team already uses, and separate development, test and production environments. Hardware choices are covered in the guide to sizing on-premises AI servers and the reference architecture for on-premises AI; if you are installing VDF AI, its self-hosting infrastructure requirements are documented separately.

The 12 steps for deploying AI agents in production

Step 1. Classify the data and draw the deployment boundary

List every source each agent will read and every system it may write to, and label each with your existing data classes. Decide which classes may ever leave the perimeter, and place each component (model, runtime, tools, vector store, logs) accordingly; the comparison of AI agent deployment models covers the options. Note which regimes apply: an Annex III use case under the EU AI Act, a critical or important function under DORA, or special-category personal data under GDPR.

Output: a boundary record per agent, listing data classes, component locations and applicable regimes, signed by the agent’s owner.

Step 2. Build the runtime environment

Kubernetes suits estates with several agents and teams, because namespaces, network policies and quotas give each agent its own fenced area; a few virtual machines are enough for a single small deployment. Kubernetes 1.37 is the newest minor release as of early November 2026, with 1.38 in beta, and every minor line has a published end-of-life date, so plan upgrades before go-live. On GPU nodes, NVIDIA’s GPU Operator manages drivers, the device plugin, the container runtime and DCGM monitoring.

For model serving, vLLM (Apache 2.0) and SGLang (Apache 2.0) are common choices for many concurrent users, and both offer OpenAI-compatible APIs; Ollama (MIT) is a simpler option for development and small teams. NVIDIA NIM packages vLLM or SGLang with NVIDIA’s optimisations in containers that can be deployed on-premises and air-gapped; NVIDIA offers it free for development under its Developer Program and asks for an NVIDIA AI Enterprise licence in production. vLLM, SGLang and Ollama each shipped a release between 2 and 3 November 2026, so pin exact versions and container digests. The comparison of vLLM, Ollama, llama.cpp and SGLang, the Kubernetes hosting guide and the overview of on-premises LLM deployment go deeper, and the enterprise handbook for running local LLMs gathers the serving stack in one PDF.

Pull model weights once, verify their hashes and serve them from an internal registry.

Output: an environment bill of materials with cluster and driver versions, serving engines and digests, and every model with its hash and licence.

Step 3. Deny outbound traffic by default

Block egress at three layers: network policies per namespace, host firewalls and the perimeter. Route the few permitted destinations through one proxy with an allow list, and give every entry a purpose and an owner. Mirror container images, packages and model weights internally so nothing is fetched at run time. Then cut the outbound link and run the regression suite: whatever breaks is an undeclared dependency. The method for proving no hidden callouts and the zero-trust network design for on-premises AI cover the detail; where there must be no outbound route at all, the air-gapped deployment guide adds the offline update and model-import steps.

Output: an egress register with destination, purpose, owner and review date.

Step 4. Wire identity and secrets

People sign in through your identity provider, with roles mapped from directory groups. Platform components get their own service accounts. Credentials for tools come from a secrets manager such as HashiCorp Vault or OpenBao (MPL 2.0), are short-lived and are scoped to one tool; the joint guidance asks for just-in-time credentials for high-impact or privileged actions. Check entitlements on every call: the same guidance warns that permissions evaluated only at start-up leave a “stale ‘allow’ decision” for an attacker to exploit.

The MCP specification revision dated 28 July 2026 makes HTTP-based MCP servers OAuth 2.1 resource servers: tokens must be issued for that specific server, and servers must not accept or pass through tokens meant for anything else. Servers run over stdio take credentials from their environment instead, so inject those from the vault at start-up rather than storing them in configuration files. Record on whose behalf each action ran; the guide to AI agent permissions covers the access model.

Output: an identity and secrets map showing every principal, its roles, its credentials and their lifetime.

Step 5. Register tools and connectors with least privilege

Every tool, whether an MCP server, an HTTP tool or a code runner, gets a register entry: owner, version, read or write scope, data classes touched, rate and value limits, whether approval is needed and which agents may call it. The joint guidance recommends restricting tool use “to an approved allow list of tools and versions”, and the MCP specification tells hosts to treat tool descriptions and annotations as untrusted unless they come from a trusted server. OWASP’s Top 10 for Agentic Applications, published on 9 December 2025, lists tool misuse and exploitation as ASI02 and agentic supply-chain vulnerabilities as ASI04. Run generated code in a sandbox, as the guide to sandboxing agent code execution explains.

Output: a tool register; anything absent from it is denied.

Step 6. Connect knowledge through permission-aware retrieval

Inventory the sources each agent needs and copy each document’s access rights into the index as metadata. Filter results at query time by the identity of the person the agent is serving, not by the agent’s service account, and synchronise permission changes and deletions on a schedule. Keep embeddings in the same zone as their source documents, and record the retrieval hits behind every answer. Treat retrieved text as data, never as instructions: OWASP lists agent goal hijack as ASI01 and memory and context poisoning as ASI06. The patterns for permission-aware private RAG show the implementation.

Output: a source inventory mapping each source’s access model, refresh schedule and deletion path.

Step 7. Orchestrate multi-agent workflows that fail safely

Give every step a timeout and every run a budget for iterations and tokens. Retry with backoff only where a step is idempotent, attach an idempotency key to every call with side effects, and define a compensating action for steps that cannot simply be repeated. Add circuit breakers between agents, send exhausted runs to a human queue rather than dropping them, and checkpoint state so a restart never repeats a payment or a record change. OWASP calls the failure mode this prevents cascading failures (ASI08).

Output: a failure-mode specification listing, for each step, its timeout, retry policy, idempotency key and compensation.

Step 8. Put approval gates on consequential actions

Classify each action by impact, likelihood and reversibility, as the joint guidance recommends, and require approval where the cost of error is high; its examples include “system resets, network egress or deletion of critical records”. The decision about which actions need approval belongs to the system’s designers and operators, never to the agent. Show the approver exactly what will change and why, record who approved and when, and quarantine any request to delete logs or audit records until a person has reviewed it. The guide to approval checkpoints for agents covers checkpoint patterns and approval fatigue.

Output: an action-risk table naming each action’s approver role and evidence.

Step 9. Instrument tracing and write the audit trail

Emit OpenTelemetry traces in which one run is one trace and every model call, retrieval, tool execution and approval is a span. The generative AI semantic conventions moved to their own repository in 2026 and were still at Development status in November 2026, with operation names such as invoke_agent, execute_tool and retrieval; pin the convention version you implement and expect attribute names to change. Capturing prompt and response content is opt-in, and the conventions also allow a reference to content held in external storage, so you can keep full text in the audit store and only a pointer in the trace.

Keep the audit trail separate from debugging traces. Make it append-only, and follow the joint guidance to isolate agents “into enclaves with no write access to logs”. For high-risk systems, the EU AI Act requires deployers to keep the automatically generated logs under their control for at least six months (Article 26(6)), and financial entities keep them within their financial-services documentation. The guide to the AI agent audit trail lists the fields worth keeping.

Output: a trace schema and a retention policy agreed with compliance.

Step 10. Gate releases on evaluation, then roll out in stages

Treat any change to the model, prompt, tools, agent definition or workflow as a release candidate. It must pass the evaluation suite against agreed thresholds and the red-team scenarios from the pre-production red-teaming method, and the owner signs the promotion. Then roll out through stages: shadow mode, approval of each action, and finally autonomy over a narrow set of reversible actions. Enforce the stage in platform configuration, per tool and per agent, so nobody can skip it by convention. This matches the joint guidance’s call for “phased deployment with progressively increasing access and autonomy”, and the explainer on bounded autonomy maps the stages to autonomy levels.

Output: a promotion record with evaluation results, red-team findings, stage and sign-off.

Step 11. Set capacity, service levels and the kill switch

Size capacity from concurrency, steps per run and tokens per step, and keep batch work from starving interactive agents. Define service levels for availability, run completion and the age of the approval queue, with error budgets the team will act on; the guide to SLOs for on-premises agent platforms suggests the indicators.

Build the kill switch at three scopes, one agent, one workflow and one model, and test each. The EU AI Act asks that people overseeing high-risk systems can interrupt them through a stop button or similar procedure that brings the system “to a halt in a safe state” (Article 14(4)(e)). Configure agents to stop and escalate to a person when uncertain. Write the incident runbook: contain, preserve evidence, notify under DORA or NIS2 where they apply, then recover. The playbook for containing and recovering from agent incidents is a starting template.

Output: an SLO sheet, an incident runbook and a record of the last kill-switch drill.

Step 12. Version everything and rehearse recovery

Write a release manifest for every deployment: model weights hash, serving engine version, prompt, agent definition, tool versions, workflow version, policy version and index snapshot. Rollback restores the manifest as a unit, because a new prompt rarely works with an old model. Back up the configuration database, the audit store as immutable copies, the vector indexes or the means to rebuild them, and the model weights. When financial entities restore backups on their own systems, DORA Article 12(3) requires ICT systems physically and logically segregated from the source; test the restore, not just the backup. See the guidance on versioning and upgrading local models and the disaster recovery plan for on-premises AI. The academy lesson on validating and operating a self-hosted deployment turns these checks into an operator routine.

Output: a release manifest per deployment and a dated restore-test record.

Running multi-agent workflows inside your own data centre

The same twelve steps apply when several agents share one workflow, with four additions.

Topology. Run the orchestrator and the agents in one cluster but in separate namespaces, grouped by agent or by risk tier. Network policies should let an agent reach only the orchestrator, the model gateway and the tool gateway. Where agents must call each other directly, authenticate both sides with mutual TLS, as the joint guidance recommends for all inter-agent and agent-to-service calls.

Protocols. Use MCP for tools and the Agent2Agent (A2A) protocol when agents built by different teams or on different frameworks delegate tasks to each other. A2A has a stable 1.0 specification, and in August 2026 it was accepted as a Growth Stage project of the Agentic AI Foundation, which the Linux Foundation hosts and which also looks after MCP. Keep A2A endpoints on the internal network and register them like tools.

Shared state. Keep one run-state store per workflow, isolate memory per user and per session, and validate anything one agent writes before another reads it. A poisoned entry in shared memory reaches every agent that reads it, which makes it more damaging than one wrong answer.

Contention. Several agents on one GPU pool compete for the same capacity. Queue model calls by priority, cap each workflow’s concurrency, and track cost, latency and tokens per step so you can see which agent is consuming the budget.

Illustrative architecture: a self-hosted AI agent stack for a bank

The design below is illustrative, assembled for this guide, and does not describe any customer’s deployment. It assumes a bank running agents for policy questions, KYC document checks and payment-exception triage, all inside its own data centres.

In this design, development and test use synthetic data, pre-production uses masked copies, and production runs active-passive across two data centres, each holding a copy of the model weights. Policy questions run at the advise level, KYC checks propose findings for an analyst to confirm, and payment exceptions never move money without an approver. The architecture guide in this series explains the components layer by layer, and the banking solutions page covers DORA and EU AI Act requirements for banks.

How VDF AI fits

VDF AI covers the platform layer of this checklist when you run it in your own data centre, private cloud or air-gapped network. VDF AI Networks orchestrates multi-agent workflows with retry with backoff, circuit breakers, fallback routing and SLO tracking (steps 7 and 11), a Human Approval node that pauses consequential work for an accountable person and a Verification node that checks evidence before the next stage (step 8). Networks are versioned and validated, can be saved as templates and branched, and keep an execution history with provenance and per-node cost, latency, token and energy figures (steps 11 and 12).

In VDF AI Agents, tools come from an MCP-powered registry where custom MCP servers and HTTP tools can be registered, with least-privilege access scoped per agent (step 5), and every agent keeps an audit log and execution trace from input and identity to the final output (step 9). VDF AI Router fails over across an ordered list of candidate models, and its air-gap mode routes to local models only (steps 2 and 3).

For steps 4, 8 and 11, the governance layer sets autonomy levels per agent, stores the reviewer and elapsed time with each approved run, and provides a kill switch that stops an agent, a Network or a model. The permission-check tool returns allow or deny with the policy reason, evaluated inside your perimeter and logged. Agents act within roles and per-role tool grants, with permission checks at call time; on-premises, Microsoft Entra ID single sign-on is native, and other identity providers connect through an SSO-aware reverse proxy.