L0 Ungoverned
Agents exist that no inventory lists. Nobody can enumerate which tools they may call.
To leave this level: Produce a complete agent inventory with a named owner for each entry.
2026 Governance & Security Handbook
Enterprise AI Agent Governance & Security Handbook
The implementation layer: what you actually build for each control, and the artefact that proves it works.
Governance fails when it stays in documents. This handbook specifies the runtime controls — identity records, permission decision order, tool contracts, approval records, retrieval enforcement, trace schemas, containment ladders — as field lists and test suites your platform and security teams can implement and your auditors can verify.
Governance is a property of the runtime, not the document library
An agent is governed when a forbidden action is denied while it is being attempted, and the denial is visible afterwards. Everything in this handbook exists to make that sentence true for your deployment. Each stage below is an enforcement point with its own evidence.
- 01 Authenticate Resolve every principal in the request
- 02 Classify Data class, purpose, risk tier
- 03 Authorize Intersect permissions; deny by default
- 04 Constrain Bound tools, context, spend, rate
- 05 Execute Act inside the approved envelope
- 06 Record Emit a reconstructable trace
- 07 Review Evaluate, escalate, reapprove
Where you are: a maturity ladder you can self-assess
L1 Documented
Policies and an inventory exist, but enforcement is by convention and code review.
To leave this level: Move at least one control from documentation into runtime enforcement.
L2 Enforced
Identity, tool permissions and data boundaries are denied at runtime, not just described.
To leave this level: Reconstruct any production run end to end from stored evidence.
L3 Evidenced
Every run produces a correlated trace; evidence exports satisfy audit without engineering help.
To leave this level: Detect a control failure from telemetry before a user reports it.
L4 Continuously assured
Controls are tested continuously, drift alerts, and approvals expire on a schedule.
To leave this level: Sustained — re-earn it at every material model, tool or policy change.
The honest test
Pick one production run from last week at random. Can you name every principal involved, every document it retrieved, every tool it called, which model version answered and who approved the result — without asking an engineer to write a query? That answer is your real maturity level.
Who owns which control
Controls without a named owner decay silently. Separate the owner who operates a control from the function that assures it — the same team should not both build and attest.
| Control | Accountable owner | Independent assurance |
|---|---|---|
| Agent inventory and ownership | Platform owner | Internal audit |
| Agent and workload identity | IAM lead | CISO |
| Tool permissions and action classes | Agent owner | Security architecture |
| Approval policy and gates | Business owner | Risk / compliance |
| Data boundaries and egress | Platform owner | CISO / DPO |
| Retrieval permissions | Data owner | DPO |
| Model inventory and routing policy | AI / model risk lead | Model risk governance |
| Logging, traces and retention | Observability owner | Internal audit |
| Human oversight design | Business owner | Risk / compliance |
| Incident response | Security operations | CISO |
Give every agent a first-class identity
An agent that cannot be named cannot be governed, contained or audited. Identity is the control everything else depends on, and it is the one most often skipped because a shared service account works on day one.
The agent identity record
| Field | Specification |
|---|---|
| Agent ID | Immutable, non-reusable identifier. Never the display name; never reassigned after retirement. |
| Owner | A named person, plus a named deputy. An owning team alone fails at 3 a.m. |
| Purpose | The approved task and the explicitly prohibited uses, written so a reviewer can judge drift. |
| Risk tier | Derived from action impact and data class — not from how visible the agent is. |
| Lifecycle state | Draft, approved, production, suspended, retired. Suspension must not delete evidence. |
| Permitted tools | Explicit allow-list by tool contract version, not a capability category. |
| Permitted data classes | Classes the agent may read and write, with the jurisdictions those classes may reach. |
| Model policy | Which approved models and routes are eligible; whether fallback is permitted. |
| Credentials | References to secret-manager entries only. No inline secrets in prompts, config or definitions. |
| Approval record | Approver, date, conditions, expiry and the superseded version. |
| Version | A content hash over the definition so any change is detectable and diffable. |
Four principals are present in every agent action
Most authorization bugs come from collapsing these into one. Resolve all four, log all four, and decide using all four.
Human initiator
The authenticated person whose intent begins the run. Absent for scheduled or event-triggered work — record the trigger instead.
Agent identity
The versioned agent definition, so behaviour can be attributed to a specific configuration.
Workload identity
The runtime service credential the platform authenticates — rotated, scoped and revocable independently.
Delegated authority
The identity whose entitlements the action consumes. This is the one most implementations get wrong.
Credential rules
- Retrieve secrets at execution time from an approved manager; never embed them in prompts, agent definitions, tool schemas or logs.
- Scope each credential to one tool and one environment, so revocation is surgical rather than service-wide.
- Rotate without redeploying the agent — if rotation requires a release, rotation will not happen at incident speed.
- Log every credential use with the correlation ID of the run that requested it, but never the value.
- Expire agent credentials on a schedule shorter than the approval expiry, so a forgotten agent stops before it drifts.
Identity anti-patterns
- One shared service account for all agents
- No action can be attributed to an agent, so containment means stopping everything.
- The agent inherits the operator's full entitlements
- Privilege becomes the union of every user who ever ran it, and it only ever grows.
- Secrets pasted into the system prompt
- They leak through traces, evaluation sets, error messages and any prompt-echo behaviour.
- Identity assigned at the application, not the agent
- Two agents with different risk tiers become indistinguishable to every downstream system.
Decide permission in a fixed order, and intersect — never union
Write the decision sequence down and implement it once at a gateway. Scattering authorization across agent code guarantees that some path will forget a step.
The permission decision order
- 01
Authenticate all four principals. If any required principal is unresolved, deny — never fall back to a service default.
- 02
Resolve the agent definition version and confirm it is in an approved lifecycle state for this environment.
- 03
Classify the request: data classes touched, purpose, jurisdiction and the risk tier of the intended action.
- 04
Compute effective permissions as the intersection of agent grant, delegated authority and workload scope.
- 05
Apply prohibitions. A deny rule always beats an allow rule, regardless of specificity or ordering.
- 06
Check the action class against the approval policy; hold for approval where required rather than proceeding optimistically.
- 07
Apply runtime bounds: rate, spend, context size, recursion depth, concurrency and tool-call count.
- 08
Emit the decision — allow, deny, hold — with the policy version and the reason, before the action runs.
Agent grant What this agent is ever allowed to do
Delegated authority What the requesting identity may do
Workload scope What this runtime is provisioned for
Effective permission The narrowest of the three, minus every explicit deny
The tool contract
A tool is not a function an agent may call; it is a contract the platform enforces around that call. Permissions bind to the contract version, so changing a schema requires re-approval rather than silently widening what the agent can do.
| Field | Specification |
|---|---|
| Tool ID and version | Permissions bind to a version. A changed schema is a new contract requiring re-approval. |
| Action class | Read, draft, write, transact or administer — the primary driver of approval requirements. |
| Reversibility | Reversible, compensable or irreversible. Irreversible actions never run unattended. |
| Destination allow-list | Exact hosts, accounts, queues or tables. Wildcards are a finding, not a configuration. |
| Parameter constraints | Types, ranges, enumerations and forbidden values, validated before execution. |
| Rate and spend limits | Per run, per agent, per hour — with a defined behaviour when the limit is reached. |
| Idempotency | The key and the retry contract, so a retry cannot double-execute a transaction. |
| Reversal procedure | The documented compensating action, its owner and its time limit. |
| Data classes | What may be sent to this tool and what may be returned into context. |
| Evidence | What the tool emits into the trace: arguments, result status, downstream reference. |
Action classes drive everything downstream
| Class | Example | Required control |
|---|---|---|
| Read | Retrieve a document, query a record, fetch a status. | Permission-filtered retrieval; log the source identifiers. |
| Draft | Compose a message, summary or proposal for a human to send. | No external effect; the human commits. |
| Write (internal) | Update a ticket, append a note, set a non-financial field. | Bounded scope, reversible, monitored; log before and after state. |
| Transact | Move money, place an order, change entitlements, send externally. | Approval gate, idempotency, spend limit, reversal procedure. |
| Administer | Change policy, grant access, alter an agent definition or model route. | Never granted to an agent in the same run it is executing. |
Hard rule
No agent receives administer-class permissions during a run in which it is also executing other work. Self-modification of policy, entitlements or its own definition must be a separate, approved, human-initiated change.
Make approval a control, not a notification
An approval that arrives after the action, or that cannot realistically be refused, is theatre with an audit trail. Define the triggers, capture what the approver saw, and measure whether the gate is still doing work.
When approval is required
| Trigger | Rule |
|---|---|
| Irreversible action | Always. No exception path that a schedule or backlog pressure can erode. |
| Above a value threshold | Set per workflow in the business owner's own units, reviewed quarterly. |
| Externally visible effect | Anything a customer, regulator or counterparty will see, before first send. |
| Cross-boundary data movement | When a class or jurisdiction boundary would be crossed by the action. |
| Low model confidence or abstention | Route to a human rather than lowering the bar to produce an answer. |
| Novel tool combination | When the run composes tools in a pattern not present in the approved plan. |
| Policy exception in force | While any compensating control is standing in for a failed gate. |
The approval record
- Approver identity
- A person, authenticated — never a shared inbox or a rubber-stamp automation.
- What was shown
- The exact payload, evidence and predicted effect presented at decision time.
- Decision and scope
- Approve, reject or approve-with-limits, plus what the approval does not cover.
- Time to decide
- Presented-at and decided-at. Sub-second approvals across a batch indicate rubber-stamping.
- Expiry
- Approvals for standing permissions expire; one-off approvals bind to a single run ID.
- Competence basis
- The role or qualification that makes this approver the right one to decide.
Approval anti-patterns
- Approval requested after the action has already executed, framed as a notification.
- A batch approval screen that presents 200 items and one Approve All button.
- The approver is the same person or service identity that configured the agent.
- Approval fatigue engineered away by lowering thresholds rather than reducing volume.
- The approval record stores the decision but not the evidence the approver actually saw.
Bind each data class to a boundary, then hunt the leak paths
A boundary is only real if every path that could cross it has been enumerated and tested. The paths below are where classified data actually escapes in production systems — rarely through the one everyone watches.
| Data class | Permitted boundary | Egress rule | Retention |
|---|---|---|---|
| Public | Any approved environment | Permitted to approved destinations | Standard |
| Internal | Enterprise-controlled compute | No third-party model or tool without a named agreement | Policy default |
| Confidential | Named tenancy or on-premises | Default deny; per-destination approval with a logged reason | Minimised, deletion verified |
| Regulated / special category | Jurisdiction-pinned, single tenant | Prohibited outside the boundary; no cross-border processing without a lawful basis | Explicit, with legal-hold handling |
| Secrets and credentials | Secret manager only | Never enters prompt, context, trace, cache or evaluation data | Rotate; never archive |
The eight leak paths to test
Prompt and system instructions
Are classified values templated into prompts that later reach a third-party model?
Retrieved context
Can retrieval pull a higher class than the run is authorised to process?
Tool arguments
Are classified fields passed to tools whose destination sits outside the boundary?
Traces and logs
Do traces store payloads rather than references, and who can read that store?
Caches and embeddings
Do derived artefacts inherit the classification and deletion duty of their source?
Evaluation sets
Has production data been copied into a test corpus with weaker controls?
Error messages
Do exceptions echo context into logs, tickets or third-party error services?
Vendor telemetry
What does the platform send home by default, and can it be disabled and proven off?
Derived data inherits
Embeddings, caches, traces, summaries and evaluation corpora inherit the classification, residency constraint and deletion duty of whatever they were derived from. Treating them as new, unclassified artefacts is how a compliant source system produces a non-compliant platform.
Make retrieval respect permissions at query time
Retrieval is where access control most often silently fails, because the index is built once and entitlements keep changing afterwards. These six rules are the difference between a search system that respects permissions and one that merely started out that way.
Enforce at query time, not ingestion time
Permissions change after documents are indexed. A trim applied only at ingestion is stale the moment access is revoked.
Filter before ranking, not after
Post-filtering leaks through result counts, scores and latency, and wastes the ranking budget on forbidden candidates.
Fail closed on permission uncertainty
If the permission service is unavailable or the claim set is incomplete, return nothing rather than everything.
Partition caches by principal
A shared answer cache silently re-serves one user's authorised result to another user who is not entitled to it.
Propagate revocation on a clock
Define the propagation objective — minutes, not "eventually" — and test it as a service level.
Cite what was used
Every material claim carries a source identifier the reader can open, which also makes leakage visible in review.
The retrieval permission test suite
Run these continuously against production configuration, not once before launch. Every one of them has a failure mode that appears only after entitlements change.
| Test | Method | Pass condition |
|---|---|---|
| Revocation propagation | Remove a user's access at source, then query for content only that document answers. | No retrieval and no cached answer within the stated propagation objective. |
| Cross-tenant isolation | Query tenant A for a distinctive string that exists only in tenant B. | Zero candidates; the attempt is logged with both tenant identifiers. |
| Deletion honouring | Delete a source, then query the index and the answer cache. | No chunk, embedding or cached answer survives beyond the deletion window. |
| Indirect injection | Plant instructions inside an indexed document that tell the agent to exfiltrate. | Instructions are treated as data; no tool call or disclosure results; the attempt is alerted. |
| Score and count leakage | Compare result metadata for authorised and unauthorised principals. | No observable difference reveals the existence of forbidden documents. |
| Aggregation exposure | Request a summary spanning many low-sensitivity chunks from one restricted source. | Aggregate remains within the source's classification and permission set. |
Treat model choice as an access-control decision
Which model answers a request determines where the data went, what terms applied to it and which jurisdiction saw it. That makes routing a governance control, not a performance tuning knob.
The approved model inventory record
| Field | Specification |
|---|---|
| Model and version | Provider, artefact revision and the exact served build — not a friendly alias that silently re-points. |
| Approved purposes | Tasks, data classes and risk tiers this model may serve. |
| Hosting and jurisdiction | Where inference physically runs and which entity operates it. |
| Data-use terms | Whether inputs, outputs or feedback may be retained or used for training, and by whom. |
| Evaluation record | Quality, safety and regression results by segment, with the test-set version. |
| Route eligibility | Which policies may select it, and whether it may serve as a fallback. |
| Owner and expiry | The accountable owner and the date approval lapses without re-evaluation. |
| Retirement plan | The successor, the migration test and the date the route is withdrawn. |
Routing rules that hold under pressure
- Routing is an access-control decision: a model the data class may not reach must be ineligible, not merely deprioritised.
- Record the route reason on every call — policy version, eligibility set and why the winner was selected.
- Pin versions for regulated workflows; a silent provider upgrade is an unapproved change to a governed system.
- Never let availability pressure route sensitive work to an out-of-boundary model. Fail closed and say so.
- Treat the system prompt as a governed artefact with the same versioning and approval as the model itself.
The alias trap
A friendly model alias that silently re-points to a new version turns an approved system into an unapproved one without any change record. Pin the served build for regulated workflows and treat a provider-side upgrade as a material change requiring regression evidence.
Emit evidence that reconstructs a decision without engineering help
The test of an evidence system is not whether logs exist. It is whether an independent reviewer can take one run identifier and rebuild exactly what happened, months later, unaided.
The execution trace schema
| Field group | Fields |
|---|---|
| Correlation | run_id · parent_run_id · session_id · trigger (user, schedule, event, agent) |
| Principals | human_initiator · agent_id + agent_version · workload_identity · delegated_authority |
| Request | purpose · data_classes · jurisdiction · risk_tier · input_reference (not payload) |
| Policy | policy_version · decision (allow/deny/hold) · reason_code · matched_rule · exceptions_in_force |
| Retrieval | query_reference · claim_set · filters_applied · candidate_ids · returned_ids · scores |
| Model | model_id + version · route_reason · parameters · token_counts · fallback_used · safety_result |
| Tools | tool_id + version · action_class · arguments_reference · destination · result_status · idempotency_key |
| Approvals | gate_id · approver · presented_at · decided_at · decision · scope · evidence_reference |
| Outcome | status · downstream_reference · error_class · abstention_reason · human_correction |
| Integrity | emitted_at · sequence · trace_hash · schema_version |
What each layer logs, and for how long
| Layer | Capture | Retain for |
|---|---|---|
| Gateway | Authentication result, classification, policy decision, applied limits, rejection reason. | Security retention period |
| Orchestration | Plan steps, tool selection, recursion depth, retries, termination reason. | Operational + audit period |
| Retrieval | Claim set, filters, candidate and returned identifiers, freshness of the index. | Audit period |
| Model | Model and version, route reason, parameters, token counts, safety verdicts. | Audit period |
| Tool | Contract version, action class, destination, result status, downstream reference. | Longest of audit and business record |
| Approval | The full approval record, including what was presented. | Business record period |
| Change | Definition, prompt, policy, model and tool version changes with approver. | Life of the system plus audit tail |
Redaction and retention rules
- Store references, not payloads, by default. A trace should point at content under its own access control rather than copy it.
- Classify the trace store itself — it is frequently the highest-value aggregation of sensitive data in the platform.
- Redact on write, not on read. A redaction applied at query time has already persisted the raw value.
- Keep the correlation skeleton even when payloads expire, so a run remains attributable after content deletion.
- Give auditors a read path that does not require engineering to run queries on their behalf.
Obligation to artefact: the audit evidence pack
| When asked to… | Produce this artefact |
|---|---|
| Show an inventory of AI systems in use | Agent register export with owner, purpose, risk tier, status and version history. |
| Demonstrate human oversight | Approval records with presented evidence, decision latency and override statistics. |
| Prove access was controlled | Policy decision log including denials, plus permission-test results with dates. |
| Reconstruct a specific decision | Full execution trace with model, retrieval, tool and approval events for the run ID. |
| Evidence data minimisation and deletion | Retention configuration, deletion verification and derived-artefact purge records. |
| Show change was managed | Release record linking evaluation, approval, deployment, observation and rollback. |
| Evidence incident handling | Incident timeline, containment actions, notification record and post-incident review. |
| Show the boundary held | Egress policy, network test results and subprocessor list with change notices. |
Design oversight that can actually change the outcome
Oversight is meaningful when a reviewer can understand, disagree, and stop the thing — in time, and with the competence the decision requires. Each test below has a matching failure you can look for in your own interface today.
| Test | You fail it when… |
|---|---|
| Can the reviewer understand? | The interface shows a recommendation but not the evidence, alternatives or confidence behind it. |
| Can the reviewer disagree? | Rejecting requires more effort than accepting, or has no route that actually stops the action. |
| Can the reviewer intervene in time? | The action executes before a human could realistically read the request. |
| Is the reviewer competent? | Oversight is assigned by availability rather than by the qualification the decision needs. |
| Is disagreement visible? | Overrides are not measured, so nobody notices the model drifting away from reviewer judgement. |
| Can the reviewer stop the class? | A reviewer can decline one item but cannot suspend the pattern producing bad items. |
Measure whether the gate is still working
| Metric | How to read it |
|---|---|
| Override rate by segment | Rising override rate is an early quality regression signal, usually before task metrics move. |
| Decision latency distribution | A collapsing distribution suggests approval has become a reflex rather than a review. |
| Post-approval correction rate | Corrections after approval indicate the presented evidence is insufficient to decide well. |
| Escalation resolution time | If escalations queue, the oversight design is nominal rather than meaningful. |
| Abstention rate | A falling abstention rate without a quality gain often means thresholds were quietly loosened. |
Contain in graduated steps, not one switch
Agent incidents have a compounding blast radius, so the useful question during the first ten minutes is which narrowest cut stops the harm. Rehearse the ladder before you need it.
Incident types specific to agents
| Type | Example | First containment move |
|---|---|---|
| Unauthorised disclosure | A user receives content their entitlements do not cover. | Suspend retrieval scope; identify all affected principals and runs. |
| Unauthorised action | A tool call executed outside the approved envelope. | Revoke the tool credential; enumerate downstream effects for reversal. |
| Prompt or content injection | Indexed or supplied content redirected agent behaviour. | Quarantine the source; re-run affected sessions against the clean corpus. |
| Boundary breach | Data reached an unapproved destination or jurisdiction. | Block egress path; preserve network evidence; start the notification assessment. |
| Model behaviour change | An upgrade or route change altered safety or quality materially. | Pin to the prior version; compare against the accepted baseline. |
| Runaway execution | Recursive loops or cascading retries consumed capacity or spend. | Apply circuit breaker; cap concurrency; identify the termination-condition defect. |
| Evidence failure | Traces missing, incomplete or unattributable for a period. | Treat as a reportable control failure; determine the blind window precisely. |
| Identity compromise | Agent or workload credential misused. | Revoke and rotate; audit every action taken under that identity. |
The containment ladder
Climb only as far as you must, and record why each step was necessary.
- 1Deny one tool for one agent — the narrowest cut that stops the harm.
- 2Suspend the agent while leaving its evidence and inventory entry intact.
- 3Revoke the workload credential to stop every instance of that runtime.
- 4Withdraw a model route and pin traffic to the prior approved version.
- 5Close the egress path at the network, independent of application state.
- 6Halt the workflow class and fall back to the rehearsed manual process.
The first hour
- 0–5 min — Declare severity, assign an incident lead, and stop or isolate the harmful action path.
- Record: Start time, reporter, symptoms, scope.
- 5–15 min — Freeze relevant changes; capture the release and configuration state; protect volatile evidence.
- Record: Agent, model, policy and tool versions in force.
- 15–30 min — Apply approved containment: revoke credentials, disable a tool, isolate a pool, shed traffic.
- Record: Decision, approver, action and observed effect.
- 30–60 min — Validate service state; notify required owners; open security, privacy and legal paths as applicable.
- Record: Impact, affected users and data, next update time.
Evidence failure is an incident
If traces are missing, incomplete or unattributable for a period, treat that as a reportable control failure in its own right. It defines a window of activity you cannot explain to a regulator, a customer or your own board.
Ask where the data is, who can compel it, and what is derived from it
Residency is a claim about storage. Sovereignty is a claim about control — including who can be legally compelled to hand over data, and whether derived artefacts stayed inside the same boundary as their source.
| Topic | The question to answer with evidence |
|---|---|
| Processing location | In which country does each inference, embedding, index and log write physically occur? |
| Operator jurisdiction | Which legal entity operates the service, and which legal regimes can compel it to disclose? |
| Support access | Can vendor staff reach production data, from where, under what approval, and is it logged for you? |
| Subprocessors | Who else touches the data, for what, and what notice and objection rights exist before a change? |
| Derived data | Do embeddings, caches, traces and evaluation sets inherit the same residency constraints as the source? |
| Cross-border transfer | What is the lawful basis for each transfer, and what happens if it is invalidated? |
| Key custody | Who holds the keys, and can the operator technically decrypt without your involvement? |
| Exit and deletion | On termination, what is exported, what is deleted, on what timeline, and how is deletion evidenced? |
Procurement link
Each of these becomes a contract schedule rather than a diligence conversation. The companion procurement guide turns them into eligibility gates, evidence requests and exit obligations.
Keep the running system identical to the approved one
Every control in this handbook degrades the moment production drifts from what was reviewed. These six controls are what keep the approval meaningful a quarter after it was granted.
- Environment separation
- Development, test and production have separate identities, data, model routes and secrets. Production data never flows backwards without an approved, minimised path.
- Definitions as code
- Agent definitions, prompts, tool contracts and policies live in version control with review, not in a console where changes are invisible.
- Promotion gates
- Nothing enters production without evaluation results, security review and a named approver recorded against the release.
- Configuration drift detection
- The running configuration is compared to the approved definition continuously; divergence raises an alert, not a quarterly finding.
- Rollback readiness
- The prior release is retained and load-tested, so rollback is a rehearsed operation rather than an improvisation.
- Restricted-network delivery
- For disconnected environments, updates arrive as signed, scanned release sets through a controlled import path with custody records.
Ninety days, in the order that actually works
Sequence matters: identity before authorization, authorization before evidence, evidence before assurance. Attempting audit-grade evidence on top of shared service accounts produces detailed records of unattributable activity.
Stop being unable to answer basic questions
- Build the agent inventory: every agent, its owner, purpose, tools and data classes.
- Give each agent a distinct identity and remove shared service accounts.
- Move secrets out of prompts and definitions into a secret manager with references.
- Turn on correlated tracing with the correlation, principal and policy field groups.
Move controls from documents into runtime
- Write tool contracts with action class, destination allow-list and parameter constraints.
- Implement the permission decision order at a gateway, including deny-by-default and the intersection rule.
- Enforce retrieval permissions at query time and partition answer caches by principal.
- Define approval triggers per workflow and capture the full approval record.
Make the evidence self-serve and test the failure paths
- Produce the obligation-to-artefact evidence pack and have audit retrieve a run unaided.
- Run the private-RAG test suite, including revocation propagation and indirect injection.
- Rehearse the containment ladder and the first-hour incident timeline as a game day.
- Set approval and model-inventory expiries so governance re-earns itself on a schedule.
Free download
Take the specifications to your security review
The PDF edition collects every record schema, decision order, test suite and timeline on this page into one document your platform, security and audit teams can work from directly — and one you can attach to an architecture review or a regulator response.
- Agent identity, tool contract, approval, model inventory and trace record schemas
- The permission decision order and the effective-permission intersection rule
- The eight data leak paths and the six-test retrieval permission suite
- Incident taxonomy, containment ladder and the first-hour timeline
- The obligation-to-artefact audit evidence map and the 90-day sequence
AI agent governance and security — direct answers
What is AI agent governance in practice, as opposed to on paper?
In practice it is a runtime enforcement path: authenticate every principal, classify the request, compute effective permissions as an intersection, apply prohibitions, gate the action class, bound rate and spend, execute, then emit a trace that lets an independent reviewer reconstruct what happened. A policy document that no gateway enforces is not governance — it is an intention. The practical test is whether a forbidden action is denied at runtime and the denial is visible in a log.
How should an AI agent be given an identity?
Every agent needs its own non-human identity with an immutable ID, a named owner and deputy, an approved purpose with prohibited uses, a risk tier derived from action impact and data class, a lifecycle state, an explicit tool allow-list bound to contract versions, a model policy, credential references rather than credentials, an approval record with expiry, and a content hash over the definition so any change is detectable. Shared service accounts across agents are the single most common failure: they make attribution impossible, so containment means stopping everything.
How does RBAC work for AI agents?
Role-based access control for agents has to account for four principals in every action: the human initiator, the agent identity, the workload identity and the delegated authority whose entitlements are actually being spent. Effective permission is the intersection of the agent grant, the delegated authority and the workload scope — never the union — and an explicit deny always beats an allow regardless of specificity. Letting an agent inherit an operator's full entitlements is the classic error, because privilege then accumulates from every user who ever ran it.
When should an AI agent action require human approval?
Always for irreversible actions; above a per-workflow value threshold; for any externally visible effect; when data would cross a classification or jurisdiction boundary; when model confidence is low or the system abstains; when the run composes tools in a pattern outside the approved plan; and while any policy exception is in force. The approval record must capture the approver, exactly what was presented, the decision and its scope, presented-at and decided-at times, an expiry, and the competence basis for that approver.
How do you enforce permissions in private RAG?
Enforce at query time rather than ingestion time, because permissions change after documents are indexed. Filter before ranking rather than after, so result counts, scores and latency cannot leak the existence of forbidden documents. Fail closed when the permission service is unavailable or claims are incomplete. Partition answer caches by principal, or one user's authorised result will be re-served to another. Define revocation propagation as a service level in minutes and test it continuously — alongside cross-tenant isolation, deletion honouring, indirect injection and aggregation exposure.
What belongs in an AI agent execution trace?
Ten field groups: correlation (run and parent IDs, session, trigger); principals (human initiator, agent ID and version, workload identity, delegated authority); request (purpose, data classes, jurisdiction, risk tier, input reference); policy (version, decision, reason code, matched rule, exceptions); retrieval (claim set, filters, candidate and returned IDs, scores); model (ID and version, route reason, parameters, token counts, fallback, safety result); tools (contract version, action class, destination, result status, idempotency key); approvals (gate, approver, presented and decided times, scope); outcome (status, downstream reference, error class, abstention, human correction); and integrity (emitted-at, sequence, trace hash, schema version). Store references rather than payloads by default.
What counts as meaningful human oversight?
Six tests: the reviewer can understand the recommendation including its evidence and alternatives; can disagree without rejection being harder than acceptance; can intervene before the action executes; is competent for the specific decision rather than merely available; disagreement is measured so drift is visible; and the reviewer can suspend the whole pattern, not just decline one item. Watch override rate, decision-latency distribution, post-approval correction rate, escalation resolution time and abstention rate — a collapsing latency distribution usually means approval has become a reflex.
How is an AI agent incident different from a normal security incident?
The action surface is generative and the blast radius compounds, so containment is graduated rather than binary: deny one tool for one agent, suspend the agent, revoke the workload credential, withdraw a model route, close the egress path, then halt the workflow class. Distinct incident types include prompt and content injection, model behaviour change after an upgrade, runaway recursive execution, and evidence failure — where traces are missing or unattributable, which is itself a reportable control failure because it defines a window you cannot explain.
Standards and regulatory references
This handbook is written to be implementable independently of any vendor. These references anchor its control vocabulary; validate the current version before citing any of them in an assessment.
- NIST AI Risk Management Framework
- NIST SP 800-207 — Zero Trust Architecture
- NIST SP 800-53 Rev. 5 — Security and Privacy Controls
- OWASP Top 10 for Large Language Model Applications
- MITRE ATLAS — adversarial threat landscape for AI systems
- ISO/IEC 42001:2023 — AI management systems
- Regulation (EU) 2024/1689 — Artificial Intelligence Act
- Regulation (EU) 2016/679 — General Data Protection Regulation
This is engineering and governance guidance, not legal advice. Confirm obligations, role classification and sector requirements with qualified counsel and your control owners.
Use these alongside the handbook
Control review
Walk your agent controls with our security team
Bring one production workflow and we will trace it against this handbook end to end — principals, tool contracts, retrieval permissions, approval records and whether the run can be reconstructed from evidence alone. You keep the findings whether or not you use our platform.