Human-in-the-Loop AI Agents: Approval Workflows, Checkpoint Patterns and Where to Put the Human

Human-in-the-loop AI agents pause before consequential actions so that an accountable person can approve, edit, reject or return the proposed step, and the decision is recorded with the run. The design work is choosing where that person sits: gate actions that are high-impact, hard to reverse, outside policy or low in confidence; show the approver the exact change and its evidence; make unanswered requests fail closed; and watch rejection rates and review times for signs of rubber-stamping. Everything else can run on its own within limits, with sampling after the fact.

Getting this wrong in either direction is costly. Too few checkpoints and an agent sends, pays or deletes something nobody meant it to. Too many and reviewers stop reading. In its 26 May 2026 guidance, Gartner cautions that without strong security testing, clear approval workflows with audit trails and agent-specific incident response, “approvals can degrade under time pressure or approval fatigue”. For the definitions, start with the explainer on human-in-the-loop AI; this guide is about placement and design.

In the loop, on the loop and out of the loop

The three terms are older than AI agents. Human Rights Watch’s 2012 report Losing Humanity used them to sort autonomous weapons by whether a person commands each action, supervises with the power to override, or is absent. For agents they describe who decides at the moment of action.

One agent usually operates in all three modes at once, one per action: it may read files unsupervised, draft a reply under watch and wait for approval before sending. Gartner’s four autonomy levels of May 2026 (observe, advise, act with approval, act autonomously) describe the same ladder from the agent’s side, and the concept page on bounded autonomy explains how to set the limits that make on-the-loop operation safe.

Seven checkpoint patterns for AI agents

1. Pre-action approval gate

The agent proposes an action and the run stops until a person decides. Use it where an action is hard to reverse or carries legal, financial or customer impact. The gate must hold the call outside the model, so a manipulated agent cannot skip it.

2. Review and edit

The agent drafts and a person edits before anything leaves: a reply, a contract clause, a change plan. This suits work where the draft saves time but judgement shapes the result, and editing should be a first-class outcome, not a rejection followed by a retry.

3. Escalation on low confidence or out-of-policy cases

The agent handles the routine and hands over what it cannot resolve: ambiguous inputs, missing data, amounts above a limit or requests outside its scope. Escalation rules should be explicit conditions evaluated by the runtime, not a hope that the model will ask.

4. After-the-fact sampling

For high-volume, reversible actions, nobody approves in advance; reviewers examine a random sample plus every flagged case. Raise the sampling rate when errors appear and lower it as the record holds.

5. Four-eyes dual control

Two separate people must confirm before the action runs, the four-eyes principle that banks apply when releasing payments. The EU AI Act writes a version of it into Article 14(5): for remote biometric identification systems, no action may be taken on an identification unless at least two competent people have verified it separately. For agents, reserve it for the highest-impact actions, and ensure the two approvers really are independent of each other and of whoever requested the action.

6. Clarification requests

Instead of guessing, the agent asks the requester for the missing fact and waits. This is not approval but it is a checkpoint, and it prevents a whole class of confident mistakes. VDF AI exposes it as a clarification request tool.

7. Interruptible execution and the kill switch

Long runs need a way to pause, redirect or stop mid-flight, and a kill switch that halts an agent everywhere it runs at once. For high-risk systems, Article 14(4)(e) of the EU AI Act asks for the same thing: a way for a person to interrupt the system, through a stop button or a comparable procedure, so that it comes to rest in a safe state.

Where to put the human: a decision matrix

Two questions place most actions: how much harm can a wrong action do, and how easily can it be undone?

Overlay three triggers that override the cell: low confidence, an out-of-policy request and atypical behaviour, such as an agent reaching for a system outside its usual scope, should always escalate. Singapore’s Model AI Governance Framework for Agentic AI, version 1.5 of 20 May 2026, names the same checkpoint types: high-stakes actions, irreversible ones, outlier behaviour and thresholds that users set for themselves, such as purchases above an amount.

Start strict, then relax with evidence

A new agent should begin with more checkpoints than it will finally need. Run it in shadow mode, or with approval on every consequential action, and record what reviewers do: how often they approve unchanged, edit or reject, and why. When an action class shows a long run of unchanged approvals across varied cases, move it one cell down the matrix, for example from an approval gate to sampling, and write down the evidence and the person who agreed. Move it back up after an incident, a model change or a new tool. Autonomy granted this way can be defended to an auditor, because every step has a record behind it; the guide to agentic AI risk management places the practice inside a wider risk framework.

Designing the approval request

A reviewer can only be as good as the request in front of them. Each request should show:

• The proposed action in one plain sentence, naming the system and the target.
• The exact change, as a diff: the record before and after, or the full message to be sent.
• The evidence, meaning the documents, tool outputs and data the agent used, linked so they can be checked. The agent’s stated reasoning is a summary, not proof; IMDA’s framework warns that chain-of-thought may not faithfully explain what an agent did.
• The policy or rule that triggered the gate, and the risk class of the action.
• The impact: amount, people or systems affected, and whether the action can be reversed.
• The actors: which agent is asking, and for which person or process.
• The deadline, and what happens if it passes.

Offer four outcomes: approve, edit, reject, and return with a comment, and record which was chosen. Research on automation bias points to the design choices that help. A 2012 systematic review of 74 studies by Goddard, Roudsari and Wyatt identified training, holding users accountable, where advice appears on screen and offering information instead of a recommendation as ways to reduce over-reliance. Buçinca and colleagues showed in 2021, with 199 participants, that “cognitive forcing” designs, which make people engage before seeing or accepting the AI’s suggestion, cut over-reliance more than simple explanations did, although participants rated those designs least favourably. A practical form for high-risk approvals is to require a short written reason, as IMDA also suggests.

Where the alert appears matters less than where the decision is taken. Reviewers can be notified wherever they work, but the decision itself belongs in an authenticated application that shows the full request and writes the outcome to the run, never in a free-text reply that anyone holding a forwarded message could send. For dual-control actions, both reviewers should see the same evidence and decide separately. The VDF approval lesson in the Academy shows a working gate, and the contract review agents walkthrough shows approval routing in one business process.

Approval fatigue and automation bias

Every approval request spends a reviewer’s attention, and the supply runs out. Clinical decision support offers a long record: a 2006 review in JAMIA by van der Sijs and colleagues, covering 17 studies, found that clinicians overrode drug safety alerts in 49% to 96% of cases. Agents that ask for approval on everything invite the same response.

Fatigue is also an attack surface. OWASP’s Top 10 for Agentic Applications for 2026 lists human-agent trust exploitation as ASI09: a manipulated agent can present a harmful action persuasively and rely on the reviewer to wave it through, one of several threats covered in the guide to agentic AI security. Four measures keep approvals meaningful:

1. Gate less, but better. Reserve approval for the cells of the matrix that need it, and move everything else to sampling.
2. Measure the reviewers. IMDA’s version 1.5 recommends tracking how often people reject or modify agent actions and how long their reviews take; a low override rate can signal rubber-stamping, and very short review times can signal fatigue or automation bias.
3. Test the gate. Seed occasional requests that should be rejected and check that they are.
4. Share the load. Route requests to trained reviewers with the domain knowledge to judge them, cap queue sizes and escalate when a queue backs up.

Article 14(4)(b) of the EU AI Act names automation bias explicitly as something people overseeing high-risk systems must be able to stay aware of, which makes these measures part of compliance design, not just usability.

Timeouts, approver identity and the approval record

Timeouts must fail closed. An unanswered request is a rejection: the action does not run, the requester is told and the request can be escalated to a deputy before the deadline. Never let silence count as consent. IMDA’s framework makes the same point more broadly, recommending that actions be denied by default when approval infrastructure fails, for example when supervisors cannot be reached.

Approvers must be real, authorised people. Each approver signs in through your directory, holds an approval right for that class of action, and is never the agent itself. Segregation of duties applies as it does elsewhere: the person who asked for a high-risk action should not be its only approver. Framework documentation is blunt about the risks. The OpenAI Agents SDK notes that restoring a saved run does not authenticate the reviewer, and advises authorising every reviewer and consuming each pending request once so it cannot be replayed. Microsoft Agent Framework binds an approval to the pending request in the same session by default and ignores unbound responses. Permission design for the agent itself is covered in the guide to AI agent permissions.

Approved must equal executed. Record a fingerprint of the exact arguments approved and refuse to run anything different; Microsoft’s framework, for one, fails closed when middleware changes approved arguments that cannot be revalidated. Then log the request, what the reviewer was shown, the decision, the comment, the reviewer’s identity, the time taken and the run it belongs to. That record is part of the agent audit trail, and it is what an auditor will ask for first. For self-hosted estates, the on-premises governance checklist lists the approval and audit evidence to keep.

Human checkpoints in multi-agent workflows

In a governed multi-agent workflow, approval at every step would stall the work, so place checkpoints at boundaries: where a planner’s plan becomes instructions for other agents, where results from several agents are merged into one decision, where the workflow touches money, customers or production, and before anything leaves the organisation. Put an automated verification step in front of each human checkpoint so evidence and conditions are checked by software first, and decide in advance whether a rejection halts the workflow or sends the work back to a named agent with the reviewer’s comments.

The workflow must also survive the wait. Approvals can take hours or days, so state has to persist outside memory. Microsoft Agent Framework’s workflow engine, for example, saves pending human requests inside each checkpoint and re-issues them when a run is restored.

How frameworks implement human-in-the-loop

Each of the three open-source frameworks below can stop a run and wait for a person, according to its documentation in November 2026. They differ in where the pause is declared and how state survives it.

All three leave the governance around the pause to you. That is manageable for one team and hard to keep consistent across an estate, which is where a platform with approval built into the runtime earns its place.

The EU AI Act connection

Article 14 of the AI Act requires high-risk AI systems to be designed for effective human oversight, including the abilities to override or reverse outputs and to stop the system; for systems listed in Annex III, those obligations start on 2 December 2027, the date set by the Digital Omnibus. The human oversight requirements of Article 14 are covered in detail elsewhere, and the overview of how the AI Act treats agents explains when an agentic system becomes high-risk in the first place.

How VDF AI fits

In VDF AI Networks, a Human Approval node holds consequential work until an accountable person approves it, rejects it or returns it for changes, and a Verification node tests evidence and conditions before letting the next stage run; placing one ahead of an approval means reviewers only see work that has passed those checks. Inside an agent, the human approval request tool blocks the run until a person decides and logs the approver’s identity, while the clarification tool lets an agent ask instead of guessing.

Autonomy levels set how far each agent may go unaided, and each approval is saved with its run, including the reviewer, the outcome of the review and how long it took. On an on-premises installation, approvers authenticate through Microsoft Entra ID, which is supported natively with its security groups mapped to roles, while Okta, Keycloak and generic SAML or OIDC providers sit behind an SSO-aware reverse proxy. Deployment options range from VDF AI Cloud to your own servers, a private or sovereign cloud and isolated air-gapped sites, and a kill switch can stop an agent, a Network or a model. The Academy lesson linked above walks through building a gate step by step.