Human-in-the-Loop Approval Gates for AI Agents: A Practical Design Guide

Every team shipping an agent that can take real actions eventually writes the same line in a design doc: "the agent will ask for human approval before risky actions." Then the implementation goes one of two ways.

In the first, the agent asks about everything. Reading a file, running ls, searching the CRM. Within a week users approve without reading, which is worse than no gate at all because it looks like oversight on paper.

In the second, the gate is an input("Proceed? [y/n]") buried inside a tool function. It works in the demo. In production the agent runs in a worker, the approval arrives four hours later from a Slack button, the worker has since been redeployed, and the pending action is gone. Or worse, it gets executed twice.

Approval gates are a product problem and a systems problem at the same time. This guide covers both: deciding what needs approval, and building the pause/resume machinery so a human decision can arrive minutes or days later without breaking anything.

Why approval gates fail

There are three distinct failure modes, and each needs a different fix:

Failure mode	Symptom	Root cause
Approval fatigue	Approve rate near 100%, median review time under 2 seconds	Gating on tools instead of on consequences
Uninformed approval	Humans approve things they would have rejected with full context	The approval request shows the tool call, not its effect
Lost or duplicated actions	Approved action never runs, or runs twice	Pause state lives in process memory; execution isn't idempotent

Most write-ups focus on the first. In practice, the second and third cause the incidents.

Step 1: Gate on consequences, not on tools

The unit of approval should be the effect of an action, not the tool name. send_email to your own address is harmless; send_email to 4,000 customers is not. run_sql with a SELECT is fine; run_sql with DELETE without a WHERE clause is not.

A useful classification has four tiers, scored on two axes: reversibility and blast radius (who or what is affected outside the agent's sandbox).

Tier	Examples	Policy
0 – Read-only	Search docs, read files, query with `SELECT`	Always auto-approve
1 – Reversible, internal	Create a draft, write to a scratch branch, add a label	Auto-approve, log for audit
2 – Reversible, external or costly	Open a PR, post in an internal channel, spend above a budget threshold	Approve once per session or per scope
3 – Irreversible or external	Send email to customers, merge to main, issue a refund, delete data	Always require explicit approval per action

The key design choice is that the policy engine inspects arguments, not just tool names. Here's a minimal Python version:

from dataclasses import dataclass
from enum import IntEnum
import re

class Tier(IntEnum):
    READ = 0
    REVERSIBLE_INTERNAL = 1
    REVERSIBLE_EXTERNAL = 2
    IRREVERSIBLE = 3

@dataclass
class ToolCall:
    name: str
    args: dict

INTERNAL_DOMAIN = "@acme.com"
DESTRUCTIVE_SQL = re.compile(r"^\s*(delete|drop|truncate|update|alter)\b", re.I)

def classify(call: ToolCall) -> Tier:
    if call.name == "run_sql":
        return Tier.IRREVERSIBLE if DESTRUCTIVE_SQL.match(call.args["query"]) else Tier.READ

    if call.name == "send_email":
        recipients = call.args["to"]
        if all(r.endswith(INTERNAL_DOMAIN) for r in recipients) and len(recipients) <= 5:
            return Tier.REVERSIBLE_EXTERNAL
        return Tier.IRREVERSIBLE

    if call.name == "issue_refund":
        return Tier.IRREVERSIBLE if call.args["amount_inr"] > 5_000 else Tier.REVERSIBLE_EXTERNAL

    if call.name in {"search_docs", "read_file", "get_ticket"}:
        return Tier.READ

    # Unknown tools fail closed.
    return Tier.IRREVERSIBLE

Two details matter here. First, unknown tools default to the highest tier. When someone adds a new MCP server next month, it should be gated until someone classifies it. Second, the classifier is plain code, not an LLM call. You can run an LLM-based risk scorer alongside it for ambiguous cases, but the baseline should be deterministic and unit-testable. An attacker who controls the agent's input through prompt injection shouldn't be able to talk their way to a lower tier.

Step 2: Make the pause durable

The agent's decision to call a tool and a human's decision to allow it happen at different times, often in different processes. Treat the pending approval as a persisted record, not a blocked coroutine.

The flow looks like this:

The agent proposes a tool call.
The policy engine classifies it. Tier 0–1 executes immediately.
For tier 2–3, write a PendingAction row with the full call, the agent's conversation state (or a checkpoint ID), and an idempotency key. Then end the agent's turn.
Notify a human through whatever channel fits: Slack, an in-app inbox, email.
When the decision arrives, load the checkpoint, execute (or reject), and feed the result back to the agent as the tool result.

import hashlib, json, uuid
from datetime import datetime, timedelta, timezone

def idempotency_key(run_id: str, call: ToolCall) -> str:
    payload = json.dumps({"run": run_id, "name": call.name, "args": call.args}, sort_keys=True)
    return hashlib.sha256(payload.encode()).hexdigest()

def propose(run_id: str, checkpoint_id: str, call: ToolCall, db) -> dict:
    tier = classify(call)
    if tier <= Tier.REVERSIBLE_INTERNAL:
        return {"status": "executed", "result": execute(call)}

    action_id = str(uuid.uuid4())
    db.insert("pending_actions", {
        "id": action_id,
        "run_id": run_id,
        "checkpoint_id": checkpoint_id,
        "tool": call.name,
        "args": call.args,
        "tier": int(tier),
        "idem_key": idempotency_key(run_id, call),
        "status": "pending",
        "expires_at": datetime.now(timezone.utc) + timedelta(hours=24),
    })
    notify_reviewer(action_id)
    return {"status": "awaiting_approval", "action_id": action_id}

def decide(action_id: str, approved: bool, reviewer: str, db, note: str = ""):
    # Atomic transition: only one decision wins, even if two reviewers click at once.
    row = db.update_returning(
        "pending_actions",
        where={"id": action_id, "status": "pending"},
        set={"status": "approved" if approved else "rejected",
             "reviewer": reviewer, "note": note,
             "decided_at": datetime.now(timezone.utc)},
    )
    if row is None:
        return  # already decided or expired

    call = ToolCall(row["tool"], row["args"])
    if approved:
        result = execute(call, idempotency_key=row["idem_key"])
    else:
        result = {"error": "rejected_by_human", "reason": note or "No reason given"}

    resume_agent(row["run_id"], row["checkpoint_id"], tool_result=result)

The db, execute, notify_reviewer, and resume_agent functions are placeholders for your own stack. What matters is the shape:

The status transition is atomic (WHERE status = 'pending'), so a double-click or two reviewers can't execute twice.
The idempotency key travels to the downstream API. Payment and email providers commonly accept one; if yours doesn't, keep an executed-keys table and check it before calling.
Pending actions expire. An approval that arrives three days late for a refund against a since-cancelled order should not execute. Expired actions resolve as rejected with a clear reason.
Rejections go back to the agent as tool results. The model gets to adapt ("the reviewer rejected sending to all customers; I'll draft it for the internal list instead") rather than crashing.

If you use a framework with checkpointing, you get much of this for free. LangGraph, for example, provides an interrupt() primitive that persists graph state and resumes with a Command(resume=...) value; durable-execution engines like Temporal model this as a workflow waiting on a signal. Check the current docs for exact signatures, since these APIs have changed across versions. The design questions above still apply regardless of framework: what's gated, what's shown, and what happens on expiry.

Step 3: Show effects, not function calls

Here's an approval request that gets rubber-stamped:

send_email(to=["list:all-customers"], template_id="t_8812", vars={...}) — Approve / Reject

Here's one that gets read:

Send email to 4,112 customers (segment: all-customers) Subject: "Your plan price is changing on 1 November" Preview: first 3 lines of rendered body ⚠️ Irreversible. Last sent to this segment: 11 days ago. Agent's reason: "User asked to notify affected customers about pricing change" (from ticket #4431) Approve · Approve for internal list only · Reject with note

The difference is a renderer per tool that turns arguments into consequences: resolved recipient counts, rendered previews, diffs for code or config changes, rupee amounts, and the rows that a DELETE would affect (by running the equivalent SELECT COUNT(*) first). This is where most of the engineering effort should go, and it's the part PMs should own the spec for.

Also notice the third button. Offer edits and narrower alternatives, not just yes/no. When the only options are approve or reject, reviewers approve borderline actions to avoid blocking work. When they can approve a narrower version, they do that instead.

Step 4: Measure whether the gate is working

A gate that nobody measures drifts toward either fatigue or irrelevance. Log every decision and track:

Metric	Healthy signal	Warning sign
Approvals requested per session	A handful at most	Dozens, meaning you're gating low-risk actions
Approve rate by tool	70–95%	~100% means a pointless gate; very low means the agent is poorly prompted
Median time-to-decision	Seconds to minutes for tier 2	Under ~2s at tier 3 suggests nobody is reading
Edit/narrow rate	Non-zero	Zero may mean reviewers lack alternatives
Expired approvals	Rare	Frequent means the notification channel is wrong

The approve-rate-by-tool metric is also your path to graduated autonomy. If a tier-2 action has been approved 500 times in a row with no edits, that's evidence for auto-approving it with audit logging. Go the other way too: if a tool's rejection rate spikes after a model upgrade, tighten the policy before you debug the prompt.

Common pitfalls

Gating inside the tool function. Blocking on user input deep inside a tool couples the approval to a live process. Gate in the orchestration layer, before execution, and persist state.
Letting the model decide what needs approval. A requires_approval: bool argument that the model fills in will be set to false by a successful prompt injection. Keep the classifier outside the model's control.
Approving a plan, then executing something else. If a human approves "update 3 config files", verify the executed calls match the approved ones. Hash the approved arguments and compare them at execution time.
Session-wide approvals that never end. "Allow for this session" is convenient; make sure sessions have a time limit and the scope is specific (this repo, this recipient list), not "all emails".
No audit trail. For regulated domains, you'll need to answer who approved what, when, and what they saw. Store the rendered approval request, not just the raw arguments.
Forgetting the rejection path in evals. Test that the agent behaves sensibly when rejected: it shouldn't retry the identical call, and it shouldn't try to reach the same effect through a different tool.

Launch checklist

Next steps

Start with an inventory: list every tool your agent can call and assign a tier, arguing about the borderline ones as a team. That conversation alone usually surfaces two or three tools that should never have been ungated. Then move your pause state out of process memory, even if the first version is a single Postgres table. Finally, pick your highest-volume gated action and build a proper effect renderer for it. It's the change reviewers will notice most.

Approval gates aren't a tax on autonomy. They're how you earn it: each well-logged approval is evidence for letting the agent do a bit more on its own next quarter.

If you're building agents that take real actions and want to go deeper on orchestration, evals, and production safety, the Joinloop AI engineering cohort covers these patterns in hands-on projects. If you're the PM deciding where the agent should and shouldn't act alone, the PM cohort covers exactly that trade-off.