Every agent platform advertises "human-in-the-loop". Few explain where the human comes in, and the difference between a good design and a bad one is the difference between an agent that operates and an agent that generates a queue for somebody to approve. This text covers the where and the how.
It is worth starting with what it is not. Human supervision is not a safety net for an immature agent, to be removed once it "gets good". It is a permanent part of the design, for the same reason a manager approves a discount above a certain amount: not because the salesperson is bad, but because the decision has consequences that call for another level of authority.
Where to place it: the four criteria
The question is not "does this agent need supervision", it is "does this action need it". Action by action, four criteria decide.
1. Irreversibility
Looking up an order is reversible: if it got it wrong, look it up again. Cancelling the order is not. Sending a message to the customer is not. Issuing an invoice is not. The harder it is to undo, the stronger the case for a person approving first.
2. Value
A 2% discount and a 30% discount are the same action with different consequences. A limit by value is the simplest and most effective rule: the agent decides on its own up to a ceiling, and above it waits for approval. The ceiling is a business decision, not an engineering one, and it can differ by team.
3. Sensitive data
An action involving health, financial or minors' data calls for human authority regardless of value, because the error has regulatory consequences on top of the operational ones.
4. Legal requirement
A decision that affects the data subject and is taken solely by automated processing gives them the right to request a human review (art. 20 of Brazil's LGPD). Approving credit, denying coverage, scoring risk: in those cases a review path is not a design option, it is an obligation. The post on governance and the LGPD covers it.
Most of an agent's actions pass none of the four criteria, and that is why it exists. Supervision that applies to everything is not care: it is the old queue, with an agent in front of it.
How to place it: the three modes
| Mode | How it works | When to use it |
|---|---|---|
| Approve first | The agent prepares the action and stops. A person approves, adjusts or refuses. Only then does the action happen. | Irreversible, above the value ceiling, sensitive data. It is the safest mode and the most expensive in time. |
| Review afterwards | The agent acts. A sample of the actions is reviewed by a person, within a deadline. An error found becomes an adjustment. | Low-risk, high-volume actions. It is what keeps quality up without creating a queue. |
| Hand over | The agent recognizes the case is not its own and passes the whole conversation, with context, to a person. | Out of scope, low confidence, an explicit request from the customer, a sign of risk. |
The three coexist in the same agent. A mature operation has few actions in "approve first", most in "review afterwards", and a clear handoff criterion. The typical mistake is having everything in the first mode, out of fear, or nothing in any of them, out of haste.
How the person does not become the bottleneck
The risk of "approve first" is obvious: the person becomes a queue. An agent that generates three hundred approval requests a day has automated nothing, it has only changed who does the typing. Four designs prevent that:
- A ceiling by value, not universal approval. If 90% of the actions fall below the ceiling, 90% generate no queue. The ceiling starts low and rises as the agent's accuracy rate is proven.
- Batch approval. The person sees ten prepared actions at once, with the context of each, and approves together what is right. One screen, not ten notifications.
- Escalation by time. If nobody approved within a given period, the action goes to another level of authority or is handed over, instead of sitting there until somebody remembers.
- The approval request carries the finished decision. The person does not investigate; they confirm. The agent delivers what it looked up, what it proposes and why. An approval that requires the person to redo the work is not supervision, it is rework.
A handoff done well
The third mode is the most visible to the customer and the most frequently done badly. A good handoff has three properties: it happens early (the agent recognizes the limit on the first message that reveals it, not the fourth), it goes to the right queue (collections to collections, not to a general queue), and it carries the full context (what the customer said, what the agent looked up, what has already been answered). The customer repeats nothing, and the person picks up where the agent stopped.
The sign that the handoff is badly designed is measurable: count how many times the customer asks for a person before getting one. If the average is above one, the stopping criterion is wrong.
What to measure
- Actions approved without changes. If the person approves 98% of what the agent proposes without changing anything, the ceiling can rise.
- Actions refused or changed, by reason. That is where the learning is: every refusal is a case for the test set.
- Time to approval. If it rises, a queue is forming.
- Errors found in the sample review. A rate that rises is the early sign that the process changed.
- Requests for a human before the handoff. The metric of a well-designed handoff.
How Runflow implements it
On the platform, human approval is configured by type of action, not as a global switch: you approve before the agent acts where the risk demands it, and the rest flows. In the Conversation Hub, the handoff to a person happens in the same conversation, to the right department's queue, with the full history, and the conversation can return to the agent afterwards. Every approval, refusal and handoff stays in the execution trail, which is what makes it possible to measure the items above and adjust the ceiling on evidence.
