Table of Contents
An industrial AI agent should ask for approval before any action that is hard to undo, leaves the company, changes the process or touches safety. Reading data and drafting documents can run without a gate. Changes to setpoints, recipes, interlocks or permits go through the site's existing change and permit procedures, approved by whoever already holds that authority.
Key takeaways
- Place approval where an action becomes hard to reverse, not on every step. Gating everything trains people to click "approve" without reading.
- CISA and partner agencies advise limiting active control of OT by AI without a human in the loop, and feeding the recommendations of advisory AI systems into the existing change management process (CISA et al., 2025).
- An approval is only as good as what the approver sees: the exact action and parameters, the raw data behind it, and what happens if it is wrong.
- Automation bias affects experts as well as novices and cannot be prevented by training or instructions alone (Parasuraman and Manzey, 2010), so approval behaviour has to be measured.
- The gate must be enforced in the tool layer and the target system, not left to the model's judgment.
What does human-in-the-loop mean for an AI agent?
For an agent, human-in-the-loop means the run pauses before a specific tool call executes. A named person reviews the proposed action and approves, edits or rejects it, and only then does the agent continue.
Agent frameworks provide this directly. LangGraph's interrupts pause a graph, save its state and wait "indefinitely until you resume execution" (LangChain docs). In the OpenAI Agents SDK, a tool can declare needs_approval, and the run stops with a list of pending calls for a person to approve or reject (OpenAI Agents SDK docs).
With a human on the loop, the agent acts alone while a person monitors it and can stop it. With a human out of the loop, nobody reviews individual actions. The choice is made per action: one agent can read freely, draft without a gate and stop for approval before it sends anything.
Accountability does not move to the software. In the words of the joint CISA guidance on AI in operational technology, "ultimately, humans are responsible for functional safety" (CISA et al., 2025).
Why does an approval gate matter more in a plant?
In an office, an agent's wrong action is usually a wrong email. In a plant it can mean a wrong spare part, an off-spec batch or an unsafe condition. The CISA guidance warns that with AI directly influencing control, "problems can escalate before operators become aware of them".
There is also a security reason. OWASP defines excessive agency as the weakness that lets an LLM system take damaging actions in response to "unexpected, ambiguous or manipulated outputs" (OWASP LLM06:2025). An agent that reads supplier documents, emails or free-text work order comments can be steered by instructions hidden in them. One of OWASP's mitigations is to "require a human to approve high-impact actions before they are taken".
The wider safeguards are covered in chatbot vs copilot vs AI agent.
Which agent actions need human approval?
Sort every tool the agent can call by what it changes and how easily that change can be undone.
| Action tier | Plant examples | Can it be undone? | Approval rule |
|---|---|---|---|
| 1. Read and analyse | Query historian exports, LIMS results and manuals; summarise a shift log | Nothing changes | No approval; log every query |
| 2. Draft in a business system | Draft work order, draft maintenance notification, draft report | Yes, the draft can be deleted | No approval to create; a person submits it |
| 3. Commit in a business system | Release a purchase requisition, close a work order, edit spare part master data | Usually, with effort | Approval per action by the person who holds that authority |
| 4. Act outside the company | Email a supplier or contractor, submit a report to an authority | No | Approval per action; preferably the person sends it |
| 5. Change the process | Setpoints, recipes, alarm limits, control logic | Physical effects cannot be undone | The agent recommends only; the change goes through management of change |
| 6. Safety and isolation | Safety instrumented functions, interlock bypasses, permits to work, isolations | No | Outside the agent's permissions entirely |
Tier 5 already has a gate in most process plants: management of change. The US process safety management rule, for example, requires written procedures for changes to "process chemicals, technology, equipment, and procedures", covering the technical basis, the impact on safety and health, changes to operating procedures and the "authorization requirements for the proposed change" (29 CFR 1910.119(l)). CISA recommends the same route for AI: for more passive systems, put the recommendations into "an existing change management process".
Some sites let automation adjust values inside a pre-approved envelope, as our earlier article on agentic AI for process automation describes. Setting that envelope is itself a change and should pass through the same review.
Tier 6 stays out of reach. The UK HSE describes permit-to-work systems as the way to control maintenance work on chemical plant and to ensure "the work is properly authorised by a responsible person" (HSE). An agent can help prepare a permit; it should never issue one.
How do you decide where a new action belongs?
For each new tool, ask these questions in order. The first "yes" sets the tier.
- Does it touch a safety function, an isolation or a permit? Then the agent does not get it.
- Does it change how the process runs? Then it is a recommendation that enters management of change.
- Does it leave the company? Then a person approves each instance.
- Does an existing procedure already require a signature? Then that signature is the approval. The agent prepares the request; it does not create a parallel route around the purchasing limit or the planner.
- Can it be fully undone within a shift, by the agent or a person? If yes, logging and periodic review may be enough.
What should the approver see?
An approval request is a small decision document. It should contain:
- The exact action as it will run: the tool name and every parameter, such as "close work order 4711 as completed, 3.5 labour hours", not "tidy up old work orders".
- The evidence, as raw data: the trend, the lab result or the manual page, linked directly. Microsoft's guidance on overreliance warns that "explanations can increase user trust even when they are incorrect" and that the mere presence of sources can raise trust in AI outputs (Microsoft Learn). The agent's summary is not the evidence.
- The consequence and the undo path: what changes, for whom, and how to reverse it.
- Three options: approve, edit or reject. LangGraph documents all three patterns, including reviewing and editing a tool call before it runs (LangChain docs).
- An expiry: a request that waits past a shift change, or whose data has changed, should lapse rather than run late.
For high-impact tiers, deliberate friction helps. Microsoft lists "cognitive forcing functions" such as friction, time to think and confirmation dialogues among the mitigations. Keep them for costly mistakes; elsewhere they only slow work down.
Why do approvals turn into rubber stamps?
People who supervise automation tend to trust it. A review of the research by Parasuraman and Manzey found that automation complacency appears under multiple-task load, in both naive and expert participants, and "cannot be overcome with simple practice". Automation bias leads to omission and commission errors when decision aids are imperfect, and "cannot be prevented by training or instructions" (Parasuraman and Manzey, 2010).
Process industries already know what happens when demands on attention outnumber the people available. In the last 11 minutes before the 1994 explosion at the Texaco Milford Haven refinery, two operators had to recognise, acknowledge and act on 275 alarms. HSE guidance, drawing on the EEMUA guide, sets a target long-term average of no more than one alarm every ten minutes in normal operation (HSE, Better alarm handling).
That target was written for alarms, but approval requests compete for the same attention. Treat them like alarms: each needs a defined response, and the number per person per shift needs a budget.
Track three numbers per action type: the share approved, the share edited and the median review time. If nearly everything is approved within seconds, either the action belongs in a lower tier or the gate has stopped working. CISA also warns that heavy reliance on AI can erode the manual skills needed when it fails, so keep people practised at doing the task without the agent.
How do you enforce approval in the software?
- Put the gate in the tool layer, not in the prompt. OWASP calls for "complete mediation": authorization is enforced by the downstream system, not decided by the LLM.
- Fail closed. In the OpenAI Agents SDK, callable approval rules "fail closed when the SDK cannot safely inspect the arguments" (OpenAI Agents SDK docs). Missing data, a timeout or an unknown tool should all mean "no".
- Persist the paused state. LangGraph interrupts need a checkpointer, and the docs say that in production it should be a persistent one, for example backed by a database. An approval may arrive hours later, after a shift change.
- Make actions idempotent. The OpenAI SDK documentation notes that "approval permits a tool call to execute; it does not guarantee exactly-once execution", and recommends a stable idempotency key for each logical action. A purchase requisition approved once must not be created twice.
- Log both identities. CISA recommends logging AI inputs and outputs with an AI identity distinct from users and machines. Record who approved, what they saw and when.
- Keep a stop switch and limits. Anthropic recommends stopping conditions such as a maximum number of iterations, and testing in sandboxed environments first (Anthropic).
On isolated networks the same rules apply, with the approval queue hosted on site. See our guide to air-gapped AI deployment and the article on integrating AI with SCADA and OT systems.
What do regulations say about human oversight?
The EU AI Act's Article 14 requires high-risk AI systems to be designed so that the people overseeing them can understand their limits, "remain aware of the possible tendency of automatically relying or over-relying on the output", override or reverse it, and interrupt the system through a stop button or similar procedure that brings it to a safe state (EU AI Act, Article 14). Algeria is not bound by it, but it is a useful reference, particularly for companies that work with European partners.
The US PSM rule quoted above is not Algerian law either; it serves here as a well-known checklist. Confirm which Algerian safety and environmental rules apply to an AI-assisted change at your site with your HSE department and legal counsel.
What we have seen at KABAS AI
Our two published projects put the human decision at the centre by design. At ADFC, a Sonatrach Group subsidiary, engineers define performance targets and constraints, Virtual Lab runs simulations and multi-criteria optimization, and the system produces a focused laboratory test playbook that engineers validate in the lab. At ENAD Shymeca, researchers test detergent formulations virtually before committing to physical trials. In both, the software proposes and people decide.
DALILE, our multi-agent assistant, runs on the client's own servers with access controls and audit logging, which is where approval records should live: next to the data, on infrastructure you control.
When scoping an agent with our agentic AI team, the practical first step is a list of every tool the agent may call, each placed in a tier from the table above and agreed with operations, maintenance and HSE before any code is written. For other reasons pilots stall before they reach production, see why industrial AI projects fail.
Sources
- CISA, ASD's ACSC and partners, Principles for the Secure Integration of Artificial Intelligence in Operational Technology (December 2025)
- OWASP GenAI Security Project, LLM06:2025 Excessive Agency
- LangChain, LangGraph documentation: Interrupts
- OpenAI Agents SDK documentation: Human-in-the-loop
- Parasuraman and Manzey, Complacency and Bias in Human Use of Automation: An Attentional Integration, Human Factors 52(3), 2010
- Microsoft Learn, Overreliance on AI: Risk Identification and Mitigation Framework
- UK HSE, Better alarm handling, Chemicals Sheet No 6
- UK HSE, Permit to work systems (COMAH technical measures)
- 29 CFR 1910.119, Process safety management of highly hazardous chemicals (Cornell LII)
- EU AI Act, Article 14: Human Oversight
- Anthropic, Building effective agents (December 2024)
Frequently asked questions
What is the difference between human-in-the-loop and human-on-the-loop?
With a human in the loop, the agent stops before a specific action and waits until a named person approves, edits or rejects it. With a human on the loop, the agent acts on its own while a person monitors it and can stop it. The same agent can use both: for example, it may read and draft freely while waiting for approval before anything is sent or committed.
Should an AI agent ever change a setpoint without approval?
As a default, no. CISA and its partner agencies advise limiting active control of OT by AI without a human in the loop, and routing AI recommendations into the existing change management process. Some sites allow bounded adjustments inside a pre-approved envelope, but defining that envelope is itself a change that should go through management of change and the site's safety review.
How do I stop approvers from approving everything without reading it?
Send fewer requests, and make each one easy to check. Move low-risk actions to logging instead of approval, show the exact action and the raw data behind it, and measure approval rates and review times. If almost every request is approved within seconds, the gate is either unnecessary or no longer working.
Does an approval step protect an agent against prompt injection?
Only partly. OWASP lists human approval of high-impact actions as one mitigation for excessive agency, alongside least privilege and authorization enforced by downstream systems. An approver may not notice a manipulated parameter, so the agent should still have narrow tools and minimal permissions.
Who should approve an AI agent's action?
The person who already holds that authority under your existing procedures: the planner for a work order, the budget holder for a purchase, the MOC approvers for a process change, the permit issuer for a permit to work. The agent should not create a new, parallel approval route, and the vendor or IT administrator should not be the approver for operational actions.



