Governing Agentic AI: Controls for Systems That Act
The first time I watched an agent open a ticket, comment on it, reassign it, and close it without anyone in the loop, I felt the same small chill I used to feel watching a self-driving forklift in a warehouse. Not fear of the machine. Fear of the silence around it. Nobody was looking, because nothing had broken.
That is the new shape of the risk. An assistant that gives you a wrong answer costs you a wasted minute and a re-prompt. An agent that takes a wrong action writes that action into a downstream system, and the wrongness lives there, accruing interest, until somebody else trips over it. The risk arithmetic changes the moment the model stops talking and starts doing.
What you will learn
- Why agentic AI changes the risk model, not just the toolchain
- The five controls every agent in production needs (and why none are new)
- Where the EU AI Act, NIST, ISO 42001, and OWASP each meet the agentic question
- How to walk one obligation, human oversight, all the way to a working checkpoint
Why agentic AI is the fastest-moving control problem in 2026
In the professional conversation I track, agentic AI sits at the intersection of close to twenty distinct domains: healthcare, legal, financial services, government, K-12 education, customer operations, IT operations, security operations, software engineering. That breadth is the tell. Agentic AI is not a vertical topic; it is a horizontal capability that crosses every line of business at once. Compliance and risk teams who built their AI playbook around model-as-tool are discovering it does not survive contact with model-as-actor.
The simplest way I have found to explain the shift is the answer-versus-act distinction. A model that answers exists inside a conversation; the human is the gating function on whether anything happens. A model that acts exists inside a workflow; the human is, at best, a reviewer of what already happened. Move the model from the first column to the second and you change what failure looks like, when you find out, and who has to clean it up.
| Answer mode | Act mode |
|---|---|
| Model output is text, advice, a draft | Model output is an API call, a transaction, a state change |
| Human in the loop by default (they decide whether to use the output) | Human out of the loop by default (the action happens, then we look) |
| Failure surfaces in the next prompt | Failure surfaces in a downstream system, possibly hours later |
| Cost of error: a wasted minute | Cost of error: a ticket, a refund, a notice, a liability event |
| Audit trail is “the chat” | Audit trail must be designed, logged, queryable |
The deeper sibling pillar that explores the building side of agents lives elsewhere; this hub is focused tightly on the governance side. If you want the build view, see the standalone Agentic AI pillar in the roadmap. Here we stay on the controls.
The five controls every agent in production needs
When you put what NIST publishes in the Generative AI Profile of the AI RMF, what ISO/IEC 42001 requires for management systems, and what the OWASP Top 10 for LLM Applications flags for agent-style failure modes side by side, you see the same five controls returning under different names. They are not new. They are the controls we have always used for anything that can act on its own. Aviation has them. Industrial robotics has them. Trading systems have them. Surgical robots have them. The discipline is to apply them to language-driven agents on purpose, before the agent goes live.
Authorization. Before an agent can act, you decide which actions it is allowed to take, on which systems, on whose behalf, within which limits. This is the boring identity-and-access work that the security team has done for thirty years, applied to a new kind of caller. The agent is a principal in your IAM model; it gets a role, a scope, and a quota, like any other service account. The mistake teams make is giving the agent the developer’s credentials so it can do “whatever it needs to”. That is not authorization, that is delegation of every privilege you have. The full breakdown sits in Authorization and Guardrails for AI Agents.
Audit logging. Every action the agent takes is logged with the prompt that triggered it, the tool that was called, the parameters that were used, the result, and the wall-clock time. The log is queryable, append-only, and retained long enough to survive a regulator’s question. This is the single most under-built control I see in production, because logging feels like infrastructure and the team is shipping features. The day you need the log, you need it complete and timestamped. The deep dive is Audit Logging and Traceability for Agentic Systems.
Human checkpoint on consequential actions. Not every action needs a human; the agent that summarises a meeting does not. The agent that issues a refund, opens a credit line, prescribes a medication, hires a candidate, files a regulatory submission, sends an email on the CEO’s behalf, those need a human eye before the action lands. The control is mechanical: declare which actions are consequential, route those into a review queue, block the action until a competent human releases it. The EU AI Act‘s human-oversight requirement for high-risk systems is exactly this control, written in legal language. See Human-in-the-Loop Checkpoints for High-Stakes Agent Actions.
Termination condition. Every agentic run has an explicit stopping rule. Maximum number of steps, maximum elapsed time, maximum spend, target-state-reached, error-budget-exceeded, take your pick. An agent without a termination condition is not a tool, it is an open invitation to a runaway loop, and runaway loops are how surprise bills, deleted records, and infinite-recursion incidents happen. “Termination condition” has been rising as a topic in the expert conversation I follow, and not by accident; the teams shipping agents in production found the cost of its absence the same week.
Hard stop. Separate from the termination condition, you need an out-of-band kill switch. Authorization can be revoked, the agent’s account can be disabled, the worker can be stopped, the queue can be drained. The hard stop is the structural-engineer’s emergency brake, and it has to be operable by someone who is not the agent’s developer. If the only person who can stop the agent is asleep, you do not have a hard stop, you have a hope.
Where the frameworks meet the agentic question
The four reference frameworks each touch agentic AI from a different angle. None of them is sufficient alone, and none of them is a surprise once you have done the IAM-plus-logging-plus-oversight work.
The NIST AI RMF Generative AI Profile treats agentic capability as a generative-AI risk amplifier. The Govern, Map, Measure, Manage functions all apply; the Profile adds explicit attention to agentic chains, tool use, and the propagation of errors through autonomous actions. The most usable part for an agentic program is the Map function, where you enumerate the actions and downstream systems the agent can touch.
ISO/IEC 42001 is a management-system standard, the AI equivalent of ISO 27001 for information security. For agentic AI, it forces you to define the AI management system that covers your agents, the roles inside it, the controls, the audits, the continuous-improvement loop. The certifiable bit is the management system, not the agent.
The OWASP Top 10 for LLM Applications gives you the offensive-security view: excessive agency, insecure plugin design, sensitive information disclosure, prompt injection, the failure modes a red team will actually try. If you are building agents that touch tools and data, OWASP is the checklist your security team should be running against your agent harness, not just your model.
The agent-safety guidance from frontier labs (Anthropic and equivalents) adds the design-pattern layer: tool-use schemas, parallel-tool-call hazards, sub-agent escalation, instruction-vs-data confusion. Useful, opinionated, and worth reading even if you do not use the underlying model, because the patterns apply to any agent that uses tools.
The shared GRC operating model that ties these together for the wider compliance program lives in Building an AI Governance Framework. For agents, you reuse that frame and add the five controls above as the agent-specific layer.
Walking one obligation end-to-end: human oversight on a consequential action
The same worked-example move I use in the compliance hub works here, on the obligation that lands hardest on agents: human oversight for high-risk systems.
The regulation says: a competent person must be able to understand the agent’s action, override it, and stop it. That sentence is the obligation. A policy that quotes it changes nothing.
The translation is two questions: which agent action is consequential, and what change to the workflow makes the obligation true by default. Take a customer-service agent that can issue refunds up to a stated limit. The consequential action is the refund itself. The workflow change is: every refund above a threshold lands in a supervisor’s queue with the conversation context, the policy reference, and a one-click approve-or-deny that the agent waits for. The agent cannot complete the refund without the approval; the supervisor cannot approve without the context. Two roles, one queue, one log entry, obligation met.
The same pattern recurs across domains. Hiring agent, the consequential action is the rejection or the advance; checkpoint on the recruiter’s queue. Coding agent, the consequential action is the merge to main; checkpoint as the pull request. Email agent, the consequential action is sending on someone’s behalf; checkpoint as the draft folder. The agent does the work; the human owns the consequence.
The honest summary
Agentic AI is the part of the AI conversation moving fastest in 2026, and the part where governance has the most catching up to do. The controls are not exotic and not new. Authorization, audit, human checkpoint, termination condition, hard stop. They are the same controls a chemical plant uses on an unattended reactor and a bank uses on an automated trading desk. The novelty is the speed at which the underlying agent gets better, and the breadth of the systems it can now touch.
You do not need a new framework. You need to apply the old ones, deliberately, before the agent ships. The cost of doing this work is small in the design phase, painful in the test phase, and very large in the incident phase. Choose your phase.
