Every company we talk to has run an agent pilot. Almost none of them have run an agent in production. The gap between those two sentences is not a model problem.
The demo works. Someone wires a capable model to a few internal APIs, shows it drafting an intake summary or reconciling a schedule, and the room agrees it is impressive. Then someone from compliance asks who the agent is, what it is allowed to touch, who signed off on the write it just made, and what happens when it is wrong. The pilot has no answers, because nobody built the part that produces answers. It stays a pilot.
The missing part is not intelligence
Models crossed the threshold for real, multi-step office work a while ago. The Model Context Protocol standardized how they call tools. What did not arrive alongside them is the layer every business actually requires before it can say yes: identity, permissions, approvals, audit, and cost control.
Those five things are not features you bolt on after the pilot succeeds. They are the reason the pilot is allowed to become production at all. Retrofitting them is how a six-week proof of concept turns into an eighteen-month platform project.
Three failure modes we see repeatedly
The agent has no seat in your access model. It runs as a shared service account with broad credentials, or as a borrowed human login. Nothing scopes what it can reach, and afterward nothing distinguishes what it did from what a person did. There is no honest answer to "who made this change."
The automation drives screens. Clicking through a user interface is the fastest way to a demo and the slowest way to production. It breaks on every redesign, it cannot be permissioned below the level of "this user can log in," and the audit trail is a screen recording rather than a record of intent.
Approval is a checkbox, not a workflow. Teams add a human confirmation step, but the human is shown a request with no evidence. Approve or decline, with nothing to judge. People rubber-stamp, or they open four other systems to verify by hand, which costs more than doing the task themselves. Either way the control is theatre.
What we build instead
We started in home health care, under HIPAA, which is an unforgiving place to learn this. You cannot ship an agent that "usually" gets it right when the output touches a patient record. That constraint produced the architecture we now bring to every engagement: agents call typed, validated operations through a single governed gateway; every agent has its own identity and least-privilege scope; every action lands in an audit trail with per-run cost accounting; and consequential actions are proposed with their evidence attached, for a person to approve.
None of that makes the agent smarter. It makes the agent allowed. That is the part the pilot was missing.