The gap between an agent that demos well and an agent you can put in front of customers is almost never the model. It is everything around the model.
What the demo skips
A demo agent has one tool, a happy path, and a human watching. Production adds:
Scoped credentials. The agent should not hold the same keys your backend does. Give it a narrow service identity with exactly the permissions its tools need, so a prompt injection buys the attacker very little.
Retry and idempotency. Models retry. Tools time out. If “create booking” can be called twice for the same intent, it will be. Every mutating tool needs an idempotency key that the agent passes through.
Cost ceilings. A loop that re-reads a large document on every step is a bill, not an outage — which is worse, because nothing alerts. Cap tokens per conversation and per tenant, and alert on the derivative, not the total.
An eval suite. Without one, you cannot change a prompt. Any edit becomes a gamble, because you have no way to know what it broke. Twenty recorded cases with expected outcomes is enough to start and is worth more than another week of prompt tuning.
An escalation path. Decide in advance what the agent hands to a person, and make handing off cheap. An agent that struggles visibly for six turns is worse than one that gives up on turn two with a good summary.
Where the time actually goes
Roughly: a day on the agent, a week on the tools, two weeks on the harness. That ratio surprises people, but the tools are where the real system lives — every one of them needs its own error semantics, its own permission scope, and its own test.
The useful mental model
Treat the model as an unreliable but capable contractor. You would not give a contractor root access, an unlimited budget, and no way to ask questions. The harness is how you give them a scoped badge, a spending limit, and a manager’s number.