AgentOps
The short version: agents don’t fail loudly. They drift. Six months after launch, an unmaintained agent is escalating more, resolving less, and nobody has noticed because nothing broke.
Last updated:
What actually degrades
- 01
Models get deprecated. The model you launched on will be retired. Migration isn’t a config change — behaviour shifts, and you need a regression suite to know how.
- 02
APIs change. Your PMS, CRM or ERP ships an update. A field is renamed. The agent starts failing silently on one branch of the workflow.
- 03
Business rules move. Your call-out policy changes. Your permit rules tighten. The agent still follows last year’s version.
- 04
Edge cases accumulate. Real customers do things nobody anticipated. Without someone reviewing escalations, the same failure recurs weekly.
- 05
Costs creep. Prompt bloat, redundant retrieval and inefficient routing raise your per-interaction cost quietly.
What the retainer covers
| Covered | What it means |
|---|---|
| Monitoring | Uptime, latency, error rates, escalation rates, cost per interaction |
| Evaluations | Regression suite run on a schedule and before every change |
| Drift detection | Statistical monitoring of output quality against a baseline |
| Model migration | Deprecation handled before it becomes urgent |
| Integration repair | API changes fixed as they land |
| Escalation review | Monthly review of what escalated and why, feeding back into thresholds |
| Expansion | A defined allocation of new workflow build each month |
Reporting
A monthly report showing: interactions handled, resolution rate, escalation rate and reasons, cost per interaction, eval scores against baseline, and changes made.
If those numbers move the wrong way, you’ll see it from us before you hear it from a customer.
Scaled to agent count and workflow criticality. Cancel with 30 days’ notice; you keep everything.
Frequently asked questions
Is this optional?
Contractually yes. Practically, an agent in production without evaluation infrastructure is an unmonitored system making decisions on your behalf.
Can our own team do this?
Yes, and some should. We’ll hand over the eval suite, documentation and runbooks. Several clients run AgentOps internally after six months — that’s a good outcome.
What if we cancel?
The agent keeps running. It runs in your infrastructure. It will degrade over time without maintenance, but nothing switches off.
How is this different from a support contract?
Support is reactive. AgentOps is measurement — the point is catching degradation before anyone reports a problem.