The situation
A subscription software company sees most of its support queue repeat the same issues: billing questions, plan changes and how-to requests already covered in the documentation. An earlier chatbot pilot impressed internally but was never trusted with real customers.
The aim is an agent that resolves routine issues end to end and hands over everything else with its full context intact.
The challenge
Accuracy has to be provable
Leadership needs evidence that quality holds up, and that it stays up when things change.
Real actions carry real risk
Refunds and plan changes need firm limits that a clever prompt cannot talk around.
Untrusted input
Customer messages and retrieved pages can contain instructions aimed at the agent.
The approach
- 1
Build the evaluation setWeeks 1–3
Convert a sample of historical tickets into a graded test suite that sets the quality bar.
- 2
Grounded answersWeeks 4–6
Add retrieval over approved content, with citations and permission checks.
- 3
Scoped tools and approvalsWeeks 7–9
Expose narrowly scoped account tools, enforce limits in code and route sensitive actions for approval.
- 4
Shadow and supervised releaseWeeks 10–12
Compare draft responses with human agents, then increase traffic as the numbers hold.
Key decisions and trade-offs
Build evaluations before the agent
Why: Without a fixed benchmark, every prompt change is a matter of opinion.
Trade-off: Stakeholders see less visible progress in the first weeks.
Enforce limits in tools, not prompts
Why: Refund caps and permissions in code hold regardless of what the model is told.
Trade-off: Each new capability requires engineering work.
Escalate with full context
Why: Customers should never have to repeat themselves when a person takes over.
Trade-off: The handoff needs deep integration with the helpdesk.
How success is measured
| Measure | Baseline | Target | How it’s measured |
|---|---|---|---|
| Automated resolution | 0% | 50% of eligible tickets | Helpdesk outcomes, excluding reopened tickets |
| Answer accuracy | Unmeasured | 95% on evaluation suite | Regression run before each change |
| First response | Hours | <30 seconds | Helpdesk timestamps |
| Audit coverage | None | 100% of actions logged | Trace review |
Key takeaways
- Your evaluation set is the most valuable asset in an AI project.
- Treat anything the agent reads as untrusted input.
- A clean handoff to a person is a feature, not a failure.