A Support Agent That Answers First, and Knows When to Hand Over

The challenge

A large share of first-line tickets are questions that have been answered before, on devices whose state we already collect, against documentation we already maintain. Every one of them still costs a technician a context switch — and the ones that genuinely need a person wait behind them.

Where we came in

The failure mode of an autonomous support agent is not that it is wrong occasionally, it is that it is confidently wrong to a customer with nobody watching. So we built the confidence gate before the capability, defaulted the pipeline to shadow mode so it drafts and logs without sending, and made escalation a first-class outcome rather than an error path. Auto-respond is opt-in, not the default.

What we did

• Built a staged pipeline where each step can fail cleanly and the run only produces output if the whole pipeline succeeded
• Gave the agent retrieval over the knowledge base, so responses cite maintained documentation rather than model recall
• Let the agent read device vitals and identify which machine a user is on before suggesting diagnostics
• Combined the stages into a single confidence score and gated the decision on it
• Made escalation to a human an explicit pipeline decision, with responses blocked on already-escalated tickets
• Ran it in shadow mode by default — drafting and logging without sending — so behaviour could be reviewed before enabling
• Surfaced drafts to technicians for review, edit and approval as a middle path between silent and autonomous
• Kept model choice swappable and supported running two models side by side to compare results

Where it landed

• Repeat first-line questions get an immediate drafted response grounded in our own documentation
• Low-confidence tickets escalate to a person by design rather than by exception
• Every interaction is logged, so the agent's behaviour is auditable rather than assumed