Noticing a Client Is Unhappy Before They Tell You

The challenge

By the time a client tells you they are unhappy, they have usually been unhappy for a while. The evidence was there all along in how their tickets were written — but nobody reads a quarter of ticket comments looking for tone. Separately, pitching a response correctly depends on knowing whether you are writing to someone comfortable with technology or someone who is not, and technicians were guessing.

Where we came in

Scores like these are worthless if you cannot trust them, so most of the work went into the validation rather than the scoring. Outputs are clamped to valid ranges, labels outside the permitted set are rejected rather than coerced, and every score carries a confidence. We also run the model ourselves on our own worker, because sending every customer comment to a third-party API is a different privacy conversation than we want to have.

What we did

• Scored each end-user comment for sentiment and for technical proficiency as it arrives
• Ran the models on our own PyTorch and Transformers worker rather than sending customer text to an external API
• Rejected invalid labels outright and clamped scores to valid ranges instead of accepting whatever came back
• Attached a confidence value to every score so weak signals can be treated as weak
• Rolled per-comment scores up into a per-client and per-technician state, so a trend is visible rather than a single reading
• Backfilled historical tickets so the trend had history behind it from the start
• Covered the scoring logic with unit tests for clamping, invalid input and serialisation

Where it landed

• Declining client sentiment is visible as a trend rather than discovered at renewal
• Responses can be pitched to the recipient's actual comfort with technology
• Customer comment text is scored on infrastructure we control rather than sent to a third-party service