Support is where post-training most often pays for itself first: high volume, a clear definition of a good outcome, and years of resolved tickets that already contain the behaviour you want. It is also where a model that sounds right and acts wrong — the confident refund outside policy — does the most damage.
Turning tickets into training data#
Resolved tickets are demonstrations, but noisy ones. Filter before training: keep tickets resolved on first contact or with high CSAT, drop agent macros that were pasted without adaptation, redact PII (names, emails, card and account numbers) upstream, and de-duplicate templated conversations — in support data near-duplicates are often a quarter or more of the corpus. Split by time so the eval contains tickets newer than anything in training.
Label each ticket with an intent and outcome. You'll need them for evaluation slices, and the long tail of intents is where models fail.
Method: SFT, then preferences, then a policy reward#
- SFT on curated resolutions teaches tone, structure and the tool-call schema (refund API, escalation, account lookup).
- DPO on pairs where a reviewer preferred one response over another sharpens judgement on the borderline cases — partial refunds, goodwill gestures, when to escalate.
- GRPO with a policy reward for the decisions that can be checked: was the refund within policy given the account state; was escalation triggered for the intents that require it. Penalise out-of-policy actions heavily; that is where reward hacking appears first.
Retrieval stays: policies and product facts change, so the model should read them from a knowledge base and be trained to answer from it — see fine-tuning vs RAG.
Evals that predict production#
| Metric | How |
|---|---|
| Resolution rate | Calibrated judge on full conversations with simulated customers |
| Policy compliance | Deterministic check of actions against account state and policy rules |
| Escalation precision/recall | Against human-labelled tickets |
| Tone | Judge, calibrated; report sensitivity/specificity |
| Per-intent slices | Every intent with ≥ 50 items; non-English slices separately |
Size the eval for the improvements you care about; at 300 conversations only large gains are detectable (sizing table).
Economics#
Support volumes are usually high enough to clear the self-hosting floor; the break-even model puts it around a few hundred million output tokens a month under representative assumptions, and support prompts carry long context, which favours self-hosting sooner. A distilled 8B–32B agent also answers faster, which customers notice.
Frequently asked questions#
Can I fine-tune an LLM on my support tickets?
Yes, after filtering for good resolutions, redacting PII and de-duplicating. Resolved tickets make strong SFT data; reviewer preferences and policy-checkable rewards refine judgement.
How do I stop a support model from offering refunds it shouldn't?
Make the refund decision checkable against account state and policy, penalise violations in the RL reward, and gate releases on a deterministic policy-compliance metric.