The problem
Support tickets are manually categorised, delaying urgent issues. The team has no consistent way to measure whether a proposed AI classifier is reliable enough to use.
Assumptions
Tickets arrive via a single inbox; categories are stable; a human reviews all urgent tickets; historical tickets are available for evaluation.
Guardrails
No automatic closure; urgent classification requires human confirmation; model version and output are logged; sensitive ticket content is not sent to a third-party model without review.
Evaluation criteria
Urgent-ticket false-negative rate; time to categorisation; human override rate; percentage of tickets where the model abstains and routes to a human.
What would change in production
Integration with the helpdesk; data retention policy; escalation rules; model update process; monitoring dashboard for override rate.
Evaluation before routing
Begin by defining the ticket categories, the meaning of “urgent,” and the consequence of a missed urgent ticket. Build a labelled evaluation set that reflects actual language, including incomplete requests, multiple issues, ambiguous wording, unusual but important cases and the current category boundaries. Split development and evaluation examples so the same examples are not repeatedly used to tune and declare success. Remove or mask personal data unless its use has been reviewed and approved.
Measure each category separately. Overall accuracy can look acceptable while urgent cases are missed, so review the false-negative rate for urgent tickets, false positives, category-level precision and recall, abstention rate, and the number of tickets changed by a human reviewer. Define acceptance thresholds before testing and include a baseline, such as the current manual process. Re-evaluate whenever categories, prompts, models or upstream ticket fields change.
Decision logic and human review
The model should return a constrained category, a brief rationale or evidence field, and an explicit “uncertain / needs review” outcome. Validate the output against an allow-list before using it. Do not treat a model’s confidence wording as a calibrated probability unless calibration has actually been tested. Urgent tickets should be routed according to a conservative rule and confirmed by an accountable person; the concept does not automatically close tickets, send sensitive responses or take irreversible action.
When the output is malformed, outside the approved category set or missing required evidence, route it to the manual queue. Log the model and prompt version, the input reference, the structured result, the routing decision and any human override, while avoiding unnecessary storage of sensitive ticket text. Keep the previous rules-based or manual route available as a fallback.
Operational controls
Test model unavailability, slow responses, duplicated events, prompt-injection attempts embedded in ticket text, changing ticket volume and an unexpected category. A timeout should not leave a ticket in an invisible intermediate state. Retries must not assign the ticket multiple times, and an alert should identify the queue or event that needs attention without leaking confidential content.
The support owner approves categories and escalation policy; the technical owner monitors integration health and versions; reviewers label errors and report emerging patterns. Review the override rate and error categories on a schedule and after significant changes. If the false-negative rate or other pre-agreed safety indicator moves outside the accepted range, disable AI-assisted routing or return to human triage until the issue is understood. Production readiness requires documented test results, a known fallback, named ownership and a repeatable re-evaluation process—not just a convincing demo.