Use case
LLM cost attribution for support chatbots
Support chatbots often hide margin leaks in routing and prompt design. Endpoint attribution makes those leaks visible.
Full guide: Cost attribution by use-case: templates for real apps
Recommended endpoint taxonomy
- chat_answer
- chat_summary
- ticket_draft
- knowledge_suggest
Metrics that matter for support teams
- cost per handled conversation
- cost/request by endpointTag
- tenant concentration in Top Users
- promptVersion drift after bot updates
Operational checklist
- Tag each chatbot flow with endpointTag.
- Use promptVersion on every support prompt release.
- Set budget thresholds for support workspace.
- Review weekly export with support and finance owners.
Unit economics: cost per ticket and cost per resolution
Support teams care about outcomes: resolved tickets, deflected tickets, and agent time saved. Token totals do not map cleanly to those outcomes.
Track cost per handled conversation and cost per resolved ticket so pricing and staffing decisions reflect reality.
- cost per resolved ticket (primary)
- cost per deflection (when bot prevents escalation)
- cost per escalation avoided (when routing improves)
- cost per tenant for high-volume support accounts
Where support bot costs usually spike
- Long conversation history injected into every turn (context creep).
- Verbose answers after prompt changes (output token inflation).
- RAG retrieval pulling too many chunks (top-k drift).
- Retry storms during provider timeouts (multiplies spend).
- Abuse traffic on public support channels (unknown-user bursts).
Guardrails that protect support margin
- Cap output tokens for auto-replies and summaries.
- Use smaller models for low-risk flows (status updates, routing).
- Apply per-tenant budgets for high-volume accounts.
- Alert on cost/request drift by promptVersion after releases.
- Throttle unknown-user traffic on public endpoints.
Routing to humans without runaway cost
Routing logic can create hidden loops: model answers, then summarizes, then drafts a ticket, then rewrites. Attribution by endpointTag exposes the loop.
Keep escalation paths explicit and measure the full workflow cost, not just one request.
- Separate endpoints for answer vs summarize vs ticket draft.
- Measure cost per stage and enforce caps on low-value stages.
- Review promptVersion changes when routing behavior shifts.
Weekly review format (10 minutes)
- Top endpoints by spend share (chat_answer vs ticket_draft).
- Top tenants by spend and concentration %.
- promptVersion deltas (before vs after) for the last release.
- Outliers: highest token-per-request conversations.
- One action owner + one permanent guardrail update.
What to alert on
- cost/request drift by endpointTag or promptVersion
- unexpected tenant concentration in Top Users
- request burst with falling success ratio
- budget warning, spend-alert, and exceeded state transitions
Execution checklist
- Confirm spike type: volume, token, deploy, or abuse signal.
- Assign one incident owner and one communication channel.
- Apply immediate containment before deep optimization.
- Document the dominant endpoint, tenant, and promptVersion driver.
- Convert findings into one permanent guardrail update.
FAQ
Is userId required?
No. userId is optional, but recommended for tenant-level attribution. If needed, send a hashed identifier.
Where should token usage values come from?
Prefer provider usage fields first. If unavailable, use tokenizer estimates and mark uncertainty in your workflow.
How should retries be handled?
Keep the same externalRequestId for the same logical request so idempotency remains stable across retries.
Can telemetry break production flow?
It should not. Use short timeouts, catch errors, and keep telemetry asynchronous so provider calls keep running.
Related guides
Evaluation resources
For security and procurement reviews, use our trust summary before final tool selection.