Track OpenAI usage per user, endpoint, and prompt version

Why totals fail in production

Monthly totals answer how much you spent, not why you spent it.

Production teams usually need three dimensions first: endpoint, tenant/user, and promptVersion.

Minimal telemetry fields for OpenAI cost monitoring

• externalRequestId (retry-safe idempotency key)
• provider, model, endpointTag, promptVersion
• userId (recommended to track OpenAI usage per user, hash when needed)
• inputTokens, outputTokens, totalTokens, latencyMs, status
• dataMode and environment for real/test/demo separation

What OpenAI dashboards and APIs do (and do not) solve

Provider dashboards are great for reconciliation: they tell you what you owe for a time window.

They are usually not enough for root cause: your bill rarely maps cleanly to your internal features, endpoints, and deploys without app-level tags.

• Use provider totals for accounting and invoice reconciliation.
• Use endpointTag + promptVersion + user/tenant mapping for operational ownership.
• Use externalRequestId to connect retries, failures, and multi-call workflows.

Dashboard workflow to find the cost driver

1. Start in Overview to confirm spend spike window.
2. Check Top Endpoints for feature-level concentration.
3. Check Top Users to find tenant concentration.
4. Check Prompt Versions for post-deploy cost/request drift.

Implementation notes that prevent noise

• Keep externalRequestId stable on retries.
• Use consistent endpointTag naming per feature path.
• Treat userId as optional and privacy-safe metadata.
• Use dataMode=test for synthetic traffic to protect real dashboards.

Identity mapping for per-user cost tracking (without privacy mistakes)

Per-user reporting only works when identity rules are stable. Treat userId and tenantId as routing metadata, not personal data.

Decide how you will represent service accounts, internal tooling, and anonymous traffic so your dashboards do not mix categories.

• Hash userId/tenantId (or use internal IDs) when privacy requires it.
• Bucket anonymous traffic (for example anon_ip_hash or session cohort).
• Tag service accounts explicitly so they do not hide inside “Top Users”.
• Normalize merges/renames so historical cost stays attributable.

PromptVersion strategy that makes cost regressions explainable

PromptVersion is your deploy signal. Without it, cost drift looks like random variance.

Bump the version whenever anything that can change spend changes: system prompt, tool policy, routing logic, retrieval config, or output formatting rules.

1. Add a promptVersion constant per endpointTag and bump on deploy.
2. Record short change notes (what changed and why).
3. Alert on cost/request and tokens/request deltas after a version change.
4. Keep rollback easy: one version change should be reversible.

Common pitfalls (the ones that break attribution)

• Generating a new externalRequestId per retry attempt (inflates volume).
• Changing endpointTag naming and losing historical continuity.
• Mixing staging/demo traffic with production baselines.
• Letting fallback routing change silently without promptVersion bumps.

What to alert on

• cost/request drift after a promptVersion change
• tokens/request increase (input or output) on a single endpointTag
• endpointTag concentration shift (one feature dominates spend)
• tenant/user concentration shift (one customer dominates spend)
• retry multiplier increase (errors + repeated externalRequestId)

Execution checklist

1. Define a stable endpointTag taxonomy and document naming rules.
2. Implement promptVersion tagging and bump versions on deploy changes.
3. Adopt externalRequestId and keep it stable across retries and multi-call flows.
4. Add privacy-safe userId/tenantId mapping rules (hashing, service accounts, anonymous buckets).
5. Validate dashboards weekly and tune alerts on the top 3 cost drivers.

FAQ

Related guides

• Full guide: LLM cost attribution: endpoint, prompt version, tenant, and user
• Prompt deploy cost regressions pillar
• OpenAI dashboard totals vs bill root cause
• Prompt deploy cost regressions
• 15-minute spike checklist
• LLM budget alerts pillar
• No-proxy telemetry quickstart
• Quickstart (5-minute setup)
• Alternatives and tradeoffs

Evaluation resources

For security and procurement reviews, use our trust summary before final tool selection.