No-SDK LLM cost tracking: production setup with direct ingest API

When no-SDK setup is the right first move

Teams often start with direct API when they want fast rollout without waiting for package adoption.

The no-SDK path keeps provider traffic unchanged and adds telemetry in app logic.

Minimum production contract

• externalRequestId stable across retries
• provider, model, endpointTag, promptVersion
• inputTokens/outputTokens/latencyMs/status
• dataMode and environment for clean operational segmentation

Architecture overview (direct ingest flow)

The simplest no-SDK pattern is: call your provider as usual, extract usage fields from the response, and send one telemetry event to Opsmeter.

Keep ingest off the critical path. Telemetry should never block user requests.

1. Provider call completes.
2. You attach endpointTag + promptVersion + externalRequestId to the event.
3. You send the event asynchronously with a short timeout and swallow-on-error behavior.
4. Dashboards aggregate by endpoint, tenant/user, and deploy so incidents are explainable.

Data quality rules (prevent noisy attribution)

• Normalize model identifiers into a stable catalog key (provider + model).
• Keep endpointTag taxonomy stable; never embed user-specific values.
• Hash userId/tenantId when needed and document the mapping rules.
• Always include dataMode + environment so synthetic traffic does not pollute baselines.

Retry safety and idempotency (why externalRequestId matters)

Without a stable externalRequestId, retries look like new work and your dashboards over-count both volume and spend.

If your app retries upstream calls, keep the same externalRequestId so you can measure retry multipliers and isolate reliability-driven cost spikes.

• Generate externalRequestId once per user action (not per attempt).
• Pass it through logs and telemetry so incidents are traceable end-to-end.
• On 429, respect Retry-After and avoid tight loops that multiply cost.

Production-safe send pattern

1. Send telemetry asynchronously (fire-and-forget).
2. Use timeout + swallow to protect user request path.
3. On 429 read Retry-After and back off.
4. On 402 pause telemetry and keep provider calls running.

Rollout plan (reduce integration risk)

1. Start with one high-volume endpointTag to validate attribution.
2. Verify tokens, cost/request, and model mapping on a small canary window.
3. Expand to the next 5 endpoints that dominate spend (80/20).
4. Add budgets/alerts only after traffic classification (dataMode + env) is clean.
5. Document naming conventions so teams add tags consistently.

What SDK wrappers will add later

• Automatic capture wrappers in common frameworks.
• Runtime enforcement patterns (clamp/fallback/queue).
• Standard policy contracts for machine-readable actions.

What to alert on

• cost/request drift by endpointTag or promptVersion
• unexpected tenant concentration in Top Users
• request burst with falling success ratio
• budget warning, spend-alert, and exceeded state transitions

Execution checklist

1. Decide your endpointTag taxonomy (feature ownership) and promptVersion rules (deploy accountability).
2. Implement externalRequestId and keep it stable across retries.
3. Send telemetry asynchronously with timeout + swallow-on-error behavior.
4. Separate demo/test from production with dataMode + environment.
5. Validate dashboards (Top Endpoints, Top Users, Prompt Versions) on a canary window before scaling.

FAQ

Related guides

• Full guide: Proxy vs no-proxy LLM observability: tradeoffs for production teams
• No-proxy attribution: how it works
• Integration docs
• Operations docs
• LLM cost attribution pillar
• LLM budget alerts pillar
• No-proxy telemetry quickstart
• Quickstart (5-minute setup)
• Alternatives and tradeoffs
• Prompt deploy regressions pillar

Evaluation resources

For security and procurement reviews, use our trust summary before final tool selection.