Implementation guide for AI cost control

Generate a workspace API key

After sign-in, rotate a new key in Settings → API keys. Keys are shown only once. Then continue with a ready payload and send your first ingest call.

Send telemetry

Post LLM request metadata to the ingest endpoint after each model call. Include externalRequestId on every request for idempotency. Treat it like a unique, per-request key you generate on your side.

Track budgets

Use the Dashboard to monitor spend, latency, and budget posture. Basic budget alerts are available on Starter and above plans; advanced alerts are available on Pro and above plans. Ingest responses include explicit status fields such as reason, telemetryPaused, providerCallsContinue, isPlanLimitReached, plus legacy budget flags (budgetWarning, budgetExceeded) for backward compatibility. See Limits & budgets and n8n integration for branching examples.

externalRequestId playbook

Generate one ID per LLM call and reuse it on retries. Store it in request context (middleware/local variable) so the same ID flows through every retry.

const ctx = { externalRequestId: existingId ?? null };
const externalRequestId = ctx.externalRequestId ?? crypto.randomUUID();
ctx.externalRequestId = externalRequestId;

for (let attempt = 0; attempt < 3; attempt++) {
  try {
    return await llmCall();
  } catch (err) {
    if (attempt === 2) throw err;
  }
}
// telemetry uses the same externalRequestId from ctx

endpointTag examples

Tag = product feature, not endpoint path. Example tags: checkout.ai_summary, support.reply, invoice.extract.

promptVersion strategy

Treat prompt versions like deploy labels: summarizer_v3, chat-v5.2. Rule: new deploy = new version.

userId optional + PII warning

userId is optional. If omitted, requests group into unknown. Never send PII; hash identifiers if you need stable user grouping.

Latency measurement

Capture a timestamp before the LLM call and compute latency after it finishes.

const start = Date.now();
const latencyMs = Date.now() - start;

Token sources

• Provider response usage fields.
• Approximate with a tokenizer library.

Telemetry should never break prod (Golden Rule)

• Non-blocking calls only.
• Timeouts: 300-800ms.
• Try/catch and swallow telemetry errors.
• Fire-and-forget ingestion.
• On 429, show/log: Telemetry throttled, retry after Xs.

402 Telemetry Paused handling

402 = telemetry pause. LLM calls continue. Do not retry telemetry immediately. Pause ingestion for X minutes (for example 10-15), show a UI banner, and surface an upgrade CTA.

Batching / queue (high volume)

For high throughput, queue and batch telemetry. Opsmeter.io ingestion can be async and should not sit on the request path.