Developer Docs

Generate a workspace API key

After sign-in, rotate a new key in Settings. Keys are shown only once.

Send telemetry

Post LLM request metadata to the ingest endpoint after each model call. Include externalRequestId on every request for idempotency. Treat it like a unique, per-request key you generate on your side.

Track budgets

Use the Dashboard to monitor spend, latency and budget alerts in real time. Ingest responses include budgetWarning and budgetExceeded plus plan metadata for client-side UX.

externalRequestId playbook

Generate one ID per LLM call and reuse it on retries. Store it in request context (middleware/local variable) so the same ID flows through every retry.

const ctx = { externalRequestId: existingId ?? null };
const externalRequestId = ctx.externalRequestId ?? crypto.randomUUID();
ctx.externalRequestId = externalRequestId;

for (let attempt = 0; attempt < 3; attempt++) {
  try {
    return await llmCall();
  } catch (err) {
    if (attempt === 2) throw err;
  }
}
// telemetry uses the same externalRequestId from ctx

endpointTag examples

Tag = product feature, not endpoint path. Example tags: checkout.ai_summary, support.reply, invoice.extract.

promptVersion strategy

Treat prompt versions like deploy labels: summarizer_v3, chat-v5.2. Rule: new deploy = new version.

userId optional + PII warning

userId is optional. If omitted, requests group into unknown. Never send PII; hash identifiers if you need stable user grouping.

Latency measurement

Capture a timestamp before the LLM call and compute latency after it finishes.

const start = Date.now();
const latencyMs = Date.now() - start;

Token sources

• Provider response usage fields.
• Approximate with a tokenizer library.

Telemetry should never break prod (Golden Rule)

• Non-blocking calls only.
• Timeouts: 300-800ms.
• Try/catch and swallow telemetry errors.
• Fire-and-forget ingestion.
• On 429, show/log: Telemetry throttled, retry after Xs.

402 Telemetry Paused handling

402 = telemetry pause. LLM calls continue. Do not retry telemetry immediately. Pause ingestion for X minutes (for example 10-15), show a UI banner, and surface an upgrade CTA.

Batching / queue (high volume)

For high throughput, queue and batch telemetry. Opsmeter ingestion can be async and should not sit on the request path.