Bot attacks and LLM cost spikes: prevention playbook

What to send (payload example)

{
  "externalRequestId": "req_01HZXB6MQZ2WQ9D2KCF9M4V2QY",
  "provider": "provider_id",
  "model": "model_id",
  "endpointTag": "public.chat",
  "promptVersion": "public_v1",
  "userId": "anon_ip_hash",
  "inputTokens": 260,
  "outputTokens": 190,
  "latencyMs": 892,
  "status": "success",
  "dataMode": "real",
  "environment": "prod"
}

Common mistakes

• Shipping provider keys to the client or logging them in plaintext.
• No per-endpoint rate limits for high-cost workflows.
• Treating retry storms as "just reliability" while costs multiply.
• Delaying containment while searching for perfect root-cause.

How to verify in the Opsmeter.io dashboard

1. Use Overview to confirm spike window and budget posture.
2. Use Top Endpoints to find feature-level concentration.
3. Use Top Users to find tenant-level concentration.
4. Use Prompt Versions to validate deploy-linked cost drift.

Prevention and response stack

• Rate-limit patterns by endpoint criticality
• Retry backoff and duplicate suppression
• Key leak response with rotation timeline
• Concentration-based alerts and owner actions

Detection signals to monitor continuously

• Sudden request burst with low identity diversity
• Retry ratio increase without corresponding provider outage
• Tenant or endpoint concentration jump in short windows
• Fast rise in token-per-request with unchanged feature traffic

Incident ownership model

1. Security owner handles key rotation and abuse source blocking.
2. Platform owner applies retry and rate-limit containment.
3. Product owner evaluates model/tokens guardrails by feature.
4. Finance owner logs cost impact and post-incident actions.

Containment first: stop the financial bleeding

When abuse hits, speed matters more than perfect diagnosis. Containment reduces the blast radius so you can investigate safely.

Treat cost spikes as incidents: identify the driver, contain, then harden.

• Throttle public endpoints and non-critical features first.
• Cap output tokens to prevent long abusive completions.
• Block obvious automation patterns (IP ranges, user agents, failed auth bursts).
• Rotate compromised keys and revoke leaked credentials immediately.

Rate-limit patterns that work for LLM endpoints

Global rate limits are rarely enough. LLM costs vary by endpointTag, so enforcement must also vary by endpoint criticality and cost profile.

• Per-endpointTag limits (high-cost endpoints get tighter limits).
• Per-tenant limits (one customer should not drain shared margin).
• Burst limits + sustained limits (stop spikes and slow drains).
• Identity-aware limits (unknown-user traffic is higher risk).

Key leak response checklist (first hour)

1. Rotate keys and invalidate all leaked credentials.
2. Audit recent request logs for new endpoints and new traffic sources.
3. Identify the top endpointTag and tenant/user concentration during the spike.
4. Add temporary strict caps (tokens, requests) until stable.
5. Create a permanent control: secret scanning, least privilege, and rotation policy.

Signals that distinguish abuse from regressions

• Abuse: burst traffic with low identity diversity and high error variance.
• Regressions: cost/request drift after deploy with stable traffic volume.
• Retry storms: higher retry ratio and longer tail latency, often with upstream errors.
• Pricing drift: unknown-model ratio rises or cost snapshots look inconsistent.

Post-incident hardening (make the next incident cheaper)

1. Add alerting on unknown-user concentration and token-per-request spikes.
2. Require per-endpoint output caps and max tool calls for agent workflows.
3. Add per-tenant budgets for high-variance accounts.
4. Document owner actions and update the response runbook.
5. Run a weekly review of top endpoints/users to catch slow-drain abuse.

Related guides

• OpenAI bill shock: bot abuse and rate-limit checklist
• Retry storms: how retries can multiply your LLM bill
• Leaked API key cost spike response
• Bot abuse on LLM endpoints
• LLM budget alerts pillar
• AI cost spike playbook
• LLM cost attribution pillar
• No-proxy telemetry quickstart
• Quickstart (5-minute setup)
• Alternatives and tradeoffs

Evaluation resources

For security and procurement reviews, use our trust summary before final tool selection.