Platform extensions

This page covers platform-level behavior that does not depend on a specific model, as the code currently behaves. Model-specific parameters and limits are on each model's page. Items marked "may change" are known to-be-fixed issues; trust the actual response over this page.

Supported endpoints

Not provided: Files (/v1/files), Batch (/v1/batches, /v1/messages/batches), Fine-tuning, Uploads, Assistants / Threads, and the legacy /v1/completions. These paths have no handler and return the plain-text 404 page not found; a POST with a JSON body that has no model gets 400 (missing model field) first. Which parameters each endpoint supports is stated on that endpoint's page.

Request headers

Authentication

Only Authorization: Bearer <key> is accepted (Bearer is case-insensitive). The x-api-key header and ?key= / ?api_key= URL parameters are not accepted and return 401. The one exception is a browser WebSocket upgrade (Realtime), which cannot set custom headers and may use the api_key query parameter.

This is why the Anthropic SDK must use auth_token (which sends Authorization), not api_key (which sends x-api-key).

Key format and invalidation rules are in Get an API key.

Client headers the gateway forwards

The gateway forwards only these headers and drops everything else:

Headers that are not forwarded include anthropic-version (the platform sets it from the upstream's own configuration; whatever you send has no effect), Idempotency-Key, OpenAI-Organization, OpenAI-Project, and any custom header of yours. Identity, billing and routing headers are injected by the platform; sending a header with the same name has no effect.

Other request-header behavior

• X-Request-ID: if you send one it is used and echoed back unchanged; otherwise the platform generates a UUID.
• A duplicated /v1/v1/ in the path is collapsed to /v1/, so a client whose base_url already ends in /v1 still works.
• Browser cross-origin calls: the public endpoint returns CORS allow headers only for the platform's own site domains, so calling with a key directly from another web page is blocked by the browser. That is one more reason not to put a key in a frontend.

Response headers

• X-Request-ID: on every response. Include it when you report a problem.
• Retry-After (seconds): the platform sets it on only two kinds of response: the 429 for a key's RPM/TPM limit (it can be shorter than you actually need to wait when TPM tripped), and the 503 for a full gateway (always 5). No other error has it.
• The platform does not return x-ratelimit-* headers or any balance header. To see your limits, use the console or ask your administrator.
• Streaming: Content-Type: text/event-stream, with Cache-Control: no-cache and X-Accel-Buffering: no.

Heartbeats in streams

An SSE stream gets a : heartbeat comment line (starting with a colon) about every 15 seconds, to keep the connection open while a model thinks silently. Standard SSE clients and the official SDKs ignore it; if you write your own parser, ignore lines that start with :.

Automatic behavior

include_usage is added for you

When the request body has stream set to true and no stream_options field, the platform adds {"include_usage": true} so the last chunk carries usage. If you send stream_options yourself the platform does not touch it (including when you leave out include_usage).

Retries and upstream switching

• When a model has more than one upstream candidate and a request fails before the first byte, the platform switches to the next candidate until they run out, then returns the last upstream's error to you.
• With the current default policy, almost every failure triggers a switch except "context too long": connection errors, timeouts, 5xx, 429, and also upstream 400, 401, 403 (which may be specific to that upstream). May change.
• A model with a single upstream has nothing to switch to; an upstream failure is returned directly.
• Once streaming has started (data has already been sent to you) there is no more switching or retrying; the failure is written into the stream. See Error codes.
• Failed attempts are not billed; you pay only for the attempt that finally succeeds. The platform has no idempotency key, and a retry you make is a new request.
• The platform never moves a request to a different model on its own.

Behavior after an upstream rate limit (429)

• When an upstream returns 429 for a model, only that model is affected. Other models from the same vendor keep working; one limited model does not take its siblings down.
• If the model has other upstream candidates, the limited upstream stops receiving requests for a short while and the others serve the traffic. Nothing changes for you.
• If the model has a single upstream (or all of its upstreams are limited at once), the platform does not leave it unavailable for the whole window: every few seconds one request is let through to test the upstream, and service resumes as soon as the limit lifts. Other requests during that interval get 429 rather than 503.
• This 429 means "try again later". Back off and retry on your own (for example wait a few seconds and lengthen the wait on each attempt); do not resend immediately in a tight loop.
• Only when the upstream clearly says the whole account's quota is used up are all of that vendor's models paused together.

Timeouts

• A stream with no data for 4 minutes by default is cut (configurable per deployment).
• If the gateway gets no response headers within 120 seconds it returns 502; for a non-streaming request that may take more than two minutes, use stream: true.
• The request body must finish uploading within 30 seconds.

Task IDs and response IDs

A video task id (starting with video_) and a Responses id (usually starting with resp_) are opaque tokens issued by the platform; they encode which upstream and model handled the task. Store and use them exactly as returned. Do not parse, truncate or modify them, and do not mix them up with the vendor's own raw IDs. Tokens can be long, so do not store them in a short column.

Rate limits and balance

Rate limits

• The default tier is 60 requests per minute (RPM) and 1 million tokens per minute (TPM) per key. The effective value is the minimum of the organization's tier, the team limit and the key's own override; tiers are configured by administrators, so your account is what counts.
• Over the limit returns 429; message is always too many requests and does not say whether RPM or TPM tripped.
• The TPM limit counts only the text you send in messages; attachments such as images and the model's output are not counted. A request that does not use messages (for example the input of Responses and Embeddings) is bound only by RPM.
• GET /v1/models and Realtime sessions are not subject to RPM/TPM limits; Realtime has its own concurrent-session cap, and exceeding it returns 429 (no Retry-After).
• The nginx at the public endpoint also limits by source IP and answers with an HTML 429; see Error codes.

Balance and settlement

• If your balance is too low for the request as you sent it, the platform returns 402 instead of sending it to the model. With a balance near zero, a very large max_tokens can be rejected even if the actual output would have been short; lower it or top up.
• The actual charge is settled after the request ends, on real usage. A failed request with no usage is not charged; for a stream that fails midway, the part already generated is billed on actual usage.
• Once the balance is exhausted, only requests that cost money return 402; read-only or free requests such as GET /v1/models, video task and response lookups, DELETE /v1/responses/{id} and cancel keep working.
• If the balance runs out while a stream is running, the platform cuts the stream and writes data: {"error":{"type":"insufficient_quota","message":"balance exhausted"}} and data: [DONE].
• If a budget envelope is set (key, member or team level), running it out also returns 402, with (budget envelope: ...) in message.
• When the account has a discount, usage.cost is the discounted amount actually charged. The discount applies to every priced endpoint (chat, images, embeddings, rerank, audio, video and the rest), not only to chat.

Default limits at a glance

These are the defaults in the code; a deployment may change them, so the actual deployment is what counts.

Custom fields

Response: usage.cost and usage.cost_details

Two fields that the official APIs do not have may appear in usage:

Rules:

• No request parameter turns it on. Whether it appears depends on account settings: by default neither is returned; an organization can be set to return only cost, or the breakdown as well; a member's setting cannot exceed the organization's.
• When a request cannot be priced neither field appears (not 0, not null).
• Money is a string; token counts are numbers.
• The platform never returns the upstream cost, the discount rate, the list price or the tier a charge fell into.
• Places confirmed to carry these fields: /v1/chat/completions (non-streaming, and the last chunk of a stream that carries usage), /v1/responses (non-streaming), and the image endpoints /v1/images/generations, /v1/images/edits, /v1/images/variations, /api/v3/images/generations and the DashScope-style /api/v1/services/aigc/multimodal-generation/generation when it returns an image. Other endpoints do not carry them; whether Responses streaming does, trust the actual response.
• Images: the fields sit in the response's usage. If the model's own response has no usage object, one appears containing only cost (and cost_details when your account is set to return it), and only when there is a figure to show; the rest of the response is unchanged. At the default setting an image response is exactly what the model returned. cost_details keys for images are output_image_cost (per generated image), plus input_image_cost where input images are charged.
• Video: when your organization returns cost, the poll of a finished task (the Seedance, MiniMax and HappyHorse/Wan task-query endpoints) carries usage.cost with the total charged; there is no breakdown for video. Creating a task never carries cost, and at the default setting polls are exactly what they were: a task is reported as completed as soon as the video is ready. When cost is returned, a finished task is reported as still in progress, without a result link, until its charge has been booked. This usually takes a few minutes and is capped at about 5 minutes; after that the task is reported as completed without cost, and a later poll includes it once it is available. A completed task without cost means the amount was not stated, not that it was free. If your client gives up on a task after a fixed time, allow up to 5 extra minutes when cost is returned. The amount charged for a task is also in your usage records in the console.
• In Chat Completions the cost_details keys are prompt_cost, completion_cost, reasoning_cost, prompt_cache_read_cost and so on; in Responses input and output are called input_cost and output_cost, the rest are the same.
• Amounts are in US dollars, the unit your balance is charged in.

Also, any field in which a vendor states a cost, price, credit or balance (in usage or at the top level of the response) is removed on every endpoint, streaming included. The only cost figures in a response are the platform's own usage.cost and usage.cost_details.

Known differences from the official APIs

These are the current facts; rows marked "may change" are known to-be-fixed issues.