Chat Completions (OpenAI-compatible)

Compatible with OpenAI Chat Completions. Point base_url at ModelSite and the same code calls DeepSeek, GLM, Qwen, MiniMax and the other live models, with no business-code changes. Authentication and billing are handled by the platform.

This page is organized by scenario: start with the scenario overview, then go to the section for request, response and caveats. Every example carries an evidence label. Tested means it was called for real with a live test account (2026-10-19); response fields come from the real reply, with id, timestamps and reasoning text replaced by placeholders. Per official docs (not tested) means it was written from the OpenAI documentation and we did not verify it with a real call.

Scenario overview

Scenarios follow OpenAI's Chat Completions, Streaming, Function calling, Structured outputs, Images and vision, Reasoning and Prompt caching guides.

Basic chat

When to use: one question, one answer, or a system message to set role and tone.

Evidence: tested (live test account, 2026-10-19).

Request

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Explain quantum entanglement in one sentence"}],
    "max_tokens": 1500
  }'

model is required and case-sensitive (billing and routing both depend on it). For thinking models, the thinking also counts toward max_tokens; if it is too small the reply is cut off during thinking (finish_reason is length and content is empty).

Response

The reply text is choices[0].message.content; the stop reason is finish_reason (stop / length / tool_calls / content_filter); usage is in usage.

{
  "id": "chatcmpl-xxxx",
  "object": "chat.completion",
  "created": 1760000000,
  "model": "glm-5.3-flash",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "量子纠缠是指两个粒子形成一种特殊的量子关联，使得无论它们相距多远，测量其中一个粒子的状态都会瞬间确定另一个粒子的对应状态。\n\n（注意：这种“瞬间关联”并不能用来超光速传递信息。）",
        "refusal": null,
        "reasoning_content": "The user is asking me to explain quantum entanglement in one sentence… (reasoning text omitted)"
      }
    }
  ],
  "usage": {
    "prompt_tokens": 17,
    "completion_tokens": 642,
    "total_tokens": 659,
    "prompt_tokens_details": {
      "cached_tokens": 0,
      "created_cache_tokens": 0
    },
    "completion_tokens_details": {
      "reasoning_tokens": 591,
      "audio_tokens": 0,
      "accepted_prediction_tokens": 0,
      "rejected_prediction_tokens": 0
    }
  },
  "system_fingerprint": "fp_xxxx"
}

The reply text above is real output for a Chinese prompt (the real request was written in Chinese).

Notes

• The *_details fields in usage differ by upstream; they are usually 0 when there is no matching usage. Cost fields are not returned by default (see "Platform extensions" below).
• Some upstream-specific usage fields (for example DeepSeek's prompt_cache_hit_tokens) appear in usage as-is; upstream fields whose names contain cost / price are removed by the platform.
• The API card below is rendered from the OpenAPI spec: HTTP method and path, an interactive request-body parameter table, and the response schema.

curl -X POST "https://example.com/chat/completions" \  -H "Content-Type: application/json" \  -d '{    "model": "string",    "messages": [      {        "role": "system"      }    ]  }'

{  "id": "chatcmpl-xxxx",  "object": "chat.completion",  "created": 1760000000,  "model": "glm-5.3-flash",  "choices": [    {      "finish_reason": "stop",      "index": 0,      "message": {        "role": "assistant",        "content": "量子纠缠是指两个粒子形成一种特殊的量子关联，使得无论它们相距多远，测量其中一个粒子的状态都会瞬间确定另一个粒子的对应状态。\n\n（注意：这种“瞬间关联”并不能用来超光速传递信息。）",        "refusal": null,        "reasoning_content": "The user is asking me to explain quantum entanglement in one sentence, in Chines…（已省略）"      },      "stop_reason": 154827    }  ],  "usage": {    "prompt_tokens": 17,    "completion_tokens": 642,    "total_tokens": 659,    "prompt_tokens_details": {      "cached_tokens": 0,      "created_cache_tokens": 0    },    "completion_tokens_details": {      "reasoning_tokens": 591,      "audio_tokens": 0,      "accepted_prediction_tokens": 0,      "rejected_prediction_tokens": 0    }  },  "system_fingerprint": "fp_xxxx"}

Multi-turn

When to use: conversations that need context. The platform does not store conversations; put the full history in messages on every request.

Evidence: per official docs (not tested). Multi-turn with tools (including assistant history and tool messages) was tested, see Tool calling.

Request

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [
      {"role": "system", "content": "Answer briefly"},
      {"role": "user", "content": "What is quantum entanglement?"},
      {"role": "assistant", "content": "Two particles share a state regardless of distance."},
      {"role": "user", "content": "Can it be used for faster-than-light communication?"}
    ],
    "max_tokens": 1500
  }'

Response: same structure as Basic chat.

Notes

• messages[].role = "developer" is rewritten to system.
• When you send assistant history back, some thinking models also need reasoning_content; see Thinking.

Streaming

When to use: show output token by token and lower time to first token.

Evidence: tested (live test account, 2026-10-19).

Request

curl -N https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Reply with one character: 好"}],
    "max_tokens": 600,
    "reasoning_effort": "low",
    "stream": true
  }'

Response

Each data: line is one JSON object; concatenate choices[0].delta.content for the text; the stream ends with data: [DONE].

data: {"id":"chatcmpl-stream-xxxx","object":"chat.completion.chunk","created":1760000000,"model":"glm-5.3-flash","choices":[{"index":0,"delta":{"role":"assistant","content":"","reasoning_content":""}}]}

data: {"id":"chatcmpl-stream-xxxx","object":"chat.completion.chunk","created":1760000000,"model":"glm-5.3-flash","choices":[{"index":0,"delta":{"content":"好"}}]}

data: {"id":"chatcmpl-stream-xxxx","object":"chat.completion.chunk","created":1760000000,"model":"glm-5.3-flash","choices":[{"delta":{},"finish_reason":"stop","index":0}]}

data: {"id":"chatcmpl-stream-xxxx","object":"chat.completion.chunk","created":1760000000,"model":"glm-5.3-flash","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":3,"total_tokens":21,"prompt_tokens_details":{"cached_tokens":0,"created_cache_tokens":0},"completion_tokens_details":{"reasoning_tokens":0,"audio_tokens":0,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"system_fingerprint":"fp_xxxx"}

data: [DONE]

Notes

• When you do not pass stream_options, the platform turns on include_usage automatically; usage is in the last chunk, whose choices is an empty array, so check its length before reading choices[0]. Some upstreams put usage in the last content chunk; go by whichever chunk has a non-empty usage.
• When thinking is on, thinking deltas come first as delta.reasoning_content, then delta.content.
• If a stream fails after output has started, the HTTP status is already 200 and the error is sent as an in-stream event; see "Errors" below.

Tool calling

When to use: let the model decide to call functions you define (look up weather, query a database, call an internal API). The flow: declare tools → model returns tool_calls → you run the function → send back a tool message → the model gives the final answer.

Evidence: tested (live test account, 2026-10-19): a single call, returning results, parallel calls, a forced function, streaming. tool_choice: "required" / "none" and parallel_tool_calls: false are per official docs (not tested).

Request (first turn)

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "What is the weather in Hangzhou tomorrow?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "parameters": {
          "type": "object",
          "properties": {"city": {"type": "string"}},
          "required": ["city"]
        }
      }
    }],
    "max_tokens": 1500
  }'

Response (first turn)

When the model wants a tool, finish_reason is tool_calls; arguments is a JSON string you must parse. Whether content is also present depends on the model (it is null here; the parallel example below has a short sentence).

{
  "id": "chatcmpl-xxxx",
  "object": "chat.completion",
  "created": 1760000000,
  "model": "glm-5.3-flash",
  "choices": [
    {
      "finish_reason": "tool_calls",
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "refusal": null,
        "tool_calls": [
          {
            "id": "chatcmpl-tool-xxxxxxxx",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"city\": \"杭州\"}"
            }
          }
        ],
        "reasoning_content": "The user is asking about the weather in Hangzhou tomorrow… (omitted)"
      }
    }
  ],
  "usage": {
    "prompt_tokens": 160,
    "completion_tokens": 77,
    "total_tokens": 237,
    "prompt_tokens_details": {
      "cached_tokens": 0,
      "created_cache_tokens": 0
    },
    "completion_tokens_details": {
      "reasoning_tokens": 65,
      "audio_tokens": 0,
      "accepted_prediction_tokens": 0,
      "rejected_prediction_tokens": 0
    }
  },
  "system_fingerprint": "fp_xxxx"
}

Returning the tool result (second turn)

Add the assistant message back to the history unchanged, then append a tool message matched by tool_call_id. If the assistant message has reasoning_content (thinking models), send it back too (see Thinking).

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [
      {"role": "user", "content": "What is the weather in Hangzhou tomorrow?"},
      {"role": "assistant", "content": null, "tool_calls": [
        {"id": "chatcmpl-tool-xxxxxxxx", "type": "function",
         "function": {"name": "get_weather", "arguments": "{\"city\": \"杭州\"}"}}
      ]},
      {"role": "tool", "tool_call_id": "chatcmpl-tool-xxxxxxxx", "content": "Cloudy, 7-13°C"}
    ],
    "tools": [ ... same as the first turn ... ],
    "max_tokens": 2000
  }'

The second-turn response is an ordinary success response (tested; finish_reason is stop):

{
  "id": "chatcmpl-xxxx",
  "object": "chat.completion",
  "created": 1760000000,
  "model": "glm-5.3-flash",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "杭州明天天气多云，气温在 7~13°C 之间。早晚较凉，建议适当添衣保暖哦。",
        "refusal": null
      }
    }
  ],
  "usage": {
    "prompt_tokens": 182,
    "completion_tokens": 28,
    "total_tokens": 210
  },
  "system_fingerprint": "fp_xxxx"
}

Parallel calls: ask about two cities in one request and the model returns two tool_calls in the same assistant message (tested). Return one tool message per call, each with its tool_call_id.

"message": {
  "role": "assistant",
  "content": "我来为您查询杭州和北京的当前天气。",
  "tool_calls": [
    {"id": "chatcmpl-tool-xxxxxxxx", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"杭州\"}"}},
    {"id": "chatcmpl-tool-yyyyyyyy", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"北京\"}"}}
  ]
}

Forced function: "tool_choice": {"type": "function", "function": {"name": "get_weather"}} forces the model to call that function even if the user only says "hello" (tested: it returned one get_weather call with an empty-string content).

Streaming tool calls: the first chunk carries id and name; arguments follow in later chunks, concatenated by index (tested). This upstream put all arguments in a single chunk; other models may split them into several fragments.

data: {"id":"chatcmpl-stream-xxxx","object":"chat.completion.chunk","created":1760000000,"model":"glm-5.3-flash","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"chatcmpl-tool-xxxxxxxx","type":"function","function":{"name":"get_weather"}}]}}]}

data: {"id":"chatcmpl-stream-xxxx","object":"chat.completion.chunk","created":1760000000,"model":"glm-5.3-flash","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"city\": \"杭州\"}"}}]}}]}

data: {"id":"chatcmpl-stream-xxxx","object":"chat.completion.chunk","created":1760000000,"model":"glm-5.3-flash","choices":[{"delta":{},"finish_reason":"tool_calls","index":0}]}

... (usage chunk and [DONE] as in basic streaming)

Notes

• parallel_tool_calls is forwarded to the upstream only when tools is also present.
• functions / function_call (the deprecated legacy fields) are not converted to tools / tool_choice; use tools directly.
• Schema constraints such as strict: true are forwarded to the upstream as-is; whether they are strictly enforced is up to the upstream.

JSON mode (json_object)

When to use: you only need syntactically valid JSON, with no fixed fields.

Evidence: tested (live test account, 2026-10-19).

Request

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Return the name and population (in 10 thousands) of Beijing as JSON, with keys city and population"}],
    "response_format": {"type": "json_object"},
    "max_tokens": 2000
  }'

Response (message.content is the JSON text; other fields as in basic chat)

"message": {
  "role": "assistant",
  "content": "{\n  \"city\": \"北京\",\n  \"population\": 2189\n}",
  "refusal": null,
  "reasoning_content": "… (omitted)"
}

Notes: OpenAI requires the word "JSON" to appear in the prompt; do the same. json_object only guarantees valid syntax, not fields; use the next section for field constraints.

Structured output (json_schema)

When to use: the output must strictly match a JSON Schema you provide.

Evidence: tested (live test account, 2026-10-19).

Request

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Extract: Zhang San, 28 years old, lives in Shanghai"}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
            "city": {"type": "string"}
          },
          "required": ["name", "age", "city"],
          "additionalProperties": false
        }
      }
    },
    "max_tokens": 2000
  }'

Response

"message": {
  "role": "assistant",
  "content": "{\n  \"age\": 28,\n  \"city\": \"上海\",\n  \"name\": \"张三\"\n}",
  "refusal": null,
  "reasoning_content": "… (omitted)"
}

Notes

• In this run the output matched the schema; key order is not guaranteed (alphabetical here).
• The platform forwards response_format to the upstream as-is and does not validate output on its side; whether strict is enforced depends on the upstream. Still validate the result yourself.
• When finish_reason is length, the output may be truncated; check before parsing.

Thinking / reasoning

When to use: you want the model to reason before answering, or you want to control reasoning effort.

Evidence: tested (live test account, 2026-10-19): glm-5.3-flash returns thinking by default, and with reasoning_effort: "low" thinking is essentially empty. The reasoning object and other thinking values are per official or vendor docs (not tested).

Request

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Reply with one character: 好"}],
    "reasoning_effort": "low",
    "max_tokens": 600
  }'

Response

By default the thinking is in message.reasoning_content (streaming: delta.reasoning_content, arriving before the answer), the thinking token count is in usage.completion_tokens_details.reasoning_tokens, and a full real response is under Basic chat (reasoning_tokens is 591). With reasoning_effort: "low" the tested reasoning_tokens was 0 and the response had no thinking text (see the response under Streaming).

Notes

• The platform never turns thinking on or off by default, and never fills in reasoning_effort for you: whatever you send is forwarded; if you send nothing, the upstream model's default applies. The one exception: for GPT / o-series models, thinking is converted to reasoning_effort (low / medium / high).
• thinking, enable_thinking and similar are not OpenAI standard fields; in the Python SDK put them in extra_body. See DeepSeek, GLM, Qwen and MiniMax for per-vendor values.
• An OpenRouter-style reasoning object in the request is normalized: reasoning.effort → reasoning_effort, reasoning.max_tokens → thinking.budget_tokens, reasoning.effort = "none" → thinking: disabled, reasoning.enabled = true with nothing else → reasoning_effort: medium. For models that do not accept medium, such as GLM-5.3, send the upstream-supported reasoning_effort directly instead of reasoning.enabled.
• Sending thinking back across turns: DeepSeek (required when tools is present), GLM, Qwen and MiniMax models read reasoning_content from earlier assistant messages, and the platform forwards it as-is, so your client must put the previous turn's value back into the history; other models drop the field. (This behavior depends on code that is not yet released.)

Image input (vision)

When to use: have a vision model answer questions about an image. Choose a model that accepts image input (such as qwen3-vl-plus); a text-only model rejects or ignores images.

Evidence: base64 images tested (live test account, 2026-10-19, qwen3-vl-plus); the public-URL form is per official docs (not tested).

Request

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-vl-plus",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "What is in this image? One sentence."},
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KGgo..."}}
      ]
    }],
    "max_tokens": 300
  }'

Response

{
  "id": "chatcmpl-xxxx",
  "object": "chat.completion",
  "created": 1760000000,
  "model": "qwen3-vl-plus",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "图中是一个红色的实心圆。",
        "refusal": null,
        "reasoning_content": ""
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 82,
    "completion_tokens": 10,
    "total_tokens": 92,
    "prompt_tokens_details": { "cached_tokens": 0, "text_tokens": 16, "image_tokens": 66 },
    "completion_tokens_details": {
      "reasoning_tokens": 0,
      "audio_tokens": 0,
      "accepted_prediction_tokens": 0,
      "rejected_prediction_tokens": 0,
      "text_tokens": 10
    }
  }
}

Notes

• Images are billed as input tokens; per-modality usage is in usage.prompt_tokens_details (image_tokens).
• A public URL must be reachable by the upstream; use base64 when unsure. Per-model limits on image size, count and video input are on the vendor pages.
• image_url.detail is kept and forwarded; image_url.format and file.format are non-standard sub-fields and are removed before forwarding.
• OpenAI's file_id (Files API) is not supported.

Prompt caching

When to use: many requests share a long prefix (system prompt, documents, tool definitions) and you want lower cost and latency.

Evidence: tested (live test account, 2026-10-19, deepseek-v4.1-flash): the same prefix of about 2,900 tokens sent twice in a row; the second request hit 2,816 tokens.

OpenAI's caching is automatic and has no request parameter. We do the same: you need to do nothing. The upstream caches automatically and the platform only reports the hit count in usage.prompt_tokens_details.cached_tokens.

Request: put stable content at the start of messages and changing content last, then send two requests with the same prefix.

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-v4.1-flash",
    "messages": [
      {"role": "system", "content": "You are a support assistant. Answer using these terms:\n(about 2,900 tokens of long text here)"},
      {"role": "user", "content": "What does clause 3 say? One sentence."}
    ],
    "max_tokens": 300,
    "reasoning_effort": "low"
  }'

Response (usage only; first request)

"usage": {
  "prompt_tokens": 2903,
  "completion_tokens": 82,
  "total_tokens": 2985,
  "prompt_tokens_details": { "cached_tokens": 0 },
  "completion_tokens_details": { "reasoning_tokens": 57 }
}

Second request (cache hit):

"usage": {
  "prompt_tokens": 2903,
  "completion_tokens": 123,
  "total_tokens": 3026,
  "prompt_tokens_details": { "cached_tokens": 2816 },
  "completion_tokens_details": { "reasoning_tokens": 98 }
}

Notes

• Whether it hits, the minimum cacheable length and the retention time are decided by the upstream; cached_tokens is the number the upstream reports. Hit tokens are billed at the cache price.
• Explicit cache markers: add cache_control: {"type": "ephemeral"} on a content block or message. It is kept only for claude / gemini-prefixed models and Qwen models (Qwen normalizes it to {"type":"ephemeral"} and drops ttl); for other models cache_control is removed. Qwen explicit caching depends on code that is not yet released.
• For upstreams that write caches, usage can also show prompt_tokens_details.cache_write_tokens and cache_write_token_details (split by 5-minute / 1-hour TTL).

Common parameters: n, stop, logprobs

When to use: get several candidates at once, stop at a given string, or read token probabilities.

Evidence: tested (live test account, 2026-10-19, glm-5.3-flash): n: 2 together with stop, and logprobs alone. seed, temperature, top_p and so on are per official docs (not tested); whether they take effect depends on the upstream.

Request

curl https://api.modelsite.ai/v1/chat/completions \
  -H "Authorization: Bearer $MODELSITE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash",
    "messages": [{"role": "user", "content": "Count from 1 to 10 separated by commas, and stop at 5"}],
    "max_tokens": 200,
    "reasoning_effort": "low",
    "n": 2,
    "stop": ["6"]
  }'

Response (n: 2, stop: ["6"]; choices has two items)

"choices": [
  {
    "finish_reason": "stop",
    "index": 0,
    "message": { "role": "assistant", "content": "1, 2, 3, 4, 5\n\n已数到5，停止输出。", "refusal": null, "reasoning_content": "Simple." }
  },
  {
    "finish_reason": "stop",
    "index": 1,
    "message": { "role": "assistant", "content": "1, 2, 3, 4, 5", "refusal": null, "reasoning_content": "Simple." }
  }
]

With logprobs: true, each choice gets an extra logprobs.content[]; each item has token, logprob, bytes and top_logprobs (top_logprobs: 2 here).

Notes

• The platform does not restrict n > 1; cost is charged on the real usage of all candidates.
• For thinking models, logprobs includes thinking-phase tokens (in testing the first token was the upstream's end-of-thinking marker); do not assume it covers only the answer.
• A string stop is converted to a one-element array before forwarding. top_logprobs ranges from 0 to 20 officially.
• logprobs: false is the same as omitting it.

Errors

401 invalid or missing key

{"error":{"type":"authentication_error","message":"invalid API key"}}

When the Authorization header is missing, message is missing or invalid authorization header.

402 insufficient balance

{"error":{"type":"insufficient_quota","message":"insufficient balance"}}

429 rate limited

{"error":{"type":"rate_limit_error","message":"too many requests"}}

The response carries Retry-After (seconds); back off accordingly.

403 no access to this model

{"error":{"type":"model_access_error","message":"model 'glm-5.3-flash' is not allowed for this API key"}}

503 gateway at capacity

{"error":{"type":"overloaded_error","message":"gateway at capacity, retry later"}}

Also carries Retry-After: 5.

502 upstream temporarily unavailable (seen in testing)

{"error":{"type":"server_error","message":"upstream service unavailable"}}

503 no upstream available right now (seen in testing)

{"error":{"code":"no_upstream_available","message":"<message text>","type":"service_unavailable"}}

Appears when all upstreams for a model are in temporary cooldown; retry shortly. The message text of these two errors may change; go by the HTTP status and error.type / error.code.

400 unknown model

{"error":{"type":"invalid_request_error","message":"unknown model: not-a-model"}}

A wrong or not-yet-live model is a 400 (not 404).

400 invalid parameters

{"error":{"message":"messages: messages is required and must not be empty","type":"invalid_request_error","code":"invalid_request_error"}}

After the request passes the gateway the routing layer validates it, and its error body has an extra code field.

If a streaming request fails after output has started, the HTTP status is already 200 and the error is sent as an in-stream event: data: {"error":{"message":"The upstream stream ended unexpectedly before completion.","type":"server_error"}}, followed by data: [DONE]. Treat an error chunk as a failure; do not treat what you already received as a complete reply. Full error codes and handling advice: Errors.

Unsupported or different parameters

The platform does not drop parameters against a parameter allowlist and does not clamp values: apart from the hard-coded rewrites listed in the table below, the request body is forwarded to the upstream as-is, and whether a parameter takes effect, is ignored or is rejected (4xx) is up to the upstream. Different models support the same parameter differently, so verify by testing. Top-level fields the platform does not recognize, such as thinking, enable_thinking and enable_search, are also forwarded unchanged; in the Python SDK put them in extra_body.

Platform extensions

What follows is defined by the platform and does not exist in the official OpenAI API.

Request parameters

Request headers

The gateway only lets these request headers through and drops all others: Authorization, Content-Type, Accept, Accept-Encoding, Origin, User-Agent, Anthropic-Beta, OpenAI-Beta. A value you send in X-Request-ID is kept and echoed in the response header; if absent, the platform generates one. There are no other custom request headers, and internal headers for identity, organization or price cannot be forged by callers.

Response headers

The platform does not return X-RateLimit-* quota headers; the only rate-limit information is the 429 status and Retry-After.

Response fields

Automatic behavior

• Streaming requests without stream_options get include_usage turned on automatically.
• When the balance runs out during streaming, the stream is cut and you receive data: {"error":{"type":"insufficient_quota","message":"balance exhausted"}}, followed by data: [DONE].
• A request body over the size limit returns 413 request body too large.
• Automatic retry and upstream switching on upstream failure are described under "Errors" above.