OpenAI-Compatible API Response Format: Fields and response_format
Two things people mean by "response format"
Search for openai compatible api response format and you land on two different questions:
- What does the response look like? The JSON object an OpenAI-compatible
/v1/chat/completionsendpoint sends back, and which fields your code can rely on. - What does the
response_formatparameter do? The request field that asks for plain text, JSON mode (json_object) or Structured Outputs (json_schema), and whether a given provider honours it.
This page answers both, then shows how to write one client that survives a provider switch. It builds on what an OpenAI-compatible API is and the wider list of compatibility issues; here we stay on response shape and response_format only.
Facts come from each vendor's own documentation, read on November 20, 2026: OpenAI's Structured Outputs guide, Anthropic's OpenAI SDK compatibility page, Google's Gemini OpenAI compatibility page, DeepSeek's JSON Output guide, vLLM's structured outputs docs and Ollama's OpenAI compatibility page. Support changes often; when this page and the docs disagree, trust the docs.
The response object: fields you can rely on
A non-streaming chat completion from an OpenAI-compatible server has the same skeleton:
{
"id": "chatcmpl-...",
"object": "chat.completion",
"created": 1763600000,
"model": "provider-model-name",
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "...", "tool_calls": null },
"finish_reason": "stop"
}
],
"usage": { "prompt_tokens": 21, "completion_tokens": 48, "total_tokens": 69 }
}The safe core is choices[0].message.content, choices[0].message.tool_calls, choices[0].finish_reason and the three usage counters. Everything else is "may be present". Anthropic's compatibility table shows how far this goes: id, choices[].message.content, tool_calls, model and the usage token counts are fully supported, while usage.prompt_tokens_details, usage.completion_tokens_details, choices[].message.refusal, logprobs, service_tier and system_fingerprint are always empty.
Three practical rules follow:
- Read
finish_reasonbefore parsing.lengthmeans the output was cut at the token limit, so JSON will usually be incomplete. - Treat optional fields as optional. Code that reads
usage.prompt_tokens_details.cached_tokenswithout a null check breaks on providers that leave it empty. - Do not rely on
refusaleverywhere. On OpenAI, Structured Outputs reports safety refusals in arefusalfield; on Anthropic's compatibility layer that field is always empty.
Streaming uses the same fields split across chat.completion.chunk events with choices[].delta; the differences there are covered in the compatibility issues guide.
The response_format parameter: three modes
On Chat Completions, response_format takes one of three shapes:
|
Mode |
Request value |
What it guarantees |
|---|---|---|
|
Text |
|
Nothing; free-form text |
|
JSON mode |
|
Valid JSON, but no particular shape |
|
Structured Outputs |
|
Output follows your JSON Schema (within the supported subset) |
OpenAI describes Structured Outputs as the evolution of JSON mode: both produce valid JSON, only Structured Outputs enforces the schema, and OpenAI recommends it whenever possible. On the newer Responses API the same setting lives under text.format instead of response_format, which matters if a provider only implements Chat Completions.
Provider support at a glance
|
Provider (OpenAI-compatible endpoint) |
|
|
Notes |
|---|---|---|---|
|
OpenAI |
Yes |
Yes, with |
Schema must fit OpenAI's supported JSON Schema subset; its examples list every field in |
|
Anthropic ( |
No, returns 400 |
Yes, only with |
|
|
Google Gemini ( |
Not covered on the compat page |
Yes |
The official example passes a Pydantic model as |
|
DeepSeek ( |
Yes |
Not documented |
Prompt must contain the word "json" plus an example; unparseable output can come back as blank content |
|
vLLM (self-hosted) |
Yes |
Yes |
Also accepts its own |
|
Ollama ( |
Yes (JSON mode) |
|
The local server ignores the API key value |
Two patterns stand out. First, json_object is not universal: a client that hard-codes JSON mode fails outright on Anthropic's layer. Second, json_schema is the closest thing to a common denominator, but only if your schema stays inside every target provider's supported subset.
Provider quirks that bite
DeepSeek's blank content. DeepSeek's JSON Output guide says that if the model's output cannot be recognised as JSON, the API replaces the non-JSON content with spaces, so content may look empty even when finish_reason is stop. Handle blank content and retry. The same guide asks you to set max_tokens high enough to avoid truncation.
Anthropic's strictness. Only json_schema with strict: true is accepted, and the structured outputs schema limitations apply. Separately, strict on function definitions is ignored there, so tool-call arguments are not guaranteed to match their schema.
vLLM's field rename. If you upgraded vLLM and requests that used guided_json, guided_regex or guided_choice stopped working, move them to structured_outputs, for example {"structured_outputs": {"json": ...}}. Plain response_format with json_schema keeps working.
Gateways in between. A gateway or OpenAI API proxy may pass response_format through untouched, translate it, or drop it. Test through the exact base URL your app uses, not the upstream directly.
A portable pattern
To keep one code path across providers:
- Prefer
json_schemawithstrict: true, and keep the schema simple: an object root, every property listed inrequired,additionalProperties: false, no exotic keywords. - Still say "JSON" in the prompt and give a short example. It costs nothing and is required on DeepSeek.
- Check
finish_reasonfirst, then checkcontentfor blank or truncated output. - Validate after parsing. OpenAI's own guide reminds you to handle refusals, incomplete responses and errors, and to check business rules the schema cannot express.
- Keep a fallback. If a provider rejects
json_schemawith HTTP 400, retry withjson_objectwhere supported, or with noresponse_formatplus prompt instructions and validation.
import json
from openai import OpenAI
client = OpenAI(base_url="https://your-gateway.example/v1", api_key="...")
schema = {
"type": "object",
"properties": {"name": {"type": "string"}, "date": {"type": "string"}},
"required": ["name", "date"],
"additionalProperties": False,
}
resp = client.chat.completions.create(
model="your-model",
messages=[
{"role": "system", "content": "Extract the event. Reply in JSON like {\"name\": \"...\", \"date\": \"...\"}."},
{"role": "user", "content": "Alice and Bob meet at the science fair on Friday."},
],
response_format={"type": "json_schema",
"json_schema": {"name": "event", "schema": schema, "strict": True}},
max_tokens=300,
)
choice = resp.choices[0]
if choice.finish_reason != "stop" or not (choice.message.content or "").strip():
raise RuntimeError(f"incomplete or empty output: {choice.finish_reason}")
event = json.loads(choice.message.content)Verify before you switch providers
Run the same three requests against the new base URL and compare:
- a plain request, checking
choices[0].message.contentandusage; - the same request with
response_format: {"type": "json_object"}, checking for a 400 or valid JSON; - the same request with your real
json_schema, checking that the output parses and passes validation.
If you route several models through one endpoint, the ai.roibest gateway exposes them behind a single OpenAI-compatible /v1 base URL, but the per-model support above still applies, so keep the three-request check in your release list.
FAQ
What is the response format of an OpenAI-compatible API?
A chat completion returns an object with id, object: "chat.completion", created, model, a choices array (each with message.content, optional message.tool_calls, and finish_reason) and a usage block with prompt, completion and total tokens. Other fields are optional, and some providers return them empty.
What is the difference between json_object and json_schema?
json_object (JSON mode) only guarantees valid JSON. json_schema with strict: true (Structured Outputs) also makes the output follow the schema you supply. OpenAI recommends Structured Outputs where possible.
Does Claude support response_format through the OpenAI SDK?
Partly. Anthropic's compatibility layer accepts only json_schema with strict: true; json_object and text return a 400 error.
Why does DeepSeek return empty content in JSON mode?
When the output cannot be recognised as JSON, DeepSeek replaces the non-JSON content with spaces. Include the word "json" and an example in the prompt, set max_tokens high enough, and retry on blank content.
Does vLLM still support guided_json?
No. The guided_json, guided_regex and guided_choice fields were removed in vLLM v0.12.0. Use response_format with json_schema, or vLLM's structured_outputs field.
The short version
The response shape is stable at the core (choices[0].message.content, finish_reason, usage) and unreliable at the edges. For response_format, json_schema with a simple strict schema is the most portable choice, json_object is not supported everywhere, and every provider still needs a finish-reason check, a blank-content check and validation after parsing.