Integration Guides

OpenAI-Compatible API Response Format: Fields and response_format

ROIBest AI

Two things people mean by "response format"

Search for openai compatible api response format and you land on two different questions:

  1. What does the response look like? The JSON object an OpenAI-compatible /v1/chat/completions endpoint sends back, and which fields your code can rely on.
  2. What does the response_format parameter do? The request field that asks for plain text, JSON mode (json_object) or Structured Outputs (json_schema), and whether a given provider honours it.

This page answers both, then shows how to write one client that survives a provider switch. It builds on what an OpenAI-compatible API is and the wider list of compatibility issues; here we stay on response shape and response_format only.

Facts come from each vendor's own documentation, read on November 20, 2026: OpenAI's Structured Outputs guide, Anthropic's OpenAI SDK compatibility page, Google's Gemini OpenAI compatibility page, DeepSeek's JSON Output guide, vLLM's structured outputs docs and Ollama's OpenAI compatibility page. Support changes often; when this page and the docs disagree, trust the docs.

The response object: fields you can rely on

A non-streaming chat completion from an OpenAI-compatible server has the same skeleton:

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1763600000,
  "model": "provider-model-name",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "...", "tool_calls": null },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 21, "completion_tokens": 48, "total_tokens": 69 }
}

The safe core is choices[0].message.content, choices[0].message.tool_calls, choices[0].finish_reason and the three usage counters. Everything else is "may be present". Anthropic's compatibility table shows how far this goes: id, choices[].message.content, tool_calls, model and the usage token counts are fully supported, while usage.prompt_tokens_details, usage.completion_tokens_details, choices[].message.refusal, logprobs, service_tier and system_fingerprint are always empty.

Three practical rules follow:

  • Read finish_reason before parsing. length means the output was cut at the token limit, so JSON will usually be incomplete.
  • Treat optional fields as optional. Code that reads usage.prompt_tokens_details.cached_tokens without a null check breaks on providers that leave it empty.
  • Do not rely on refusal everywhere. On OpenAI, Structured Outputs reports safety refusals in a refusal field; on Anthropic's compatibility layer that field is always empty.

Streaming uses the same fields split across chat.completion.chunk events with choices[].delta; the differences there are covered in the compatibility issues guide.

The response_format parameter: three modes

On Chat Completions, response_format takes one of three shapes:

Mode

Request value

What it guarantees

Text

{"type": "text"} (or omit)

Nothing; free-form text

JSON mode

{"type": "json_object"}

Valid JSON, but no particular shape

Structured Outputs

{"type": "json_schema", "json_schema": {"name": "...", "schema": {...}, "strict": true}}

Output follows your JSON Schema (within the supported subset)

OpenAI describes Structured Outputs as the evolution of JSON mode: both produce valid JSON, only Structured Outputs enforces the schema, and OpenAI recommends it whenever possible. On the newer Responses API the same setting lives under text.format instead of response_format, which matters if a provider only implements Chat Completions.

Provider support at a glance

Provider (OpenAI-compatible endpoint)

json_object

json_schema

Notes

OpenAI

Yes

Yes, with strict: true

Schema must fit OpenAI's supported JSON Schema subset; its examples list every field in required and set additionalProperties: false

Anthropic (api.anthropic.com/v1/)

No, returns 400

Yes, only with strict: true

text also returns 400; Anthropic positions the layer for testing and comparison, not as a long-term production path

Google Gemini (.../v1beta/openai/)

Not covered on the compat page

Yes

The official example passes a Pydantic model as response_format through the SDK's parse helper

DeepSeek (api.deepseek.com)

Yes

Not documented

Prompt must contain the word "json" plus an example; unparseable output can come back as blank content

vLLM (self-hosted)

Yes

Yes

Also accepts its own structured_outputs field; the old guided_json fields were removed in v0.12.0

Ollama (localhost:11434/v1/)

Yes (JSON mode)

response_format listed as supported

The local server ignores the API key value

Two patterns stand out. First, json_object is not universal: a client that hard-codes JSON mode fails outright on Anthropic's layer. Second, json_schema is the closest thing to a common denominator, but only if your schema stays inside every target provider's supported subset.

Provider quirks that bite

DeepSeek's blank content. DeepSeek's JSON Output guide says that if the model's output cannot be recognised as JSON, the API replaces the non-JSON content with spaces, so content may look empty even when finish_reason is stop. Handle blank content and retry. The same guide asks you to set max_tokens high enough to avoid truncation.

Anthropic's strictness. Only json_schema with strict: true is accepted, and the structured outputs schema limitations apply. Separately, strict on function definitions is ignored there, so tool-call arguments are not guaranteed to match their schema.

vLLM's field rename. If you upgraded vLLM and requests that used guided_json, guided_regex or guided_choice stopped working, move them to structured_outputs, for example {"structured_outputs": {"json": ...}}. Plain response_format with json_schema keeps working.

Gateways in between. A gateway or OpenAI API proxy may pass response_format through untouched, translate it, or drop it. Test through the exact base URL your app uses, not the upstream directly.

A portable pattern

To keep one code path across providers:

  1. Prefer json_schema with strict: true, and keep the schema simple: an object root, every property listed in required, additionalProperties: false, no exotic keywords.
  2. Still say "JSON" in the prompt and give a short example. It costs nothing and is required on DeepSeek.
  3. Check finish_reason first, then check content for blank or truncated output.
  4. Validate after parsing. OpenAI's own guide reminds you to handle refusals, incomplete responses and errors, and to check business rules the schema cannot express.
  5. Keep a fallback. If a provider rejects json_schema with HTTP 400, retry with json_object where supported, or with no response_format plus prompt instructions and validation.
import json
from openai import OpenAI

client = OpenAI(base_url="https://your-gateway.example/v1", api_key="...")

schema = {
    "type": "object",
    "properties": {"name": {"type": "string"}, "date": {"type": "string"}},
    "required": ["name", "date"],
    "additionalProperties": False,
}

resp = client.chat.completions.create(
    model="your-model",
    messages=[
        {"role": "system", "content": "Extract the event. Reply in JSON like {\"name\": \"...\", \"date\": \"...\"}."},
        {"role": "user", "content": "Alice and Bob meet at the science fair on Friday."},
    ],
    response_format={"type": "json_schema",
                     "json_schema": {"name": "event", "schema": schema, "strict": True}},
    max_tokens=300,
)

choice = resp.choices[0]
if choice.finish_reason != "stop" or not (choice.message.content or "").strip():
    raise RuntimeError(f"incomplete or empty output: {choice.finish_reason}")
event = json.loads(choice.message.content)

Verify before you switch providers

Run the same three requests against the new base URL and compare:

  • a plain request, checking choices[0].message.content and usage;
  • the same request with response_format: {"type": "json_object"}, checking for a 400 or valid JSON;
  • the same request with your real json_schema, checking that the output parses and passes validation.

If you route several models through one endpoint, the ai.roibest gateway exposes them behind a single OpenAI-compatible /v1 base URL, but the per-model support above still applies, so keep the three-request check in your release list.

FAQ

What is the response format of an OpenAI-compatible API?

A chat completion returns an object with id, object: "chat.completion", created, model, a choices array (each with message.content, optional message.tool_calls, and finish_reason) and a usage block with prompt, completion and total tokens. Other fields are optional, and some providers return them empty.

What is the difference between json_object and json_schema?

json_object (JSON mode) only guarantees valid JSON. json_schema with strict: true (Structured Outputs) also makes the output follow the schema you supply. OpenAI recommends Structured Outputs where possible.

Does Claude support response_format through the OpenAI SDK?

Partly. Anthropic's compatibility layer accepts only json_schema with strict: true; json_object and text return a 400 error.

Why does DeepSeek return empty content in JSON mode?

When the output cannot be recognised as JSON, DeepSeek replaces the non-JSON content with spaces. Include the word "json" and an example in the prompt, set max_tokens high enough, and retry on blank content.

Does vLLM still support guided_json?

No. The guided_json, guided_regex and guided_choice fields were removed in vLLM v0.12.0. Use response_format with json_schema, or vLLM's structured_outputs field.

The short version

The response shape is stable at the core (choices[0].message.content, finish_reason, usage) and unreliable at the edges. For response_format, json_schema with a simple strict schema is the most portable choice, json_object is not supported everywhere, and every provider still needs a finish-reason check, a blank-content check and validation after parsing.