What Is an OpenAI-Compatible API? Endpoints, Clients, and How to Switch

What an OpenAI-compatible API actually is

An OpenAI-compatible API is any inference endpoint that accepts the same HTTP request shape as OpenAI's Chat Completions API and returns the same response shape. It is not an OpenAI product, and it does not imply any relationship with OpenAI. It is a contract: if a server accepts POST /v1/chat/completions with a model, a messages array, and a bearer token, and answers with a choices[].message.content object, then every client written for OpenAI already works against it.

That contract became the de-facto standard for a boring reason — it was the first widely adopted one, so tooling was written against it first. Today most inference servers (vLLM, llama.cpp, Ollama, Together, Groq, OpenRouter, and gateway services such as ROIBest AI) expose it, whatever runs behind the scenes.

The practical consequence is the point of this article: switching providers is a configuration change, not a rewrite.

The endpoints that make up the compatible surface

"OpenAI-compatible" is a spectrum, not a certification. Almost every provider implements the first two rows below; support gets thinner as you go down.

When someone says "point it at an OpenAI-compatible endpoint," they nearly always mean a base URL that serves /v1/chat/completions. Everything else is a bonus you should verify rather than assume.

How to point an existing client at one

Every official OpenAI SDK exposes a base URL override. The API key and the base URL are the only two things that change.

Python

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_KEY",
    base_url="https://your-endpoint.example/v1",
)

resp = client.chat.completions.create(
    model="the-model-id-your-provider-lists",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
)
print(resp.choices[0].message.content)

Node.js

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: "https://your-endpoint.example/v1",
});

const resp = await client.chat.completions.create({
  model: "the-model-id-your-provider-lists",
  messages: [{ role: "user", content: "Say hello in one sentence." }],
});
console.log(resp.choices[0].message.content);

Environment variables

Most CLI tools and IDE extensions read these two, which means you often do not have to touch code at all:

export OPENAI_BASE_URL="https://your-endpoint.example/v1"
export OPENAI_API_KEY="YOUR_KEY"

Where compatibility usually breaks

Compatibility failures are rarely dramatic. They are almost always one of these six, in roughly this order of frequency:

1. A doubled or missing /v1. Some SDKs append /v1 themselves, some do not. https://host/v1/v1/chat/completions returns a 404 that looks like an auth problem. Check the actual request path before checking anything else.
2. Model IDs are not portable. gpt-4o is an OpenAI identifier. A compatible endpoint accepts whatever names it publishes — call GET /v1/models and copy one from the list rather than guessing.
3. Unsupported optional parameters. seed, logprobs, response_format, logit_bias, and n > 1 are the usual casualties. Some servers ignore them silently, others return a 400. Silent ignoring is the more dangerous of the two, because determinism you think you have is determinism you do not have.
4. Tool calling has real dialect differences. The tools / tool_calls schema is widely implemented, but parallel tool calls, streamed partial arguments, and strict JSON-schema enforcement vary a lot. Test the exact shape your agent depends on.
5. Streaming chunk details. Everyone sends data: {...} SSE lines terminated by data: [DONE], but the presence of a final usage chunk, of empty keep-alive lines, and of finish_reason on the last delta all differ. Parsers that assume one specific arrangement break.
6. Usage and rate-limit metadata. usage.prompt_tokens is usually present; x-ratelimit-* response headers frequently are not. Any code that reads those headers to schedule retries needs a fallback.

Verifying an endpoint in three curl calls

Before wiring a compatible endpoint into an application, confirm it in the terminal. These three calls take a minute and isolate auth, inference, and streaming from each other.

# 1. Auth + model discovery
curl -s https://your-endpoint.example/v1/models \
  -H "Authorization: Bearer $OPENAI_API_KEY"

# 2. A non-streaming round trip
curl -s https://your-endpoint.example/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"MODEL_ID","messages":[{"role":"user","content":"ping"}]}'

# 3. The same request, streamed
curl -N -s https://your-endpoint.example/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"MODEL_ID","messages":[{"role":"user","content":"ping"}],"stream":true}'

If call 1 works and call 2 fails, the problem is the model ID. If 2 works and 3 hangs, something between you and the endpoint is buffering the response — a proxy, a load balancer, or a client library with buffering enabled.

What speaks this dialect

The compatible surface is why a single endpoint can serve very different tools at once: coding CLIs and IDE extensions, orchestration frameworks like LangChain and LlamaIndex, self-hosted chat front-ends, and one-off scripts. Each of them takes a base URL and a key.

Two integrations are worth calling out because they are asked about constantly. Codex reads the standard base-URL and key configuration, which is walked through step by step in Connect Codex to ROIBest AI. Clients built for Anthropic's message format — Claude Code among them — do not speak this dialect natively and need a translating layer in front; that mechanism is covered in Claude API Proxy.

Frequently asked questions

Is an OpenAI-compatible API the same as OpenAI's API? No. It copies the request and response format so that existing clients work. The models, pricing, rate limits, data handling, and available parameters are entirely the provider's own.

Do I have to change my application code? Usually only the base URL, the key, and the model ID. If your code reads rate-limit headers, relies on seed determinism, or parses streaming chunks by hand, budget time to check those three.

Does function/tool calling work? The basic tools request and tool_calls response are widely supported. Parallel calls and strict schema enforcement are where implementations diverge — verify with the exact tool definitions your agent uses, not a toy example.

How do I know which models an endpoint offers? GET /v1/models is the authoritative answer for that endpoint. Documentation goes stale; the models list does not.

Can one key serve several different tools? Yes. That is the main practical benefit: one base URL and one key configured across your CLI, your editor, and your scripts, instead of a separate integration per tool.

The short version

An OpenAI-compatible API is a shape, not a brand. Confirm the base URL resolves with GET /v1/models, take the model ID from that list, run one streaming and one non-streaming request, and check the handful of optional parameters your code actually depends on. Once those pass, the rest of your stack is already compatible.

ROIBest AI exposes an OpenAI-compatible endpoint at https://ai.roibest.com, usable from the OpenAI SDKs, compatible CLIs, and any client that accepts a custom base URL. The integration guides cover the per-tool configuration.