OpenAI-Compatible Endpoints

Connect any endpoint that speaks the OpenAI /v1 API — vLLM, Ollama, LM Studio, LocalAI, Groq, Together, and more.

Add Settings → Platform → Extensions → Add extension → AI providers → OpenAI-compatible server to connect any server that implements the OpenAI /v1 API — vLLM, Ollama, LM Studio, LocalAI, Groq, Together AI, and more.

This is the recommended path for:

Self-hosted inference servers (Ollama, vLLM, LM Studio) in fully local or air-gapped deployments.
Third-party OpenAI-compatible hosted services (Groq, Together AI, Perplexity, etc.) that are not listed as a named extension.
Custom or internal endpoints that implement the OpenAI /v1 contract.

Configure

Navigate to Settings → Platform → Extensions → Add extension, find OpenAI-compatible server under AI providers (or search for it), and click Connect. The connect step asks for:

Capabilities — tick which capabilities this endpoint provides (LLM, Embeddings, STT, TTS, Image, Moderation). A best-effort GET /v1/models probe pre-fills available models; if your server doesn't implement /models, you can enter model IDs manually.
Configure — set Base URL and optional authentication.
Verify — the wizard tests reachability before saving.

Configuration fields

Field	Required	Notes
Base URL	Yes	The `/v1` root, e.g. `https://api.groq.com/openai/v1`. The `/v1` suffix is appended automatically if omitted.
Authentication	No	Choose API Key (sent as `Authorization: Bearer <key>`, stored encrypted) or No authentication for keyless local servers such as Ollama.
Content-Type	No (advanced)	Defaults to `application/json`. Leave blank unless your server requires a non-standard value. Audio uploads automatically use `multipart/form-data` regardless of this setting.

Capability coverage

Capability	OpenAI path	Status
Chat / LLM completions	`POST /chat/completions`	✅ Available
Embeddings	`POST /embeddings`	✅ Available
Model discovery	`GET /models`	✅ Available (best-effort probe)
Transcription / translation	`POST /audio/transcriptions`, `/audio/translations`	✅ Available
Text-to-speech	`POST /audio/speech`	✅ Available
Image generation	`POST /images/generations`	✅ Available
Content moderation	`POST /moderations`	✅ Available

All six capabilities — plus best-effort model discovery — support keyless (no-auth) endpoints. Choose No authentication in the wizard for local servers such as Ollama, Whisper.cpp, or any self-hosted endpoint that does not require an API key.

One Base URL can serve multiple capabilities simultaneously — for example, a vLLM instance that handles both chat completions and embeddings. Tick all relevant capabilities in the wizard step and they resolve through the same connection.

Multiple endpoints

If you need separate servers for different capabilities (e.g. a dedicated audio transcription service alongside a chat server), add a second OpenAI Compatible connection. The platform supports multiple named connections per vendor and lets you select which connection serves each capability.

Common self-hosted configurations

Ollama (fully local / air-gapped)

Field	Value
Base URL	`http://localhost:11434/v1` (or your internal host)
Authentication	No authentication

Ollama exposes GET /v1/models so the wizard can discover available models automatically.

Local and loopback endpoints are reachable by default: the Private & Loopback Networks toggle under Settings → Governance → Egress ships enabled. If your administrator has turned it off, connections to localhost/private-IP endpoints fail until it is re-enabled. Cloud metadata endpoints are always blocked, regardless of this setting.

vLLM (self-hosted GPU inference)

Field	Value
Base URL	`https://your-vllm-host/v1`
Authentication	API Key (if your vLLM instance requires one)

Groq / Together AI / hosted providers

Field	Value
Base URL	Provider's `/v1` root (e.g. `https://api.groq.com/openai/v1`)
Authentication	API Key (your provider key)

Troubleshooting

412 LLM_NOT_CONFIGURED / 412 EMBEDDING_NOT_CONFIGURED / 412 STT_NOT_CONFIGURED / 412 TTS_NOT_CONFIGURED / 412 IMAGE_NOT_CONFIGURED

The endpoint is not set as the org's default for that capability, or no connection is enabled for it. Go to Settings → Platform → Extensions, open the OpenAI Compatible connection, and ensure the relevant capability product is enabled and set as default.

Model not listed after adding the connection

If your server doesn't implement GET /v1/models, the auto-discovery probe returns nothing. You can enter model IDs manually in the capability settings after the connection is saved.

A reason is shown beside the connection instead of a model list

The server never answered the probe: the Base URL does not resolve, the port refuses connections, or the endpoint presents a TLS certificate the platform cannot verify (a self-signed or internal-CA certificate is the usual cause). Every model call on that connection fails the same way, so the platform reports the cause rather than an empty list. The message names the endpoint and the transport's own text. Fix the address, or put a certificate from a trusted authority in front of the server, then select Refresh models. A certificate problem is not retried on its own; a model call on the connection answers CAPABILITY_NOT_CONFIGURED until the endpoint is corrected.

Connection verification fails

Check that:

The Base URL is reachable from the platform's network (not localhost unless the server and the platform run on the same host).
The API key is correct (if authentication is enabled).
Self-signed TLS certificates are trusted by the platform's trust store (operator-level configuration).

For fully air-gapped deployments, see the air-gapped deployment guide. Ollama or vLLM via this extension is the standard configuration.

OpenAI-Compatible Endpoints

On this page