OpenAI-Compatible Endpoints
Connect any endpoint that speaks the OpenAI /v1 API — vLLM, Ollama, LM Studio, LocalAI, Groq, Together, and more.
Add Settings → Platform → Extensions → Add extension → AI providers → OpenAI-compatible server to connect
any server that implements the OpenAI /v1 API — vLLM, Ollama, LM Studio, LocalAI, Groq,
Together AI, and more.
This is the recommended path for:
- Self-hosted inference servers (Ollama, vLLM, LM Studio) in fully local or air-gapped deployments.
- Third-party OpenAI-compatible hosted services (Groq, Together AI, Perplexity, etc.) that are not listed as a named extension.
- Custom or internal endpoints that implement the OpenAI
/v1contract.
Configure
Navigate to Settings → Platform → Extensions → Add extension, find OpenAI-compatible server under AI providers (or search for it), and click Connect. The connect step asks for:
- Capabilities — tick which capabilities this endpoint provides (LLM, Embeddings, STT,
TTS, Image, Moderation). A best-effort
GET /v1/modelsprobe pre-fills available models; if your server doesn't implement/models, you can enter model IDs manually. - Configure — set Base URL and optional authentication.
- Verify — the wizard tests reachability before saving.
Configuration fields
| Field | Required | Notes |
|---|---|---|
| Base URL | Yes | The /v1 root, e.g. https://api.groq.com/openai/v1. The /v1 suffix is appended automatically if omitted. |
| Authentication | No | Choose API Key (sent as Authorization: Bearer <key>, stored encrypted) or No authentication for keyless local servers such as Ollama. |
| OpenAI API | No | auto (blank), chat-completions, or responses. See OpenAI API. |
| Content-Type | No (advanced) | Defaults to application/json. Leave blank unless your server requires a non-standard value. Audio uploads automatically use multipart/form-data regardless of this setting. |
Capability coverage
| Capability | OpenAI path | Status |
|---|---|---|
| Chat / LLM completions | POST /chat/completions | ✅ Available |
| Embeddings | POST /embeddings | ✅ Available |
| Model discovery | GET /models | ✅ Available (best-effort probe) |
| Transcription / translation | POST /audio/transcriptions, /audio/translations | ✅ Available |
| Text-to-speech | POST /audio/speech | ✅ Available |
| Image generation | POST /images/generations | ✅ Available |
| Content moderation | POST /moderations | ✅ Available |
All six capabilities — plus best-effort model discovery — support keyless (no-auth) endpoints. Choose No authentication in the wizard for local servers such as Ollama, Whisper.cpp, or any self-hosted endpoint that does not require an API key.
One Base URL can serve multiple capabilities simultaneously — for example, a vLLM instance that handles both chat completions and embeddings. Tick all relevant capabilities in the wizard step and they resolve through the same connection.
Multiple endpoints
If you need separate servers for different capabilities (e.g. a dedicated audio transcription service alongside a chat server), add a second OpenAI Compatible connection. The platform supports multiple named connections per vendor and lets you select which connection serves each capability.
Common self-hosted configurations
Ollama (fully local / air-gapped)
| Field | Value |
|---|---|
| Base URL | http://ollama.internal:11434/v1 (your Ollama host on a private network) |
| Authentication | No authentication |
Ollama exposes GET /v1/models so the wizard can discover available models automatically.
A private address is reachable only when your Scrydon operator has opened it to your
organization as a private network (global.egress.privateNetworks, with the port Ollama
listens on) and the Private networks switch under Settings → Governance → Egress
is on. localhost is never reachable: from the platform it is its own container, not your
machine. Cloud metadata endpoints are always blocked.
vLLM (self-hosted GPU inference)
| Field | Value |
|---|---|
| Base URL | https://your-vllm-host/v1 |
| Authentication | API Key (if your vLLM instance requires one) |
Groq / Together AI / hosted providers
| Field | Value |
|---|---|
| Base URL | Provider's /v1 root (e.g. https://api.groq.com/openai/v1) |
| Authentication | API Key (your provider key) |
Troubleshooting
409 CAPABILITY_NOT_CONFIGURED, or 412 from a Scrydon feature
The endpoint is not set as the org's default for that capability, or no connection is enabled for it. Go to Settings → Platform → Extensions, open the OpenAI Compatible connection, and ensure the relevant capability product is enabled and set as default.
The Platform API answers 409 CAPABILITY_NOT_CONFIGURED. Scrydon features that call it on your
behalf answer 412 instead: creating or uploading to a knowledge base returns
code: "EMBEDDING_NOT_CONFIGURED", and image generation returns a message naming the vendor.
Model not listed after adding the connection
If your server doesn't implement GET /v1/models, the auto-discovery probe returns nothing.
You can enter model IDs manually in the capability settings after the connection is saved.
A reason is shown beside the connection instead of a model list
The server never answered the probe: the Base URL does not resolve, the port refuses
connections, or the endpoint presents a TLS certificate the platform cannot verify (a
self-signed or internal-CA certificate is the usual cause). Every model call on that
connection fails the same way, so the platform reports the cause rather than an empty list.
The message names the endpoint and the transport's own text. Fix the address, or put a
certificate from a trusted authority in front of the server, then select Refresh models.
A certificate problem is not retried on its own; a model call on the connection answers
CAPABILITY_NOT_CONFIGURED until the endpoint is corrected.
Function tools with reasoning_effort are not supported … use /v1/responses
The connection's OpenAI API is pinned to chat-completions. Clear it (auto), or
set responses for this endpoint.
Connection verification fails
Check that:
- The Base URL is reachable from the platform's network (not
localhostunless the server and the platform run on the same host). - The API key is correct (if authentication is enabled).
- Self-signed TLS certificates are trusted by the platform's trust store (operator-level configuration).
For fully air-gapped deployments, see the air-gapped deployment guide. Ollama or vLLM via this extension is the standard configuration.
OpenAI API
OpenAI-compatible servers implement Chat Completions (/chat/completions), the
Responses API (/responses), or both, and some models only work fully on one of
them. For example, gpt-6.1-sol refuses function tools with reasoning on Chat
Completions, and some gateways serve a Claude deployment only through Responses.
Leave OpenAI API blank and Scrydon works it out per model:
- Chat Completions is tried first.
- If the endpoint answers that the route does not exist (404 or 405), or rejects
the request and names the Responses API as the fix, the same request is sent
once to
/responses. The reverse applies as well. - Whichever API answered is remembered for that endpoint and model, so the extra request happens once, not on every call.
Set chat-completions or responses only to pin a server that supports just one of
them. A pinned value is never switched, so a refusal from the pinned API is shown to
you as is. Requests that carry audio input always use Chat Completions, because the
Responses API does not accept it. Requests sent to /responses set store: false,
so the provider does not keep the conversation.
Chat compatibility profiles
Set Chat compatibility profile on the connection. It applies to chat only;
model discovery and other capabilities continue using their existing endpoints.
The default is openai. Model IDs are discovered or configured by the customer,
not inferred from a provider catalog.
| Profile | Outbound behavior |
|---|---|
openai (or blank) | Standard Chat Completions; no provider-specific thinking fields. |
reasoning-content | Standard Chat Completions with reasoning preserved in assistant tool history. |
glm | Reasoning history plus object-shaped thinking controls. |
mimo | Reasoning history, object-shaped thinking, completion-token limits, and no reasoning-effort field. |
qwen | Reasoning history and boolean thinking control. |
deepseek | Reasoning history plus object-shaped thinking controls. |
Choose the server's API behavior, not merely its model family. For example, a self-hosted Qwen model may use the standard or reasoning-content profile rather than Alibaba's hosted Qwen API profile. Unknown profile values are rejected.
Explicit version roots such as /api/paas/v4 are preserved for discovery and chat;
a root without a version receives /v1. Both API-key and keyless endpoints are
supported. Different connections can select different profiles.
Expose the enabled model through the AI Gateway. Clients can use Chat Completions, Responses, or Anthropic Messages while the upstream receives Chat Completions. Protocol conversion does not emulate unsupported hosted tools, remote conversation storage, or model capabilities.
For on-premises endpoints, deploy the executor where it can reach the server and configure organization egress appropriately. Setting a private URL does not create network connectivity. Test text, tool follow-ups, and streaming against your actual server before relying on a capability; discovery alone is not qualification.