OpenAI-Compatible Endpoints
Connect any endpoint that speaks the OpenAI /v1 API — vLLM, Ollama, LM Studio, LocalAI, Groq, Together, and more.
Add Settings → Integrations → Add Integration → Generic → OpenAI Compatible to connect
any server that implements the OpenAI /v1 API — vLLM, Ollama, LM Studio, LocalAI, Groq,
Together AI, and more.
This is the recommended path for:
- Self-hosted inference servers (Ollama, vLLM, LM Studio) in fully local or air-gapped deployments.
- Third-party OpenAI-compatible hosted services (Groq, Together AI, Perplexity, etc.) that are not listed as a named integration.
- Custom or internal endpoints that implement the OpenAI
/v1contract.
Configure
Navigate to Settings → Integrations → Add Integration, open the Generic group, and select OpenAI Compatible. The wizard walks you through three steps:
- Capabilities — tick which capabilities this endpoint provides (LLM, Embeddings, STT,
TTS, Image). A best-effort
GET /v1/modelsprobe pre-fills available models; if your server doesn't implement/models, you can enter model IDs manually. - Configure — set Base URL and optional authentication.
- Verify — the wizard tests reachability before saving.
Configuration fields
| Field | Required | Notes |
|---|---|---|
| Base URL | Yes | The /v1 root, e.g. https://api.groq.com/openai/v1. The /v1 suffix is appended automatically if omitted. |
| Authentication | No | Choose API Key (sent as Authorization: Bearer <key>, stored encrypted) or No authentication for keyless local servers such as Ollama. |
| Content-Type | No (advanced) | Defaults to application/json. Leave blank unless your server requires a non-standard value. Audio uploads automatically use multipart/form-data regardless of this setting. |
Capability coverage
| Capability | OpenAI path | Status |
|---|---|---|
| Chat / LLM completions | POST /chat/completions | ✅ Available |
| Embeddings | POST /embeddings | ✅ Available |
| Model discovery | GET /models | ✅ Available (best-effort probe) |
| Transcription / translation | POST /audio/transcriptions, /audio/translations | ✅ Available |
| Text-to-speech | POST /audio/speech | ✅ Available |
| Image generation | POST /images/generations | ✅ Available |
All five capabilities — plus best-effort model discovery — support keyless (no-auth) endpoints. Choose No authentication in the wizard for local servers such as Ollama, Whisper.cpp, or any self-hosted endpoint that does not require an API key.
One Base URL can serve multiple capabilities simultaneously — for example, a vLLM instance that handles both chat completions and embeddings. Tick all relevant capabilities in the wizard step and they resolve through the same connection.
Multiple endpoints
If you need separate servers for different capabilities (e.g. a dedicated audio transcription service alongside a chat server), add a second OpenAI Compatible connection. The platform supports multiple named connections per vendor and lets you select which connection serves each capability.
Common self-hosted configurations
Ollama (fully local / air-gapped)
| Field | Value |
|---|---|
| Base URL | http://localhost:11434/v1 (or your internal host) |
| Authentication | No authentication |
Ollama exposes GET /v1/models so the wizard can discover available models automatically.
Local and loopback endpoints are reachable by default: the Private & Loopback Networks
toggle under Settings → Governance → Egress ships enabled. If your administrator has
turned it off, connections to localhost/private-IP endpoints fail until it is re-enabled.
Cloud metadata endpoints are always blocked, regardless of this setting.
vLLM (self-hosted GPU inference)
| Field | Value |
|---|---|
| Base URL | https://your-vllm-host/v1 |
| Authentication | API Key (if your vLLM instance requires one) |
Groq / Together AI / hosted providers
| Field | Value |
|---|---|
| Base URL | Provider's /v1 root (e.g. https://api.groq.com/openai/v1) |
| Authentication | API Key (your provider key) |
Troubleshooting
412 LLM_NOT_CONFIGURED / 412 EMBEDDING_NOT_CONFIGURED / 412 STT_NOT_CONFIGURED / 412 TTS_NOT_CONFIGURED / 412 IMAGE_NOT_CONFIGURED
The endpoint is not set as the org's default for that capability, or no connection is enabled for it. Go to Settings → Integrations, open the OpenAI Compatible connection, and ensure the relevant capability product is enabled and set as default.
Model not listed after adding the connection
If your server doesn't implement GET /v1/models, the auto-discovery probe returns nothing.
You can enter model IDs manually in the capability settings after the connection is saved.
Connection verification fails
Check that:
- The Base URL is reachable from the platform's network (not
localhostunless the server and the platform run on the same host). - The API key is correct (if authentication is enabled).
- Self-signed TLS certificates are trusted by the platform's trust store (operator-level configuration).
For fully air-gapped deployments, see the air-gapped deployment guide. Ollama or vLLM via this integration is the standard configuration.