AI Gateway

The AI Gateway is a single endpoint an administrator hands to developers. Their existing coding tool points at it instead of a vendor API, and nothing about how they work changes. What changes is that every turn now runs through the same governed pipeline every other Scrydon capability call uses — model policy, DLP, moderation, audit, and metering — and is attributable to the developer who made it.

There is no second execution path. The router is a protocol translator in front of the pipeline your platform already runs.

Choose the walkthrough for your identity: an AI app connected from the browser, your Scrydon account, a service account in Claude Code, or a CI pipeline.

Connect your AI apps

Use the tools your organisation has connected to Scrydon (email, documents, tickets) from Claude Desktop, Codex, Cursor, or Claude Code. Nothing needs to be installed first. This connects tools only: the app keeps its own sign-in and its own models. To route an app's model traffic through the gateway as well, follow For developers.

Open Settings → Account → AI Gateway and choose the tab for your app. Every tab has the same three steps.

echo 'export SCRYDON_API_KEY="scry_…"' >> ~/.zshrc
codex mcp add scrydon --url 'https://auth.your-company.com/api/mcp/v1' --bearer-token-env-var SCRYDON_API_KEY

The Developer tools tab holds the base URLs and shell lines for routing model traffic. They are the same lines scrydon router env prints.

The Your keys tab lists every key you created for an AI app in this organisation, newest first, including keys scrydon router setup made: its environment, when an app last used it, and when it expires. Revoke a key there when you stop using the app, or if someone else may have seen it. The app is refused from its next call.

For developers

bunx scrydon login https://auth.your-company.com

bunx scrydon router setup

export ANTHROPIC_BASE_URL="https://auth.your-company.com/api/router"
export ENABLE_TOOL_SEARCH="true"
export OPENAI_BASE_URL="https://auth.your-company.com/api/router/v1"
export GOOGLE_GEMINI_BASE_URL="https://auth.your-company.com/api/router"
export ANTHROPIC_API_KEY="scry_…"
export OPENAI_API_KEY="scry_…"
export GEMINI_API_KEY="scry_…"

✓ Claude MCP configured (https://auth.your-company.com/api/mcp/v1)
✓ Codex MCP configured (https://auth.your-company.com/api/mcp/v1)

claude mcp add-json --scope user scrydon \
  '{"type":"http","url":"https://auth.your-company.com/api/mcp/v1","headers":{"Authorization":"Bearer ${ANTHROPIC_API_KEY}"}}'

codex mcp add scrydon --url 'https://auth.your-company.com/api/mcp/v1' --bearer-token-env-var OPENAI_API_KEY

bunx scrydon router doctor

Checking https://auth.your-company.com

  ✓ credential accepted
  ✓ 4 models available
      anthropic/claude-opus-5
      anthropic/claude-sonnet-5
      openai/gpt-5
      openai/gpt-5-mini
  ✓ completion succeeded (claude-opus-5 via anthropic)
  ✓ Claude MCP registered and connected
      https://auth.your-company.com/api/mcp/v1
  ✓ Codex MCP registered
      https://auth.your-company.com/api/mcp/v1
  ✓ 61 organization tools published by Scrydon MCP

Everything checks out. Models route, Claude Code and Codex are registered, and organization tools are published.
Personal OAuth account readiness is checked when each tool is called.

Commands

When something fails

The router answers in your tool's own error dialect, so the message reaches you rather than being swallowed. What the common ones mean:

Tool calls Scrydon refuses. When Scrydon refuses an MCP tool call — no connected account, a guardrail, a spend cap, an unavailable provider — the call returns a tool result marked as an error (isError: true) whose text is the sentence from the table above, so your AI app can read it and tell you what to do. The Scrydon code and HTTP status are in _meta["scrydon/error"]. Only a request Scrydon cannot accept at all — an unknown tool name, malformed parameters, or an access key it does not recognise — is a JSON-RPC error.

Errors after a stream starts. A streamed turn that fails partway is still an HTTP 200: the stream ends with one error event in your tool's dialect, after whatever text was already delivered. That event carries the same Scrydon code and details a non-streamed error does, so a policy refusal mid-answer still names its policyCode. They sit under scrydon at the top of the Anthropic and Ollama error bodies, under error.scrydon for OpenAI Chat Completions, on the error event itself for OpenAI Responses, and in error.details[] for Gemini. A stream the provider stopped without finishing ends with code INTERNAL_ERROR and details.reason STREAM_ENDED_WITHOUT_TERMINAL. Treat that answer as truncated, not as complete. It is recorded as a failed turn, not a successful one.

What is not supported

The router refuses these rather than ignoring them, because a parameter silently dropped gives you wrong output with no way to find out:

• n, logprobs, top_logprobs, seed, logit_bias, presence_penalty, frequency_penalty (and their Gemini / Ollama spellings)
• the deprecated functions / function_call fields — use tools / tool_choice
• server-side conversation state: Responses previous_response_id, item_reference, background; Gemini cachedContent. Send the full input each turn — your tool already does.
• a vendor's hosted tools (web_search, file_search, googleSearch, codeExecution, …). Organisation tools reach you through the Scrydon MCP server instead, governed.
• pre-tokenized embedding input (integer arrays) — the router has no tokenizer for the target model.
• a single text part above 1,000,000 characters (roughly 250k tokens); the request as a whole is bounded at 16 MiB.
• Anthropic count_tokens, Ollama /api/generate, /api/embed and model management.

Sending any of them returns a 400 naming the field. Sending them as null (which some SDKs do for unset options) is fine. Fields that carry no request semantics for a hosted model — store, include, safetySettings, num_ctx, keep_alive, user — are accepted and ignored.

The OpenAI-compatible /chat/completions endpoint carries provider reasoning in the optional reasoning_content field, including streaming deltas and assistant history for tool follow-ups. Reasoning stays separate from visible content; providers that do not expose reasoning do not produce this field. The Anthropic /v1/messages shape carries thinking blocks in full, including their signatures.

Use Claude Code with a service account

Use this walkthrough to send a message from your terminal as an organization-owned service account. You need Bun, curl, jq, and Claude Code installed. Your Scrydon deployment and CLI must support service-account machine-token login. For CI jobs with an OIDC provider, use secretless federation.

Prepare the service account

As an organization admin, open Settings → Members → Service accounts and create an account with capabilities:write. Pin it to an environment with a configured Claude provider connection and an enabled Claude model. Copy its client ID and the secret shown once at creation; rotate the secret if you no longer have it.

This login uses the account's ordinary client ID, not its Federated client ID. The secret authenticates the account to Scrydon; Claude receives the short-lived bearer issued by that login. You do not need scrydon router setup for this identity. bunx scrydon router doctor currently checks the separate router key saved by personal-account setup; use the bearer model-list request and Claude message below to verify this service account.

Log in from your terminal

Run these commands in Bash or Zsh. Replace the URL with your platform origin, without a trailing slash, and replace the client ID. The prompt reads the secret without echoing it or putting its value in shell history.

export SCRYDON_PLATFORM_URL="https://auth.your-company.com"
export SCRYDON_CLIENT_ID="YOUR_SERVICE_ACCOUNT_CLIENT_ID"

printf 'Client secret: '
IFS= read -rs SCRYDON_CLIENT_SECRET
printf '\n'

bunx scrydon login "$SCRYDON_PLATFORM_URL" \
  --service-account "$SCRYDON_CLIENT_ID" \
  --service-account-secret "$SCRYDON_CLIENT_SECRET" \
  --ttl 2h

unset SCRYDON_CLIENT_SECRET

Continue only after login succeeds. If the CLI redirects you from an application URL to its platform URL, update SCRYDON_PLATFORM_URL to the platform URL it prints before the next step. The credential file is keyed by that canonical URL. --ttl 2h requests two hours; organization policy and the deployment ceiling can shorten it, and the CLI reports the issued lifetime.

Configure Claude and choose a model

In the same terminal, load the bearer saved by login and configure Claude's gateway:

TOKEN="$(jq -er --arg url "$SCRYDON_PLATFORM_URL" \
  '.[$url].token' \
  "${SCRYDON_CREDENTIALS_DIR:-$HOME/.scrydon}/credentials.json")"

unset ANTHROPIC_API_KEY CLAUDE_CODE_OAUTH_TOKEN
export ANTHROPIC_BASE_URL="$SCRYDON_PLATFORM_URL/api/router"
export ANTHROPIC_AUTH_TOKEN="$TOKEN"
export ENABLE_TOOL_SEARCH=true
unset TOKEN

curl --fail --silent --show-error \
  "$ANTHROPIC_BASE_URL/v1/models" \
  -H "Authorization: Bearer $ANTHROPIC_AUTH_TOKEN" \
  | jq -r '.data[].id'

The base URL ends at /api/router: Claude appends /v1/messages. ANTHROPIC_AUTH_TOKEN sends the Scrydon bearer in the Authorization header. These exports configure this terminal and the Claude process started from it. See Claude's gateway setup if existing Claude settings override your shell configuration.

Copy an enabled Claude model ID from the list, including any provider prefix, into the command below. An empty list means an administrator needs to check the provider and model configuration for the account's environment.

Send your first message

export SCRYDON_CLAUDE_MODEL="PASTE_AN_ENABLED_CLAUDE_MODEL_ID"

claude --model "$SCRYDON_CLAUDE_MODEL" \
  -p "Reply with one sentence confirming you received this message."

-p prints the response and exits. For an interactive conversation, run:

claude --model "$SCRYDON_CLAUDE_MODEL"

In the interactive session, /status should show your gateway base URL and the environment token as the credential source. Check Settings → Governance → Cost Insights → Usage, select AI Gateway and the Principal breakdown: the request belongs to the service account's service account, shown by its name, not your personal account. Listing models alone does not demonstrate completion spend.

When the bearer expires, exit Claude, repeat login and token export, then start Claude again. Do not persist the bearer in your shell profile or commit it to Claude settings. Disabling the service account rejects its bearer on the next authenticated request; rotating its client secret only changes future logins. See credential lifecycle.

For pipelines

A CI pipeline is not a developer at a terminal — there is no browser to approve a device-flow login in, and the credential it holds needs to be revocable and time-bounded on its own terms, not borrowed from a person's session. A service account is the pipeline's own identity.

Create the account

Settings → Members → Service accounts → New service account. Grant it the capabilities:write scope — the same scope every router call declares — and pin its environment.

The environment is a property of the account, not of a flag or the token: a dev pipeline and a prd pipeline are two separate service accounts, each pinned to its own environment, not one account switched between them at run time. See Service accounts for what pinning does and how to change it later.

Store the credentials

For CI providers that issue OIDC tokens, use secretless federation. That recipe registers the issuer, attaches an exact pipeline subject, and exchanges a fresh assertion with the account's Federated client ID for an AI Gateway bearer. It requires no stored Scrydon client secret. The CLI secret-login instructions below are the alternative when your pipeline cannot federate.

Save the client id and secret as your pipeline's repository secrets — for example SCRYDON_CLIENT_ID and SCRYDON_CLIENT_SECRET in GitHub Actions.

Unlike the bearer it is exchanged for, the secret is not meant to be minted per run — but it does expire. Your organization has a maximum lifetime for it (180 days by default, up to a 365-day hard ceiling); Service accounts covers where that number comes from and how rotation works. Rotating replaces the secret immediately — there is no overlap window — so update the pipeline's stored secret in the same change that rotates it.

Authenticate

Login and export belong in the same step. Each - run: step is a fresh shell, so exporting a variable in one step and reading it in the next reaches nothing — the export dies with the shell that made it. A separate step invoking your coding tool needs the variables written somewhere later steps actually inherit: on GitHub Actions, that is $GITHUB_ENV; other CI systems have their own equivalent (a shared env file, a pipeline-variable API — check yours before assuming export alone is enough).

- run: |
    scrydon login https://auth.your-company.com \
      --service-account "$SCRYDON_CLIENT_ID" \
      --service-account-secret "$SCRYDON_CLIENT_SECRET" \
      --ttl 2h

    TOKEN="$(jq -r '."https://auth.your-company.com".token' ~/.scrydon/credentials.json)"
    {
      echo "ANTHROPIC_BASE_URL=https://auth.your-company.com/api/router"
      echo "OPENAI_BASE_URL=https://auth.your-company.com/api/router/v1"
      echo "ANTHROPIC_API_KEY=$TOKEN"
      echo "OPENAI_API_KEY=$TOKEN"
    } >> "$GITHUB_ENV"

Use your platform URL — the same one this whole page uses, covered below in Which URL do I point at?. A pipeline's login resolves it exactly the way a developer's does: type either URL and the CLI asks the deployment which origin owns its auth. If you typed the application URL instead, scrydon login prints "<the url you typed> is the application URL; its platform URL is <the one it found>. Logging in there instead." and continues — that is the CLI correcting a URL you did not have to get exactly right, not a failure.

--ttl is a request, not a guarantee. Your organization's credential policy — and, under that, a hard ceiling your operator sets for the whole deployment — can clamp it shorter, and the server clamps rather than refusing. Ask for 2h (7200s) against a deployment whose ceiling is lower and the command prints, for example, "Requested 7200s; this organization's policy caps it at 3600s. The token is valid for the shorter time." — the login still succeeded. Ask for nothing and you get the organization's default (an hour, until an administrator sets one). An organization admin sets that maximum, default, and the client-secret lifetime at Settings → Governance → Credentials → Policy — the deployment ceiling itself is shown there too, read-only, since only your operator can move it.

The token lands in ~/.scrydon/credentials.json, the same file a developer's login uses. There is no separate router-key mint step for a service account the way there is for a person's router setup — the bearer the login returns is itself a valid router credential, which is why the snippet above reads it straight back out rather than minting anything new.

Log in again at the start of every job rather than caching the exported key across jobs — see the expiry failure mode below.

Choose a model

GET /api/router/v1/models, sent with the bearer above, lists what your organization enabled — the same allowlist a developer's key is bound by:

curl -H "authorization: Bearer $ANTHROPIC_API_KEY" https://auth.your-company.com/api/router/v1/models

When something fails

Re-pinning an account's environment takes effect on its next request, including for a bearer already issued — the environment is read from the account on every request, not stamped into the token, so there is nothing to re-mint.

Connect an app that only takes an API key

Some apps cannot sign in or refresh a token. Their AI settings ask for a base URL and an API key, store both, and send the same key on every call: Twenty CRM, Open WebUI, n8n, LibreChat. Give such an app a router key issued to a service account. Its calls are then governed, billed and audited as that account — not as you, and not as whoever set the app up.

Prepare the service account

Create a service account for the app and name it after the app, for example "Twenty CRM", so its spend is easy to find. Grant it capabilities:write and limit it to one environment with the models the app should use. A router key acts in its account's environment and carries no other scope. See Router keys for every requirement.

Issue the key

On the account, select Add router key, name it after the app, and choose a lifetime. Copy the key — it is shown once — and select Test key to confirm Scrydon accepts it.

Configure the app

Use the base URL for the API the app speaks:

The AI Gateway accepts the key in either header, so you do not need to choose. The sheet that shows the key lists both base URLs for your deployment. As with a developer key, they differ by one path segment, and that is deliberate: each SDK appends its own path.

For Twenty CRM, add a provider to its AI_PROVIDERS configuration that uses the OpenAI-compatible SDK (@ai-sdk/openai-compatible), with baseUrl set to the OpenAI base URL above and apiKey set to the router key. Keep the key in the secret your deployment already mounts for Twenty rather than in plain configuration.

Before configuring the app, you can confirm the key reaches a model:

curl -H "authorization: Bearer $ROUTER_KEY" https://auth.your-company.com/api/router/v1/models

The list shows the models enabled for the account's environment — the same allowlist the app's calls are held to.

Qualification examples

The repository includes pinned LibreChat configuration and a LangGraph example that reopens its SQLite checkpoint and switches models through Chat Completions (tests/qualification/governed-ai). The application's graph, conversation and tools remain application-owned. Its review interrupt is an application decision; privileged Scrydon operations still require their own execution grants. A shared service-account key identifies the application, not an employee.

Local HTTP/SDK regression checks cover streamed responses, tool-call continuity, checkpoint resume and errors. Live client, provider, privacy and collector runs must be qualified against your deployment; these examples do not certify them.

Keep it working

• Rotate before it expires. The account's creator is warned six days before the key expires. Rotate key keeps the old key working for a day or a week, so update the app within that window.
• A change to the account reaches the key at once. Disabling the account, removing capabilities:write, or setting it to act in every environment stops the key on its next call.
• Revoke it from the account or from the credential ledger when the app no longer needs it.

For administrators

Enable models

Settings → Platform → Extensions → the provider → Includes. What you select there is what developers can reach.

Three states, and the difference matters:

See what it costs

Settings → Governance → Cost Insights → Usage, with the source set to AI Gateway.

Router traffic is metered with its own source, separate from application API traffic, so the per-principal table answers "who is using this and what is it costing me" — a person or a service account — without an application's workload muddying the numbers. Each row carries calls, tokens, indicative spend, and modelled CO₂.

The breakdown switches between Principal and Model, the tool table lists what was reached through the MCP endpoint, and the budget line at the top of the page shows the organisation cap. The same data is available through billing.summary with filters: { source: "router" }.

Set spend caps

Cost Insights → Limits. Every cap sits beside the scope's spend so far this period, with the highest team or developer named. A cap can be set on a developer, a team, or the whole organisation, measured over a rolling day or month, against either indicative spend or call count.

• Caps are evaluated from the usage ledger, so they hold across every replica.
• Enforce refuses further calls; Audit records the breach and lets the call through.
• A blank field means uncapped. 0 means capped at zero — that distinction is real.
• When a cap is reached the developer gets a 429 whose Retry-After is the actual time until the period resets, so a well-behaved client backs off properly instead of hammering.

Choose a DLP posture

Cost Insights → Limits, under Router guardrails.

DLP runs over every prompt, and for a chat message that is straightforwardly right. A coding agent's prompt is a different shape: it carries the working set of a repository, re-sent every turn, and source code is adversarial input for rules tuned on prose — a credential pattern matches test fixtures, .env.example, and base64 in a lockfile.

So the router's posture is a decision you make rather than one you inherit:

Audit is not "DLP off." Every rule still runs under the same limits as in enforce mode: built-in detectors always run, and each custom rule runs within its field limit and the per-field time budget, with compiling the policy's rules limited to 1 second per request (see DLP limits). A custom rule those limits stop is not silently dropped: it fails closed on that field — which, under audit, means the hit is reported — and the scan row names it. Every finding is still counted in the audit log, and every kind of dlp.violation an auditor reads is still produced — one row per rule, finding type and channel per request, with the number of occurrences. The only thing it changes is whether a finding refuses the turn. Changing it affects the router only; chat, workflows, and every application surface keep the organisation policy.

Enforce streams too. A streamed answer under an enforce posture is governed in windows: DLP scans roughly every 512 characters, holds the last 256 back until the next window confirms them, and releases the rest — so text flows while the model is still writing instead of arriving all at once when it finishes. A finding later in the answer still stops it: the stream ends with a policy error after the clean prefix was delivered, and the text that carried the finding is never sent. Moderation in enforce mode still judges the whole answer before releasing it, because its judge is a model call.

Define routes — virtual models that pick a real one

A route is a model id you define. A developer names it (ANTHROPIC_MODEL=team/agent, or --model team/agent) and an algorithm picks the real, enabled, allowlisted model for each turn. Nothing is substituted unless the developer asked for the route by name, and the model a route picks still has to pass your allowlist — a route cannot reach a model you did not enable.

Routes live in the organisation policy platform.llm_router.routes, written through the organisation policy API (organization.policy.set). Three route types:

{
  "version": 1,
  "routes": {
    "team/fast": { "type": "passthrough", "target": "anthropic/claude-sonnet-5" },
    "team/ab": {
      "type": "random",
      "targets": [
        { "model": "anthropic/claude-opus-5", "weight": 1 },
        { "model": "openai/gpt-5", "weight": 3 }
      ]
    },
    "team/agent": {
      "type": "stage_router",
      "capable": "anthropic/claude-opus-5",
      "efficient": "anthropic/claude-sonnet-5",
      "picker": "efficient_first",
      "confidenceThreshold": 0.5,
      "recentTurnWindow": 3
    }
  }
}

What is recorded: the usage event and audit row carry the model that actually served the turn and the route that chose it (route.id, route.type, the decision source and tier), and the Anthropic message_start frame names the served model, so Cost Insights shows the real split and an auditor can ask "which turns did team/agent send to the capable tier". Routes are not listed on /v1/models — name them directly. A policy that fails validation deactivates every route and is logged; the developer then gets the ordinary "model not enabled" answer.

Revoke a developer's access

Settings → Governance → Credentials lists every member's keys, router keys included, by the name the developer gave them. Revoking one takes effect on the next call. A developer revokes their own under Settings → Account → AI Gateway → Your keys.

Removing the local copy with scrydon router forget does not revoke the key. If a key may have leaked, revoke it here.

What the router does with a request

1. Authenticate. The router key carries its own organisation and environment — the request never asserts a tenant, and the router strips any header that tries to.
2. Admit. Scope, deadline, data classification, clearance, and your spend caps.
3. Resolve a target. Your enabled provider, your allowlist, the connection bound to that environment, and its credential. A developer's tool never sees a vendor credential.
4. Apply guardrails. DLP, then moderation, under the posture you chose.
5. Dispatch. Through the same runtime every other capability call uses.
6. Record. One audit event and one usage event, attributed to the person or service account.

Steps 2 through 6 are not router code. They are the pipeline your platform already runs; the router's own job is steps 1 and the translation on either side of it.

Capacity and back-pressure

Each API Platform pod accepts a bounded number of open calls at once — one budget shared by router turns, direct platform API calls and MCP tool calls (the chart value auth.maxInFlight, default 64; env PLATFORM_MAX_IN_FLIGHT; the older auth.llmRouter.maxInFlight / LLM_ROUTER_MAX_IN_FLIGHT still work as a fallback). When the pod is more than half full and another organization is using it, no organization may take more than a quarter of it (platform_org_in_flight_share), so one busy tenant cannot refuse everyone else. An organization alone on the pod can use all of it but four slots (a quarter, on a small pod), kept free for a newcomer. Calls still being authorized draw on a separate allowance of the same size, so an organization's own retries cannot hold those four slots. A newcomer gets in unless one tenant keeps that many calls in authorization at once. If the pod fills while a call is being authorized, that call is refused after authorization with the same 429. A streamed turn counts until its stream ends or you disconnect; a non-streamed turn counts until its response is returned. Model listing (/models) is exempt: it is a short catalog read that never holds a slot. Past that bound the router answers 429 with a Retry-After header — rate_limit_error in the Anthropic dialect, rate_limit_error / rate_limit_exceeded in the OpenAI dialect — and does not touch the database or start any work for the refused request. The Anthropic and OpenAI SDKs both honour Retry-After automatically.

This is a per-pod overload guard, not a per-developer quota: it exists so a saturated pod sheds load instead of slowing every stream and then running out of memory. Sustained 429s at steady state mean the deployment needs another replica or a higher CPU limit, not client-side retries. Per-developer and per-organisation limits are the spend caps above.

Operators can watch scrydon_llm_router_streams_in_flight, scrydon_llm_router_admission_rejections_total and the TTFT / duration histograms on the AI Gateway SigNoz dashboard shipped with the operations chart. Two histograms separate the platform from the provider: scrydon_llm_router_overhead_ms is a turn's wall time minus the time spent inside the provider call (queue wait, guardrails and audit settlement included), and scrydon_llm_router_provider_ttfb_ms is the provider's own time to first event. scrydon_llm_router_turns_total carries a response_class (success, retryable_error, other_error) and every provider attempt, retries included, is counted in scrydon_extension_outbound_attempts_total with the same classes — the difference between the two error rates is what the router's retries are worth.

Protocol coverage

Every dialect is a translation in front of the same pipeline; GET /api/router/v1/health lists the ones your deployment speaks.

Which URL do I point at?

Your platform URL — the one serving /api/router, usually auth.<your-company>. Not the application URL you sign in to every day.

The distinction matters more than it should, because the application origin proxies /api/auth. A login against it therefore succeeds, and nothing looks wrong until your editor reports a bare 404 from an endpoint that knows nothing about Scrydon.

The CLI handles this for you: type either URL and scrydon login asks the deployment which origin owns its auth, tells you if it redirected, and stores the right one. Every scrydon router subcommand accepts either URL too — when the one you typed has no stored credentials it follows the same redirect, says which URL it used, and continues. scrydon router setup then probes /api/router before minting a key, so a mismatch fails immediately with the cause named rather than silently later.

If you are configuring a client by hand, use the platform URL.

Related

• Billing and usage metering — the billing.* API behind Cost Insights
• DLP overview — what the guardrails scan for
• Governance usage — the wider Cost Insights surface

Customer-managed models

Connect a hosted or on-premises inference server through OpenAI Compatible, then discover its models or enter the deployment IDs in the connection settings. Use the model ID published by the gateway; no new first-party extension or catalog entry is required for each model. Model names can include organization/path prefixes.

The connection's Chat compatibility profile selects the upstream API behavior. Leave it blank for standard Chat Completions. Use reasoning-content for compatible servers that require reasoning content in tool history, or glm, mimo, qwen, or deepseek when the endpoint implements that dialect. Select based on the server's API, not the model's brand: Qwen hosted by vLLM can expose a different contract from Alibaba's hosted endpoint.

Gateway Chat Completions, Responses, and Anthropic Messages requests are normalized before reaching the selected upstream connection, which then chooses the upstream OpenAI API for that model: Chat Completions or /responses, set by the connection's OpenAI API field (blank means auto). A Responses client does not require the upstream server to implement /responses, and a Chat Completions client can reach a model that works only through it. Existing gateway limits still apply, including rejection of unsupported hosted tools and server-side conversation state.

The executor must have a network route to the private endpoint. Connection settings do not establish a VPN or tunnel. Existing organization egress controls still apply. See OpenAI-compatible endpoints for connection setup and profile details.