AI Gateway

The AI Gateway is a single endpoint an administrator hands to developers. Their existing coding tool points at it instead of a vendor API, and nothing about how they work changes. What changes is that every turn now runs through the same governed pipeline every other Scrydon capability call uses — model policy, DLP, moderation, audit, and metering — and is attributable to the developer who made it.

There is no second execution path. The router is a protocol translator in front of the pipeline your platform already runs.

Choose the walkthrough for your identity: your Scrydon account, a service account in Claude Code, or a CI pipeline.

For developers

bunx scrydon login https://auth.your-company.com

bunx scrydon router setup

export ANTHROPIC_BASE_URL="https://auth.your-company.com/api/router"
export ENABLE_TOOL_SEARCH="true"
export OPENAI_BASE_URL="https://auth.your-company.com/api/router/v1"
export GOOGLE_GEMINI_BASE_URL="https://auth.your-company.com/api/router"
export ANTHROPIC_API_KEY="scry_…"
export OPENAI_API_KEY="scry_…"
export GEMINI_API_KEY="scry_…"

✓ Claude MCP configured (https://auth.your-company.com/api/mcp/v1)

claude mcp add-json --scope user scrydon \
  '{"type":"http","url":"https://auth.your-company.com/api/mcp/v1","headers":{"Authorization":"Bearer ${ANTHROPIC_API_KEY}"}}'

bunx scrydon router doctor

Checking https://auth.your-company.com

  ✓ credential accepted
  ✓ 4 models available
      anthropic/claude-opus-5
      anthropic/claude-sonnet-5
      openai/gpt-5
      openai/gpt-5-mini
  ✓ completion succeeded (claude-opus-5 via anthropic)
  ✓ Claude MCP registered and connected
      https://auth.your-company.com/api/mcp/v1
  ✓ 61 organization tools published by Scrydon MCP

Everything checks out. Models route, Claude MCP is connected, and organization tools are published.
Personal OAuth account readiness is checked when each tool is called.

Commands

When something fails

The router answers in your tool's own error dialect, so the message reaches you rather than being swallowed. What the common ones mean:

What is not supported

The router refuses these rather than ignoring them, because a parameter silently dropped gives you wrong output with no way to find out:

• n, logprobs, top_logprobs, seed, logit_bias, presence_penalty, frequency_penalty (and their Gemini / Ollama spellings)
• the deprecated functions / function_call fields — use tools / tool_choice
• server-side conversation state: Responses previous_response_id, item_reference, background; Gemini cachedContent. Send the full input each turn — your tool already does.
• a vendor's hosted tools (web_search, file_search, googleSearch, codeExecution, …). Organisation tools reach you through the Scrydon MCP server instead, governed.
• pre-tokenized embedding input (integer arrays) — the router has no tokenizer for the target model.
• a single text part above 1,000,000 characters (roughly 250k tokens); the request as a whole is bounded at 16 MiB.
• Anthropic count_tokens, Ollama /api/generate, /api/embed and model management.

Sending any of them returns a 400 naming the field. Sending them as null (which some SDKs do for unset options) is fine. Fields that carry no request semantics for a hosted model — store, include, safetySettings, num_ctx, keep_alive, user — are accepted and ignored.

Reasoning content is not surfaced on the OpenAI /chat/completions shape — that wire format has no channel for it, and putting it in content would corrupt the assistant turn. The Anthropic /v1/messages shape carries thinking blocks in full, including their signatures.

Use Claude Code with a service account

Use this walkthrough to send a message from your terminal as an organization-owned service account. You need Bun, curl, jq, and Claude Code installed. Your Scrydon deployment and CLI must support service-account machine-token login. For CI jobs with an OIDC provider, use secretless federation.

Prepare the service account

As an organization admin, open Settings → Members → Service accounts and create an account with capabilities:write. Pin it to an environment with a configured Claude provider connection and an enabled Claude model. Copy its client ID and the secret shown once at creation; rotate the secret if you no longer have it.

This login uses the account's ordinary client ID, not its Federated client ID. The secret authenticates the account to Scrydon; Claude receives the short-lived bearer issued by that login. You do not need scrydon router setup for this identity. bunx scrydon router doctor currently checks the separate router key saved by personal-account setup; use the bearer model-list request and Claude message below to verify this service account.

Log in from your terminal

Run these commands in Bash or Zsh. Replace the URL with your platform origin, without a trailing slash, and replace the client ID. The prompt reads the secret without echoing it or putting its value in shell history.

export SCRYDON_PLATFORM_URL="https://auth.your-company.com"
export SCRYDON_CLIENT_ID="YOUR_SERVICE_ACCOUNT_CLIENT_ID"

printf 'Client secret: '
IFS= read -rs SCRYDON_CLIENT_SECRET
printf '\n'

bunx scrydon login "$SCRYDON_PLATFORM_URL" \
  --service-account "$SCRYDON_CLIENT_ID" \
  --service-account-secret "$SCRYDON_CLIENT_SECRET" \
  --ttl 2h

unset SCRYDON_CLIENT_SECRET

Continue only after login succeeds. If the CLI redirects you from an application URL to its platform URL, update SCRYDON_PLATFORM_URL to the platform URL it prints before the next step. The credential file is keyed by that canonical URL. --ttl 2h requests two hours; organization policy and the deployment ceiling can shorten it, and the CLI reports the issued lifetime.

Configure Claude and choose a model

In the same terminal, load the bearer saved by login and configure Claude's gateway:

TOKEN="$(jq -er --arg url "$SCRYDON_PLATFORM_URL" \
  '.[$url].token' \
  "${SCRYDON_CREDENTIALS_DIR:-$HOME/.scrydon}/credentials.json")"

unset ANTHROPIC_API_KEY CLAUDE_CODE_OAUTH_TOKEN
export ANTHROPIC_BASE_URL="$SCRYDON_PLATFORM_URL/api/router"
export ANTHROPIC_AUTH_TOKEN="$TOKEN"
export ENABLE_TOOL_SEARCH=true
unset TOKEN

curl --fail --silent --show-error \
  "$ANTHROPIC_BASE_URL/v1/models" \
  -H "Authorization: Bearer $ANTHROPIC_AUTH_TOKEN" \
  | jq -r '.data[].id'

The base URL ends at /api/router: Claude appends /v1/messages. ANTHROPIC_AUTH_TOKEN sends the Scrydon bearer in the Authorization header. These exports configure this terminal and the Claude process started from it. See Claude's gateway setup if existing Claude settings override your shell configuration.

Copy an enabled Claude model ID from the list, including any provider prefix, into the command below. An empty list means an administrator needs to check the provider and model configuration for the account's environment.

Send your first message

export SCRYDON_CLAUDE_MODEL="PASTE_AN_ENABLED_CLAUDE_MODEL_ID"

claude --model "$SCRYDON_CLAUDE_MODEL" \
  -p "Reply with one sentence confirming you received this message."

-p prints the response and exits. For an interactive conversation, run:

claude --model "$SCRYDON_CLAUDE_MODEL"

In the interactive session, /status should show your gateway base URL and the environment token as the credential source. Check Settings → Governance → Cost Insights → Usage, select AI Gateway and the Principal breakdown: the request belongs to the service account's client ID, not your personal account. Service accounts currently appear by client ID rather than friendly name. Listing models alone does not demonstrate completion spend.

When the bearer expires, exit Claude, repeat login and token export, then start Claude again. Do not persist the bearer in your shell profile or commit it to Claude settings. Disabling the service account rejects its bearer on the next authenticated request; rotating its client secret only changes future logins. See credential lifecycle.

For pipelines

A CI pipeline is not a developer at a terminal — there is no browser to approve a device-flow login in, and the credential it holds needs to be revocable and time-bounded on its own terms, not borrowed from a person's session. A service account is the pipeline's own identity.

Create the account

Settings → Members → Service accounts → New service account. Grant it the capabilities:write scope — the same scope every router call declares — and pin its environment.

The environment is a property of the account, not of a flag or the token: a dev pipeline and a prd pipeline are two separate service accounts, each pinned to its own environment, not one account switched between them at run time. See Service accounts for what pinning does and how to change it later.

Store the credentials

For CI providers that issue OIDC tokens, use secretless federation. That recipe registers the issuer, attaches an exact pipeline subject, and exchanges a fresh assertion with the account's Federated client ID for an AI Gateway bearer. It requires no stored Scrydon client secret. The CLI secret-login instructions below are the alternative when your pipeline cannot federate.

Save the client id and secret as your pipeline's repository secrets — for example SCRYDON_CLIENT_ID and SCRYDON_CLIENT_SECRET in GitHub Actions.

Unlike the bearer it is exchanged for, the secret is not meant to be minted per run — but it does expire. Your organization has a maximum lifetime for it (180 days by default, up to a 365-day hard ceiling); Service accounts covers where that number comes from and how rotation works. Rotating replaces the secret immediately — there is no overlap window — so update the pipeline's stored secret in the same change that rotates it.

Authenticate

Login and export belong in the same step. Each - run: step is a fresh shell, so exporting a variable in one step and reading it in the next reaches nothing — the export dies with the shell that made it. A separate step invoking your coding tool needs the variables written somewhere later steps actually inherit: on GitHub Actions, that is $GITHUB_ENV; other CI systems have their own equivalent (a shared env file, a pipeline-variable API — check yours before assuming export alone is enough).

- run: |
    scrydon login https://auth.your-company.com \
      --service-account "$SCRYDON_CLIENT_ID" \
      --service-account-secret "$SCRYDON_CLIENT_SECRET" \
      --ttl 2h

    TOKEN="$(jq -r '."https://auth.your-company.com".token' ~/.scrydon/credentials.json)"
    {
      echo "ANTHROPIC_BASE_URL=https://auth.your-company.com/api/router"
      echo "OPENAI_BASE_URL=https://auth.your-company.com/api/router/v1"
      echo "ANTHROPIC_API_KEY=$TOKEN"
      echo "OPENAI_API_KEY=$TOKEN"
    } >> "$GITHUB_ENV"

Use your platform URL — the same one this whole page uses, covered below in Which URL do I point at?. A pipeline's login resolves it exactly the way a developer's does: type either URL and the CLI asks the deployment which origin owns its auth. If you typed the application URL instead, scrydon login prints "<the url you typed> is the application URL; its platform URL is <the one it found>. Logging in there instead." and continues — that is the CLI correcting a URL you did not have to get exactly right, not a failure.

--ttl is a request, not a guarantee. Your organization's credential policy — and, under that, a hard ceiling your operator sets for the whole deployment — can clamp it shorter, and the server clamps rather than refusing. Ask for 2h (7200s) against a deployment whose ceiling is lower and the command prints, for example, "Requested 7200s; this organization's policy caps it at 3600s. The token is valid for the shorter time." — the login still succeeded. Ask for nothing and you get the organization's default (an hour, until an administrator sets one). An organization admin sets that maximum, default, and the client-secret lifetime at Settings → Governance → Credentials → Policy — the deployment ceiling itself is shown there too, read-only, since only your operator can move it.

The token lands in ~/.scrydon/credentials.json, the same file a developer's login uses. There is no separate router-key mint step for a service account the way there is for a person's router setup — the bearer the login returns is itself a valid router credential, which is why the snippet above reads it straight back out rather than minting anything new.

Log in again at the start of every job rather than caching the exported key across jobs — see the expiry failure mode below.

Choose a model

GET /api/router/v1/models, sent with the bearer above, lists what your organization enabled — the same allowlist a developer's key is bound by:

curl -H "authorization: Bearer $ANTHROPIC_API_KEY" https://auth.your-company.com/api/router/v1/models

When something fails

Re-pinning an account's environment takes effect on its next request, including for a bearer already issued — the environment is read from the account on every request, not stamped into the token, so there is nothing to re-mint.

For administrators

Enable models

Settings → Platform → Extensions → the provider → Includes. What you select there is what developers can reach.

Three states, and the difference matters:

See what it costs

Settings → Governance → Cost Insights → Usage, with the source set to AI Gateway.

Router traffic is metered with its own source, separate from application API traffic, so the per-principal table answers "who is using this and what is it costing me" — a person or a service account — without an application's workload muddying the numbers. Each row carries calls, tokens, indicative spend, and modelled CO₂.

The breakdown switches between Principal and Model, the tool table lists what was reached through the MCP endpoint, and the budget line at the top of the page shows the organisation cap. The same data is available through billing.summary with filters: { source: "router" }.

Set spend caps

Cost Insights → Limits. Every cap sits beside the scope's spend so far this period, with the highest team or developer named. A cap can be set on a developer, a team, or the whole organisation, measured over a rolling day or month, against either indicative spend or call count.

• Caps are evaluated from the usage ledger, so they hold across every replica.
• Enforce refuses further calls; Audit records the breach and lets the call through.
• A blank field means uncapped. 0 means capped at zero — that distinction is real.
• When a cap is reached the developer gets a 429 whose Retry-After is the actual time until the period resets, so a well-behaved client backs off properly instead of hammering.

Choose a DLP posture

Cost Insights → Limits, under Router guardrails.

DLP runs over every prompt, and for a chat message that is straightforwardly right. A coding agent's prompt is a different shape: it carries the working set of a repository, re-sent every turn, and source code is adversarial input for rules tuned on prose — a credential pattern matches test fixtures, .env.example, and base64 in a lockfile.

So the router's posture is a decision you make rather than one you inherit:

Audit is not "DLP off." Every rule still runs, every finding still lands in the audit log, and every dlp.violation an auditor reads is still produced. The only thing it changes is whether a finding refuses the turn. Changing it affects the router only; chat, workflows, and every application surface keep the organisation policy.

Enforce streams too. A streamed answer under an enforce posture is governed in windows: DLP scans roughly every 512 characters, holds the last 256 back until the next window confirms them, and releases the rest — so text flows while the model is still writing instead of arriving all at once when it finishes. A finding later in the answer still stops it: the stream ends with a policy error after the clean prefix was delivered, and the text that carried the finding is never sent. Moderation in enforce mode still judges the whole answer before releasing it, because its judge is a model call.

Define routes — virtual models that pick a real one

A route is a model id you define. A developer names it (ANTHROPIC_MODEL=team/agent, or --model team/agent) and an algorithm picks the real, enabled, allowlisted model for each turn. Nothing is substituted unless the developer asked for the route by name, and the model a route picks still has to pass your allowlist — a route cannot reach a model you did not enable.

Routes live in the organisation policy platform.llm_router.routes, written through the organisation policy API (organization.policy.set). Three route types:

{
  "version": 1,
  "routes": {
    "team/fast": { "type": "passthrough", "target": "anthropic/claude-sonnet-5" },
    "team/ab": {
      "type": "random",
      "targets": [
        { "model": "anthropic/claude-opus-5", "weight": 1 },
        { "model": "openai/gpt-5", "weight": 3 }
      ]
    },
    "team/agent": {
      "type": "stage_router",
      "capable": "anthropic/claude-opus-5",
      "efficient": "anthropic/claude-sonnet-5",
      "picker": "efficient_first",
      "confidenceThreshold": 0.5,
      "recentTurnWindow": 3
    }
  }
}

What is recorded: the usage event and audit row carry the model that actually served the turn and the route that chose it (route.id, route.type, the decision source and tier), and the Anthropic message_start frame names the served model, so Cost Insights shows the real split and an auditor can ask "which turns did team/agent send to the capable tier". Routes are not listed on /v1/models — name them directly. A policy that fails validation deactivates every route and is logged; the developer then gets the ordinary "model not enabled" answer.

Revoke a developer's access

Settings → Access Tokens. Router keys appear there by the name the developer gave them. Revoking one takes effect on the next call.

Removing the local copy with scrydon router forget does not revoke the key. If a key may have leaked, revoke it here.

What the router does with a request

1. Authenticate. The router key carries its own organisation and environment — the request never asserts a tenant, and the router strips any header that tries to.
2. Admit. Scope, deadline, data classification, clearance, and your spend caps.
3. Resolve a target. Your enabled provider, your allowlist, the connection bound to that environment, and its credential. A developer's tool never sees a vendor credential.
4. Apply guardrails. DLP, then moderation, under the posture you chose.
5. Dispatch. Through the same runtime every other capability call uses.
6. Record. One audit event and one usage event, attributed to the person or service account.

Steps 2 through 6 are not router code. They are the pipeline your platform already runs; the router's own job is steps 1 and the translation on either side of it.

Capacity and back-pressure

Each router pod accepts a bounded number of open turns at once (the chart value auth.llmRouter.maxInFlight, default 64; env LLM_ROUTER_MAX_IN_FLIGHT). A streamed turn counts until its stream ends or you disconnect; a non-streamed turn counts until its response is returned. Past that bound the router answers 429 with a Retry-After header — rate_limit_error in the Anthropic dialect, rate_limit_error / rate_limit_exceeded in the OpenAI dialect — and does not touch the database or start any work for the refused request. The Anthropic and OpenAI SDKs both honour Retry-After automatically.

This is a per-pod overload guard, not a per-developer quota: it exists so a saturated pod sheds load instead of slowing every stream and then running out of memory. Sustained 429s at steady state mean the deployment needs another replica or a higher CPU limit, not client-side retries. Per-developer and per-organisation limits are the spend caps above.

Operators can watch scrydon_llm_router_streams_in_flight, scrydon_llm_router_admission_rejections_total and the TTFT / duration histograms on the AI Gateway SigNoz dashboard shipped with the operations chart. Two histograms separate the platform from the provider: scrydon_llm_router_overhead_ms is a turn's wall time minus the time spent inside the provider call (queue wait, guardrails and audit settlement included), and scrydon_llm_router_provider_ttfb_ms is the provider's own time to first event. scrydon_llm_router_turns_total carries a response_class (success, retryable_error, other_error) and every provider attempt, retries included, is counted in scrydon_extension_outbound_attempts_total with the same classes — the difference between the two error rates is what the router's retries are worth.

Protocol coverage

Every dialect is a translation in front of the same pipeline; GET /api/router/v1/health lists the ones your deployment speaks.

Which URL do I point at?

Your platform URL — the one serving /api/router, usually auth.<your-company>. Not the application URL you sign in to every day.

The distinction matters more than it should, because the application origin proxies /api/auth. A login against it therefore succeeds, and nothing looks wrong until your editor reports a bare 404 from an endpoint that knows nothing about Scrydon.

The CLI handles this for you: type either URL and scrydon login asks the deployment which origin owns its auth, tells you if it redirected, and stores the right one. Every scrydon router subcommand accepts either URL too — when the one you typed has no stored credentials it follows the same redirect, says which URL it used, and continues. scrydon router setup then probes /api/router before minting a key, so a mismatch fails immediately with the cause named rather than silently later.

If you are configuring a client by hand, use the platform URL.

Related

• Billing and usage metering — the billing.* API behind Cost Insights
• DLP overview — what the guardrails scan for
• Governance usage — the wider Cost Insights surface