LLM

For AI model providers (OpenAI, Anthropic, …). Implements chat / completion / streaming and optionally exposes a token estimator and a dynamic model fetcher.

Define the capability

import { defineCapabilityLLM } from "@scrydon/sdk-authoring/extensions/authoring/define";

const llmCapability = defineCapabilityLLM({
  models: [
    {
      id: "my-model-v1",
      pricing: { input: 3.0, output: 15.0 }, // per 1M tokens
      contextWindow: 128000,
      capabilities: {
        temperature: { min: 0, max: 2 },
        toolUsageControl: true,
        nativeStructuredOutputs: true,
      },
    },
  ],
  runtime: {
    async executeRequest(request, executor, logger) {
      const response = await fetch("https://api.example.com/v1/chat", {
        method: "POST",
        headers: {
          Authorization: `Bearer ${request.apiKey}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          model: request.model,
          messages: request.messages,
          temperature: request.temperature,
        }),
      });
      return await response.json();
    },
  },
  tokenEstimator: {
    avgCharsPerToken: 4,
    estimate(text) {
      return { count: Math.ceil(text.length / 4), confidence: "medium" };
    },
  },
  // Top-level on LLMCapabilityConfig — not nested under a wrapper.
  async fetchModels(config) {
    // A request that gets no response (DNS, refused connection, a
    // certificate that cannot be verified) throws a classified failure.
    // A non-OK response is returned, so decide what best-effort means here.
    const response = await fetchDiscoveryEndpoint("https://api.example.com/v1/models", {
      headers: { Authorization: `Bearer ${config.apiKey}` },
    });
    if (!response.ok) return STATIC_MODELS;
    const data = await response.json();
    return data.models.map((m: any) => ({ id: m.id }));
  },
});

fetchDiscoveryEndpoint comes from @scrydon/sdk-authoring/extensions/extension-failure. Don't wrap the listing request in a blanket catch that returns your static list or []: an endpoint that never answered fails every model call on that connection too, and swallowing it certifies models nobody can call. When discovery throws, the platform keeps whatever static models you declared, marks the provider with a discoveryFailure that names the endpoint and the cause, and shows that reason beside the connection under Settings → Platform → Extensions. A response the server chose to send (a 404, a 401) is yours to interpret.

Wire it into a model family

An LLM is a MODEL FAMILY, not a toolkit member. defineModels({ family: "llm" }) carries the runtime, the model list, the default and the family-wide capability defaults in one provided item — which is also what the platform's model picker reads:

export default defineExtension({
  // ...
  provides: {
    models: [
      defineModels({
        id: "my-service-llm",
        name: "My Service",
        family: "llm",
        runtime: llmCapability,
        defaultModel: "my-model-v1",
        capabilities: {
          toolUsageControl: true,
          nativeStructuredOutputs: true,
        },
        models: [
          {
            id: "my-model-v1",
            pricing: { input: 3.0, output: 15.0, updatedAt: "2026-01-15" },
            capabilities: { temperature: { min: 0, max: 2 } },
            contextWindow: 128000,
          },
        ],
      }),
    ],
  },
});

The family's capabilities define defaults for every model in it, including models returned by dynamic discovery with an extension/model ID. A model's own capabilities fields override those defaults field by field; an explicit false therefore overrides a family-level true. Unknown model IDs do not inherit capabilities.

Dynamic discovery does not infer capabilities from a model name. Declare only behavior the vendor adapter can truthfully guarantee for every discovered model, and use model-level declarations when support differs by model.

Benchmarks

Runtime ownership and model selection

LLM archives implement the vendor adapter, but only Scrydon's platform capability host invokes it. Applications send messages, tool schemas, generation options, and an optional model to the platform. The platform resolves the target, selects an eligible environment-scoped connection, loads the selected archive, applies LLM DLP, records usage, and returns a normalized result.

The workflow model selector starts with Default. Default stores no model override and delegates to platform.defaultLlmExtension; choosing a model overrides only the model for that workflow. Connections and credentials are configured in Platform settings, scoped to the active environment, and never selected or stored by an LLM block. /api/providers rejects caller-supplied apiKey, oauthCredentialId, and binding fields.