LLM
Implement the LLM capability — chat, completion, streaming, token estimation, dynamic model discovery.
For AI model providers (OpenAI, Anthropic, …). Implements chat / completion / streaming and optionally exposes a token estimator and a dynamic model fetcher.
Define the capability
import { defineCapabilityLLM } from "@scrydon/sdk-authoring/integrations/define";
const llmCapability = defineCapabilityLLM({
models: [
{
id: "my-model-v1",
pricing: { input: 3.0, output: 15.0 }, // per 1M tokens
contextWindow: 128000,
capabilities: {
temperature: { min: 0, max: 2 },
toolUsageControl: true,
nativeStructuredOutputs: true,
},
},
],
runtime: {
async executeRequest(request, executor, logger) {
const response = await fetch("https://api.example.com/v1/chat", {
method: "POST",
headers: {
Authorization: `Bearer ${request.apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: request.model,
messages: request.messages,
temperature: request.temperature,
}),
});
return await response.json();
},
},
tokenEstimator: {
avgCharsPerToken: 4,
estimate(text) {
return { count: Math.ceil(text.length / 4), confidence: "medium" };
},
},
// Top-level on LLMCapabilityConfig — not nested under a wrapper.
async fetchModels(config) {
const response = await fetch("https://api.example.com/v1/models", {
headers: { Authorization: `Bearer ${config.apiKey}` },
});
const data = await response.json();
return data.models.map((m: any) => ({ id: m.id }));
},
});Wire it into a product
const myProduct = defineProduct({
// ...
capabilities: {
tools: [/* ... */],
runtimes: { llm: llmCapability },
},
});Vendors providing LLM capabilities also set the top-level llm field for the platform's model picker:
export default defineVendor({
// ...
llm: {
defaultModel: "my-model-v1",
capabilities: {
toolUsageControl: true,
nativeStructuredOutputs: true,
},
models: [
{
id: "my-model-v1",
pricing: { input: 3.0, output: 15.0, updatedAt: "2026-01-15" },
capabilities: { temperature: { min: 0, max: 2 } },
contextWindow: 128000,
},
],
},
});llm.capabilities defines vendor defaults for every model owned by that vendor, including models returned by dynamic discovery with a vendor/model ID. A model's own capabilities fields override those defaults field by field; an explicit false therefore overrides a vendor-level true. Unknown model IDs do not inherit capabilities.
Dynamic discovery does not infer capabilities from a model name. Declare only behavior the vendor adapter can truthfully guarantee for every discovered model, and use model-level declarations when support differs by model.
Benchmarks
| Benchmark | Direction |
|---|---|
| MMLU | higher is better |
| HumanEval | higher is better |
| GPQA | higher is better |
Add benchmarks to your model entries and the UI displays them alongside model name, context window, and pricing.
Runtime ownership and model selection
LLM bundles implement the vendor adapter, but only Scrydon's platform capability host invokes it. Applications send messages, tool schemas, generation options, and an optional model to the platform. The platform resolves the target, selects an eligible environment-scoped connection, loads the selected bundle, applies LLM DLP, records usage, and returns a normalized result.
The workflow model selector starts with Default. Default stores no model override and delegates to platform.defaultLlmIntegration; choosing a model overrides only the model for that workflow. Connections and credentials are configured in Platform settings, scoped to the active environment, and never selected or stored by an LLM block. /api/providers rejects caller-supplied apiKey, oauthCredentialId, and binding fields.