SDK Reference
Complete API reference for the @scrydon/sdk-authoring/extensions/authoring SDK
Installation
bun add -d @scrydon/sdk-authoring zodnpm install --save-dev @scrydon/sdk-authoring zodImport Paths
The SDK uses explicit subpath exports — there is no root barrel import.
// Definition helpers (all define* functions)
import {
defineExtension, defineToolkit, defineTool, defineBlock, defineTrigger,
defineModels, defineWebSearch, defineWebhooks, defineDiscovery,
} from "@scrydon/sdk-authoring/extensions/define";
// Runtime types (used in execute() signatures)
import type {
PureContext, PureContextAuth, ToolResponse, ExecutorResult,
RealtimeSession, RealtimeMessage,
} from "@scrydon/sdk-authoring/extensions/context";
// Credential narrowing helpers (throw CredentialKindError on kind mismatch)
import {
requireOAuthToken, requireOAuth,
requireApiKey, requireBasicAuth, requireBotToken,
} from "@scrydon/sdk-authoring/extensions/context";
// Manifest types (for advanced use cases)
import { CodeManifestSchema, type CodeManifest } from "@scrydon/sdk-authoring/extensions/manifest";defineExtension(config)
Top-level container for an extension. The entry point's default export must be a defineExtension() result.
interface ExtensionInput {
/** Unique vendor ID — lowercase alphanumeric + hyphens, starts with letter */
id: string;
/** Display name */
name: string;
/** Semantic version (e.g. "1.0.0") */
version: string;
/** Short description */
description?: string;
/** Hex brand color (e.g. "#FF6B35") */
color?: string;
/** SVG logo as a string */
logo: string;
/** Links shown on the extension's page */
links?: {
website?: string;
docs?: string;
releaseNotes?: string;
support?: string;
};
/** Categories (e.g. ["tools"], ["llm"], ["database"]) */
categories?: string[];
/** Whether this vendor is enabled by default for new orgs */
defaultEnabled?: boolean;
/** Whether this vendor supports system-mode (client_credentials) */
supportsSystemMode?: boolean;
/** Whether this vendor requires external network connectivity */
connectivity?: "cloud" | "local" | "hybrid";
/** Minimum requested isolation. Platform and organization floors may only strengthen it. */
isolation?: "memory" | "microvm";
/** Public HTTPS hosts that tool code may contact — exact, or `*.` wildcards. */
egress?: {
httpsHosts: string[];
};
/** Platform operations your tools need beyond their own code (see "Platform permissions") */
platformPermissions?: readonly ExtensionPlatformPermission[];
/** Authentication configuration */
auth?: AuthConfig;
/** Admin-configurable fields (e.g. base URL for self-hosted services) */
configFields?: ConfigFieldInput[];
/** Extension-level secrets (resolved per-invocation via Dapr Secret Store) */
secrets?: SecretInput[];
/** What this extension provides — at least one toolkit, model family or content kind */
provides: {
toolkits?: ToolkitDefinition[];
models?: ModelFamilyDefinition[];
};
/** Wire protocol IDs this extension makes available to orgs that enable it */
wireProtocols?: string[];
}Tool Isolation and Egress
isolation is a minimum request, not a backend selector. Exact reviewed Scrydon tools may use the
approved memory backend. Every other shipped or organization-installed tool has a platform
microvm floor, even if its manifest requests memory or omits the field. An organization policy
may strengthen an exact reviewed tool to microvm; neither setting can weaken the platform floor.
If the attested Runtime Plane backend is unavailable, the tool fails closed.
Declare every public HTTPS hostname a tool may contact:
export default defineExtension({
// ...
egress: {
httpsHosts: ["api.acme.example", "uploads.acme.example"],
},
});Hostnames must be lowercase DNS names without a scheme, path, or port.
When your API host is per-tenant, prefix an entry with *.:
export default defineExtension({
// ...
egress: {
// Covers acme.my.acme-crm.example and acme--dev.sandbox.my.acme-crm.example
httpsHosts: ["*.my.acme-crm.example"],
},
});A *. entry matches subdomains at any depth but not its own apex — add the apex separately if
your tools contact it. The wildcard must cover at least two labels, so *.example is rejected: no
manifest may claim an entire TLD. The *. prefix is the only wildcard form; a variable segment
inside a label (*-api.acme.example) is not expressible, so declare the parent domain instead.
Prefer exact hosts wherever the destination set is fixed — a wildcard is for hosts you genuinely
cannot know until runtime.
At execution the platform intersects this list with organization egress policy; a wildcard here never widens what the organization permits. Direct Bun/Node sockets, alternate ports, private addresses, DNS rebinding, and redirect escapes are denied at the vendor microVM boundary. Outbound DLP scans the materialized request that would leave the VM, not only the tool's input arguments.
Platform permissions
Some platform operations act for the organization beyond your extension's own code. Your extension declares each one it needs. An organization consents by installing or enabling the extension: the admin sees what it declares on the Connect screen, on the extension's Version & security tab, and on its catalog row beside Enable. There is no separate approval step.
import { EXTENSION_PLATFORM_PERMISSIONS } from "@scrydon/sdk-authoring/extensions/types";
export default defineExtension({
id: "acme",
// …
platformPermissions: [EXTENSION_PLATFORM_PERMISSIONS.CORTEX_MESSAGES_SEND],
});| Permission | What it lets your tools do | Without it |
|---|---|---|
cortex.messages.send | Call ctx.action.collaboration.post to post in a Cortex direct message or channel as Cortex, on behalf of the run's Run As. Offer the Person and Channel pickers (selectorKey: "cortex.people" or "cortex.channels"). | The call fails with an Error whose message starts with extension_permission_not_granted:. Match on that prefix. |
The platform SDK (ctx.action.*, ctx.data.*, ctx.ai.*) is available today only to reviewed Scrydon tools. A tool of an extension your organization installs runs isolated without it, and any SDK call there fails with Platform SDK unavailable in vendor tool execution. A reviewed tool keeps the SDK when the organization or its manifest raises it to microVM isolation, for the operations the tool lists in platformOperations (see platformOperations). Each call is checked the same way as in the platform's own process. Declaring a permission still matters for every extension: the manifest refuses a Cortex recipient picker without cortex.messages.send, and the platform enforces the declaration wherever the platform SDK is available.
- The platform refuses the operation when your manifest does not declare the permission, or when the product your tool runs as is not enabled for the organization. An extension that is enabled by default counts as enabled until an admin disables it.
- Disabling the product withdraws the consent. The next call is refused, including a call made later in a run that is already in progress.
- If the platform cannot read whether the product is enabled, the call fails with a platform error and nothing is posted. That is not an
extension_permission_not_grantedrefusal. - The declaration is read from the version that runs. A version that stops declaring a permission loses it as soon as it runs.
- A release that newly declares a permission takes effect under the organization's update policy. Installing or enabling is the consent, and an update is neither, so the update review does not flag it.
- A block or trigger field that uses a Cortex picker key needs
cortex.messages.send. A manifest that uses one without declaring it is refused when it is built or uploaded. - Listing a permission twice is a manifest error.
Authentication Config
The auth field defines which credential types your vendor supports:
auth: {
credentials: {
none: { type: "none" },
},
default: "none",
}auth: {
credentials: {
apiKey: {
type: "apiKey",
label: "API Key",
description: "Your service API key",
headerName: "Authorization", // HTTP header name
headerPrefix: "Bearer", // Prefix before the key value
identityScope: "application", // "application" or "principal" — required for apiKey/basicAuth/oauthApp
},
},
default: "apiKey",
}auth: {
credentials: {
oauth: {
type: "oauth",
label: "Connect Account",
authorizationUrl: "https://example.com/oauth/authorize",
tokenUrl: "https://example.com/oauth/token",
scopes: ["read", "write"],
pkce: true,
useBasicAuth: false,
supportsRefreshTokenRotation: true,
},
},
default: "oauth",
}auth: {
credentials: {
botToken: {
type: "botToken",
label: "Bot Token",
description: "Your bot authentication token",
headerName: "Authorization",
headerPrefix: "Bot",
},
},
default: "botToken",
}You can define multiple credential types and let the user choose:
auth: {
credentials: {
oauth: { type: "oauth", /* ... */ },
apiKey: { type: "apiKey", /* ... */ },
},
default: "oauth", // Which one to show first
}Config Fields
For vendors that need admin-configured settings (e.g. a self-hosted base URL):
configFields: [
{
key: "baseUrl",
label: "Base URL",
placeholder: "https://your-instance.example.com",
description: "The base URL of your self-hosted instance",
required: true,
type: "url",
},
],Secrets
For vendor-level secrets that are stored in the Dapr Secret Store:
secrets: [
{
key: "WEBHOOK_SIGNING_SECRET",
label: "Webhook Signing Secret",
description: "Used to verify incoming webhook payloads",
required: true,
},
],Secrets are referenced at runtime via ctx.secrets["WEBHOOK_SIGNING_SECRET"]. In the governed
vendor-microVM tool path, credential and secret strings are invocation-scoped placeholders. Pass
them unchanged to the global fetch, which the sandbox routes through the trusted egress broker. The broker substitutes the real value only in an
allowed request location. Do not inspect, transform, persist, log, or return these values.
Connection-only extensions
auth and configFields describe what a connection to your extension holds — the form an
administrator fills in. An extension whose only members are data sources declares them the same
way, on defineExtension. It needs no toolkit:
export default defineExtension({
id: "home-assistant",
name: "Home Assistant",
version: "1.0.0",
logo: "./assets/icon.svg",
configFields: [
{ key: "baseUrl", label: "Home Assistant URL", type: "url", purpose: "endpoint", required: true },
],
auth: {
credentials: {
token: { type: "apiKey", label: "Long-lived access token", identityScope: "application" },
},
default: "token",
},
provides: { dataSources: [homeAssistantStates] }, // reads endpointRef "baseUrl", authRef "token"
});A source that needs no credential but takes its URL from the connection declares the endpoint
field and a none credential:
export default defineExtension({
// ...
configFields: [
{ key: "baseUrl", label: "Server URL", type: "url", purpose: "endpoint", required: true },
],
auth: { credentials: { none: { type: "none" } }, default: "none" },
provides: { dataSources: [publicFeed] }, // reads endpointRef "baseUrl", no authRef
});extension build writes the declaration to code/manifest.json with toolkits: [] and
models: []. The platform never executes that bundle: nothing in it can be called. An extension
declares a connection when it has at least one config field, or a credential of any type other
than none. With neither, the build emits no code/, and a source that names an endpoint field
or a credential is refused.
Both extension build and extension test --level static refuse a source whose endpointRef,
authRef or credentialRef names something the extension does not declare. The message says
which key to add. How a source reads these fields is described in
Endpoints from a connection.
A platform release that predates connection-only extensions refuses such an archive at upload
with code_validation_failed.
defineToolkit(config)
Groups tools, triggers and a block under a toolkit that can be enabled/disabled per organization. Model runtimes are NOT declared here — they belong to a model family, see defineModels() below.
interface ToolkitInput {
/** Unique provided-item ID within the extension */
id: string;
/** Display name */
name: string;
/** Short description */
description?: string;
/** Inline SVG string (falls back to the extension's logo if omitted) */
logo?: string;
/** Which credential from auth.credentials this toolkit uses */
credentialRef?: string;
/** OAuth scopes required by this toolkit */
credentialScopes?: string[];
/** Executable tools */
tools?: ToolDefinition[];
/** Triggers — rendered in the editor's "Triggers" section */
triggers?: TriggerDefinition[];
/** Main toolkit block on the canvas (omit for trigger-only toolkits) */
block?: BlockDefinition;
webhooks?: WebhookCapability;
discovery?: DiscoveryCapability;
webSearch?: WebSearchCapability;
/** Resource catalog; actions opt into exact resources separately */
resources?: Array<{
resourceType: string;
displayName: string;
description?: string;
allowMultiple?: boolean;
}>;
/** NPM package dependencies (for risk assessment at upload time) */
dependencies?: Array<{
type: "npm";
package: string;
version: string;
reason?: string;
}>;The dependencies array is for human-readable context — each entry includes a reason explaining why the dependency is needed. The CLI automatically generates a machine-readable SBOM (meta/sbom.cdx.json) that captures the complete set of archived packages with versions and licenses. Both are complementary: declared dependencies explain intent, the SBOM provides the auditable inventory.
/** System-mode (client_credentials) configuration */
systemMode?: {
supported: boolean;
permissions?: string[];
};
}System Consent Extensions
OAuth credentials can declare manifest-safe extension points for provider-specific setup flows such as Microsoft admin consent. Keep this logic owned by the extension: the manifest should describe the provider URL shape, the required organization config fields, and the callback query parameters.
For most providers, implement system consent with a declarative redirect template. This is the supported path when the platform only needs to build a URL from runtime values:
auth: {
credentials: {
oauth: {
type: "oauth",
label: "Microsoft Account",
authorizationUrl: "https://login.microsoftonline.com/{tenantId}/oauth2/v2.0/authorize",
tokenUrl: "https://login.microsoftonline.com/{tenantId}/oauth2/v2.0/token",
requiredConfig: ["tenantId"],
supportsClientCredentials: true,
clientCredentialScopes: ["https://graph.microsoft.com/.default"],
extensions: {
systemConsent: {
kind: "redirect-template",
urlTemplate: "https://login.microsoftonline.com/{tenantId}/adminconsent",
params: {
client_id: "{clientId}",
redirect_uri: "{redirectUri}",
scope: "{scope}",
state: "{state}",
},
requiredConfig: ["tenantId"],
callback: {
successParam: "admin_consent",
tenantParam: "tenant",
},
},
},
},
},
}The platform substitutes these placeholders:
| Placeholder | Value |
|---|---|
{clientId} | OAuth client ID resolved for the organization |
{redirectUri} | Platform admin-consent callback URL |
{scope} | Space-joined clientCredentialScopes |
{state} | CSRF state generated by the platform |
Any requiredConfig key, such as {tenantId} | Organization provider config value |
If a provider cannot be represented as a URL template, declare a handler reference in the manifest and implement the referenced method in the vendor runtime. The method should be pure: accept resolved runtime values, validate any provider-specific config it requires, and return the redirect URL instead of hardcoding provider logic in the platform.
extensions: {
systemConsent: {
kind: "handler",
handlerRef: "microsoft.systemConsent.buildUrl",
callback: {
handlerRef: "microsoft.systemConsent.callback",
},
},
}type SystemConsentBuildUrlInput = {
clientId: string;
redirectUri: string;
state?: string;
scopes: string[];
providerConfig: Record<string, string | undefined>;
};
export async function buildUrl(input: SystemConsentBuildUrlInput): Promise<string> {
const tenantId = input.providerConfig.tenantId;
if (!tenantId) {
throw new Error("Microsoft admin consent requires tenantId");
}
const url = new URL(`https://login.microsoftonline.com/${tenantId}/adminconsent`);
url.searchParams.set("client_id", input.clientId);
url.searchParams.set("redirect_uri", input.redirectUri);
url.searchParams.set("scope", input.scopes.join(" "));
if (input.state) {
url.searchParams.set("state", input.state);
}
return url.toString();
}
type SystemConsentCallbackInput = {
query: Record<string, string | undefined>;
providerConfig: Record<string, string | undefined>;
};
export async function callback(input: SystemConsentCallbackInput) {
return {
approved: input.query.admin_consent === "True",
tenantId: input.query.tenant,
};
}Handler references are manifest metadata. Uploaded archives should use
kind: "redirect-template" today unless the host application has explicitly
wired runtime execution for the referenced handler.
Multi-Toolkit Extensions
Extensions like Google or Microsoft provide several toolkits. Each toolkit gets its own block in the workflow editor and can be enabled independently:
export default defineExtension({
id: "acme",
// ...
provides: {
toolkits: [
defineToolkit({
id: "acme-crm",
name: "Acme CRM",
tools: [listContacts, createContact],
block: crmBlock,
triggers: [],
}),
defineToolkit({
id: "acme-email",
name: "Acme Email",
tools: [sendEmail, listInbox],
block: emailBlock,
triggers: [],
}),
],
},
});defineTool(config)
Defines the runtime logic for a tool. This is where your actual API calls, data transformations, or computations happen.
interface ToolInput<TInput, TOutput> {
/** Unique tool ID — convention: {vendor}:{product}:{action} */
id: string;
/** Display name */
name: string;
/** Semantic version */
version: string;
/** Description shown to users and LLM agents */
description?: string;
/** Zod schema for input validation */
input: z.ZodType<TInput>;
/** Zod schema for output validation */
output: z.ZodType<TOutput>;
/** Parameter metadata — controls visibility in the UI and to LLMs */
params?: Record<string, ParamDef>;
/** Exact product resources authorized before this action dispatches */
resourceAuthorization?: Array<{
required?: boolean;
oneOf: Array<{
resourceType: string;
source: { kind: "input"; key: string } | { kind: "binding" };
}>;
}>;
/** Platform SDK operations this tool calls, e.g. "action.sms.send" */
platformOperations?: PlatformDescriptorKey[];
/** The function that runs when this tool is invoked */
execute: (input: TInput, ctx: PureContext) => Promise<ToolResponse<TOutput>>;
}
interface ParamDef {
type: string;
required?: boolean;
description?: string;
default?: unknown;
/** Controls where this parameter appears */
visibility?: "user-only" | "user-or-llm" | "hidden";
}product.resources is a catalog, not a product-wide execution requirement. Declare each
action's actual target with resourceAuthorization. Input-sourced gates are required; only a
binding-sourced gate may set required: false, which preserves delegated account-root behavior
such as /me. Delegated targets are verified live with the provider. App/system targets require
an exact workspace assignment for the same product, resource type, and resource ID.
Manifest extraction preserves Zod descriptions and supported primitive
constraints such as string length and integer minimum/maximum bounds. For
z.preprocess(...), the generated JSON Schema describes the validated output
schema rather than the preprocessing transform, which cannot be represented in
JSON Schema. Keep that output schema faithful to the values the tool accepts;
its optionality and constraints are what workflow editors and Agent tool
descriptors consume.
platformOperations
List every platform SDK operation the tool calls, by its key: ctx.action.sms.send(...) is "action.sms.send". Only a reviewed Scrydon tool gets the platform SDK.
export const sendSms = defineTool({
id: "send",
name: "Send SMS",
version: "1.0.0",
platformOperations: ["action.sms.send"],
// …
});- When the tool runs isolated in a microVM, it can call only the operations it lists. Any other call fails with an
Errorwhose message says the operationis not declared by this tool (platformOperations). - An unknown or repeated key is a manifest error.
- Each call is subject to your organization's DLP and moderation policy and its spend caps. The policy applies to the call's input and its result, and past an enforced spend cap the call is refused before it runs, with an
Errorwhose message isPlatform operation quota was exceeded. - A tool that lists no operation runs isolated without the platform SDK.
- If the deployment cannot give an isolated tool its SDK, the tool fails with
TOOL_VENDOR_PLATFORM_SDK_UNAVAILABLEbefore it runs. It does not fail part-way through. - Some operations admit only one Scrydon product, whatever a manifest lists.
ctx.action.notification.notify("Notify me", see Notifications) is refused withexecution_refusedfor any tool that is not Scrydon's own Notify me: it notifies the run's Run As, so no other tool may send in their name.
description
Write a description for every tool. Cortex and the Agent block list only a few tools up front and find the rest with a tool search. That search ranks tools by name, description and product name, and returns the best eight. A tool with no description is ranked on its name alone, so the search may not find it, and the model has nothing to choose between similar tools.
- Start with a verb, and say what the tool does and what it returns.
- Name the key inputs, their defaults and their limits.
- Use the words people search with. A tool that reads mail should say "mail", "emails" and "inbox", not only "messages".
- For a write or destructive tool, state the side effect plainly: "Sends …", "Permanently deletes …".
When two tools match a search equally well, the one whose effect is read
is listed first.
Parameter Visibility
| Visibility | Who can provide the value | Use case |
|---|---|---|
user-or-llm | User sets it in the UI, or an AI agent generates it | Most parameters |
user-only | Only the user can configure it (hidden from LLM) | Connection settings, credentials |
hidden | Not shown in UI or to LLM — set programmatically | Internal IDs, computed values |
effect
Say what running the tool does. Assistants read it to decide whether they may run the tool without asking the person first.
| Value | Meaning | Example |
|---|---|---|
read | Only fetches data that already exists. | List messages, get an issue |
write | Creates, changes or sends something, or spends on the person's behalf. | Send mail, create an event, generate an image |
destructive | Deletes something, or cannot be undone. | Delete a file, purge a queue |
import { defineTool, TOOL_EFFECTS } from "@scrydon/sdk-authoring/extensions/define";
export const listMessages = defineTool({
id: "list-messages",
name: "List messages",
version: "1.0.0",
effect: TOOL_EFFECTS.READ,
// …
});effect is optional. A tool that does not declare one is treated as write, so an assistant asks before running it. When you are not sure, say write.
Execute Function
The execute function receives validated input and a PureContext:
import { requireOAuthToken } from "@scrydon/sdk-authoring/extensions/context";
async execute(input, ctx) {
// Use the logger (visible in workflow execution logs)
ctx.logger.info(`Processing request for org ${ctx.execution.orgId}`);
// Narrow on ctx.auth.kind before reading credential fields.
// This example assumes the vendor declares an OAuth credential —
// requireOAuthToken throws a CredentialKindError if the kind doesn't match.
const accessToken = requireOAuthToken(ctx.auth);
// Make authenticated API calls
const response = await fetch("https://api.example.com/data", {
headers: {
Authorization: `Bearer ${accessToken}`,
"Content-Type": "application/json",
},
body: JSON.stringify(input),
});
if (!response.ok) {
return {
success: false,
output: { error: `API returned ${response.status}` },
error: `HTTP ${response.status}: ${response.statusText}`,
};
}
const data = await response.json();
return {
success: true,
output: data,
metadata: { requestId: response.headers.get("x-request-id") },
};
},PureContext
The PureContext interface is the only interface between the platform and archive code. It is injected into every execute() call at runtime:
/**
* Discriminated union of all credential shapes the platform can inject.
* Always narrow on `kind` before reading credential fields.
* Import narrowing helpers from @scrydon/sdk-authoring/extensions/context:
* requireOAuthToken(auth) → string (accessToken, throws on mismatch)
* requireOAuth(auth) → { accessToken, refreshToken?, tokenType?, expiresAt? }
* requireApiKey(auth) → string (apiKey)
* requireBasicAuth(auth) → { username, password }
* requireBotToken(auth) → string (botToken)
* Each helper throws a CredentialKindError with an actionable message on mismatch.
*/
type PureContextAuth =
| { kind: "oauth"; accessToken: string; refreshToken?: string; tokenType?: string; expiresAt?: number }
| { kind: "apiKey"; apiKey: string }
| { kind: "basicAuth"; username: string; password: string }
| { kind: "botToken"; botToken: string }
| { kind: "none" };
interface PureContext {
/** Credential resolved by the platform before invocation */
auth: PureContextAuth;
/** Execution metadata for logging and correlation */
execution: {
executionId: string; // Unique execution run ID
workflowId?: string; // Workflow ID (if running in a workflow)
nodeId?: string; // Block node ID in the workflow
orgId: string; // Organization ID
workspaceId?: string; // Workspace ID
userId?: string; // User who triggered the execution
};
/** Structured logger — console calls are intercepted in the sandbox */
logger: {
info(message: string, ...args: unknown[]): void;
warn(message: string, ...args: unknown[]): void;
error(message: string, ...args: unknown[]): void;
debug(message: string, ...args: unknown[]): void;
};
/** Vendor secrets configured by org admin (resolved per-invocation via Dapr Secret Store) */
secrets: Record<string, string>;
/** Present when executing via an Extension Profile — contains profile-specific overrides */
profileConfig?: Record<string, unknown>;
}PureContext is imported from @scrydon/sdk-authoring/extensions/context. The platform constructs it before each tool invocation — your code never needs to create one manually. All fields except auth and execution may be empty objects if not applicable. auth is a kind-discriminated union (PureContextAuth); always narrow on auth.kind or use a require* helper before accessing credential-specific fields. See Authentication Modes for the full per-mode shape reference.
ToolResponse
Every execute() must return a ToolResponse:
interface ToolResponse<T = unknown> {
/** Whether the tool execution succeeded */
success: boolean;
/** The typed output data */
output: T;
/** Stable machine-readable error code */
error?: string;
/** Human-readable failure explanation; preferred over error for display */
message?: string;
/** Optional metadata (logged but not passed to downstream blocks) */
metadata?: Record<string, unknown>;
}defineBlock(config)
Defines the workflow editor UI — how the block appears, what form fields it has, and how data flows through it.
interface BlockInput {
/** Unique block type — lowercase with underscores (e.g. "acme_crm") */
type: string;
/** Display name in the block palette */
name: string;
/** Short description */
description?: string;
/** Sidebar section: "integration" for an extension's toolkit block ("trigger" is set by defineTrigger, "core" is built-in) */
category: string;
/** Background color in hex */
bgColor?: string;
/** Auth mode: "none", "apiKey", "oauth", "botToken" */
authMode?: string;
/** Form fields displayed in the block panel */
subBlocks?: SubBlockInput[];
/** Which tools this block can invoke */
tools?: {
access: string[];
config?: {
/** Dynamic tool selection based on user input */
tool: (params: Record<string, any>) => string;
/** Parameter transformation before passing to tool */
params?: (params: Record<string, any>) => any;
};
};
/** Input connections from upstream blocks */
inputs?: Record<string, { type: string; description?: string }>;
/** Output connections to downstream blocks */
outputs?: Record<string, { type: string; description?: string }>;
/** Trigger configuration (for trigger blocks) */
triggers?: { enabled: boolean; available: string[] };
}What outputs declares
A block's output at run time is the output payload its tool returned, with error added when
the tool failed. It is not the { success, output } object execute() returns: a downstream
block reads <my_block.records>, never <my_block.output.records> or <my_block.success>. A
payload that is not an object (an array, a string) arrives under value.
Declare in outputs the top-level keys of your tools' output schemas, plus error. The editor
offers exactly these as <my_block.key> references, so a key no tool returns is a reference that
always resolves to nothing. First-party extensions are held to this by a test that compares each
block's outputs with its tools' output schemas.
const listRecords = defineTool({
// ...
output: z.object({ records: z.array(z.record(z.string(), z.unknown())), nextLink: z.string().optional() }),
});
defineBlock({
// ...
outputs: {
records: { type: "json", description: "Rows returned by List Records" },
nextLink: { type: "string", description: "Link to the next page, when there is one" },
error: { type: "string", description: "Error message when the operation failed" },
},
});SubBlock Types
SubBlocks are the form fields that appear when a user clicks on a block:
| Type | Description | Key Props |
|---|---|---|
short-input | Single-line text input | placeholder, password, defaultValue, generatedDefault, readOnly, showCopyButton |
long-input | Multi-line textarea | placeholder |
code | Code editor with syntax highlighting | placeholder, mode, generationType |
dropdown | Select menu | options: [{ label, id }] |
combobox | Searchable dropdown | options, fetchOptions |
slider | Range slider | min, max, step |
switch | Toggle switch | defaultValue |
oauth-input | OAuth credential selector | serviceId, requiredScopes |
SubBlock Definition
interface SubBlockInput {
/** Unique field ID within the block */
id: string;
/** Field label */
title?: string;
/** SubBlock type */
type: string;
/** Layout: "full" (default) or "half" */
layout?: "full" | "half";
/** Whether this field is required */
required?: boolean;
/** Placeholder text */
placeholder?: string;
/** Mask input as password dots */
password?: boolean;
/** Default value */
defaultValue?: unknown;
/** Generate a fresh value once when each new block is created */
generatedDefault?: "uuid";
/**
* For `type: "code"` fields only: what kind of content this field holds
* (e.g. `"json-object"`, `"javascript-function-body"`, `"system-prompt"`).
* Drives both the editor's default placeholder and the AI wand's prompt.
*/
generationType?:
| "javascript-function-body"
| "typescript-function-body"
| "json-schema"
| "json-object"
| "system-prompt"
| "custom-tool-schema"
| "sql-query"
| "postgrest"
| "mongodb-filter"
| "mongodb-pipeline"
| "mongodb-sort"
| "mongodb-documents"
| "mongodb-update"
| "neo4j-cypher"
| "neo4j-parameters";
/** Merge this field's JSON object into the tool input (see below) */
spreadIntoToolInput?: boolean;
/** Send this field as a tool input of another name (see below) */
canonicalParamId?: string;
/** Options for dropdown/combobox */
options?: Array<{ label: string; id: string }>;
/** Help text shown below the field */
description?: string;
/** Conditional visibility based on another field's value */
condition?: { field: string; value: string };
/** Whether the field is read-only */
readOnly?: boolean;
/** Show a copy-to-clipboard button */
showCopyButton?: boolean;
/** Hide this field from the UI */
hidden?: boolean;
}Use generatedDefault: "uuid" when every newly added block needs its own
initial value, such as a webhook secret. The declarative marker survives the
extension manifest's JSON boundary; the generated UUID is stored as the
block's ordinary field value and is not regenerated when the workflow loads.
generationType on code fields
Set generationType on a type: "code" field to declare what kind of content
it holds. This is what the workflow editor uses to pick a default placeholder
and to prompt the AI wand correctly — without it, a code field with no
sibling language: "json" value falls back to showing a JavaScript
function-body template, even for a field that is really a JSON object:
{
id: "params",
title: "Operation Parameters (JSON)",
type: "code",
generationType: "json-object",
placeholder: '{ "key": "value" }',
}If you set your own placeholder, it is always shown — generationType only
supplies a placeholder when you have not. The supported values are
"javascript-function-body", "typescript-function-body", "json-object",
"json-schema", "system-prompt", "sql-query", "custom-tool-schema", and
a handful of database-specific query shapes ("postgrest",
"mongodb-filter", "mongodb-pipeline", "mongodb-sort",
"mongodb-documents", "mongodb-update", "neo4j-cypher",
"neo4j-parameters"). This field is optional and additive: extensions built
before it existed keep working unchanged.
Dynamic Tool Selection
A block whose tools.access lists more than one tool must declare a
tools.toolSelector, which maps each option of a dropdown sub-block to the tool
it runs. extension build and extension test --level static refuse a
multi-tool block without one (static check S19): the platform would otherwise
call the first tool in access for every operation the user picks.
defineBlock({
// ...
subBlocks: [
{
id: "operation",
title: "Operation",
type: "dropdown",
options: [
{ label: "Create", id: "create" },
{ label: "Read", id: "read" },
{ label: "Update", id: "update" },
{ label: "Delete", id: "delete" },
],
},
// ... other fields
],
tools: {
access: ["acme_create", "acme_read", "acme_update", "acme_delete"],
toolSelector: {
param: "operation",
map: {
create: "acme_create",
read: "acme_read",
update: "acme_update",
delete: "acme_delete",
},
default: "acme_read",
},
},
});Catch-all JSON Parameters
A block with many operations usually cannot give every tool parameter its own
field, because the parameters differ per operation. Give the common ones a
field, and collect the rest in one JSON field marked spreadIntoToolInput:
defineBlock({
// ...
subBlocks: [
{ id: "operation", title: "Operation", type: "dropdown", options: [/* ... */] },
{ id: "owner", title: "Repository Owner", type: "short-input" },
{
id: "params",
title: "Operation Parameters (JSON)",
type: "code",
placeholder: '{ "title": "...", "body": "..." }',
spreadIntoToolInput: true,
},
],
});When the block runs, the platform parses that field and merges its keys into
the tool input, so { "title": "Bug" } reaches the tool as title, not as a
string under params. The value is read after workflow references
resolve, so it can be a single reference to an object another block produced
— <compose.result.params> — which is the reliable way to pass text
containing quotes or newlines.
| Rule | Behaviour |
|---|---|
| A filled-in field and a JSON key share a name | The field wins. An empty field does not. |
| The JSON is invalid, or is not an object | The block fails with TOOL_INPUT_SPREAD_INVALID; the tool is not called. |
A key names a credential or connection (accessToken, apiKey, credentialId, …) or starts with _ | The block fails. Credentials come from the connection, never from block parameters. |
| The field is empty | Nothing is merged. |
This replaces a tools.config.params function. A manifest cannot carry a
function, so such a mapper never runs in a workflow: the JSON field would
reach your tool as one unparsed string.
From Fields to Tool Inputs
A field reaches the tool as the input of the same name. Everything else a block needs to hand its tool the right input is declared, not computed:
| You need | Declare |
|---|---|
A field sent under another input name (fileIdManual → fileId) | canonicalParamId: "fileId" on the field. Several fields may share one canonicalParamId; the one the selected operation shows, and the user filled in, is sent. |
| One value sent as differently named inputs per operation | One field per input, each with a condition on the operations that use it. |
| A number, boolean, list or object | Nothing. Declare the type in the tool's input schema: the platform converts the field's text to it before the tool runs (below). |
A different shape ("2026-10-21T09:00:00Z" vs { dateTime }) | Accept both in the tool's input schema and normalize inside execute(). |
A field on the canvas stores text. When a block runs, the platform converts
each text value to the type the tool's input schema declares for it:
| The schema declares | The field holds | The tool receives |
|---|---|---|
number / integer | 10, -1.5, 2e3 | 10, -1.5, 2000 (an integer must be whole) |
boolean | true / false, any case | true / false |
array / object | JSON text | The parsed array or object |
| An array of strings | PROJ-1, PROJ-2 | ["PROJ-1", "PROJ-2"] (use JSON for an item containing a comma) |
| Anything but a string | Nothing, or only spaces | No input at all, so the tool's default applies — never 0, false or [] |
| A string, or no type | Anything | The text unchanged |
A value that does not convert cleanly (ten for a number) is passed through
unchanged, so the tool's own validation reports it. Only a block's own tool
call is converted: an Agent's tool arguments are already typed.
tools.config.tool and tools.config.params functions are rejected by
extension build. They cannot be written to manifest.json, so the
platform never ran them: routing fell back to the first tool, and every
rename and conversion they did was silently skipped.
Conditional Visibility
Show or hide fields based on other field values:
subBlocks: [
{
id: "authType",
title: "Auth Type",
type: "dropdown",
options: [
{ label: "API Key", id: "apiKey" },
{ label: "Username/Password", id: "basic" },
],
},
{
id: "apiKey",
title: "API Key",
type: "short-input",
password: true,
condition: { field: "authType", value: "apiKey" },
},
{
id: "username",
title: "Username",
type: "short-input",
condition: { field: "authType", value: "basic" },
},
{
id: "password",
title: "Password",
type: "short-input",
password: true,
condition: { field: "authType", value: "basic" },
},
],defineTrigger(config)
Defines a trigger block — a first-class workflow member that renders in the "Triggers" sidebar section and carries runtime metadata describing how it fires (webhook, polling, or both). The category is auto-set to "trigger".
interface TriggerInput {
/** Unique block type */
type: string;
/** Display name */
name: string;
/** Description */
description?: string;
/** Background color in hex */
bgColor?: string;
/** Auth mode: "none", "apiKey", "oauth", "botToken" */
authMode?: string;
/** Form fields for trigger configuration */
subBlocks?: SubBlockInput[];
/** Which tools this trigger block can invoke */
tools?: {
access: string[];
config?: {
tool?: (params: Record<string, any>) => string;
params?: (params: Record<string, any>) => any;
};
};
/** Input connections */
inputs?: Record<string, { type: string; description?: string }>;
/** Output schema — what data the trigger emits */
outputs?: Record<string, { type: string; description?: string }>;
/** Provider identifier */
provider?: string;
/** Semantic version */
version?: string;
/** Runtime configuration — how events reach the workflow engine */
runtime?: {
webhook?: {
method?: "POST" | "GET" | "PUT" | "DELETE";
headers?: Record<string, string>;
};
polling?: PollingProvider;
};
}Secrets Management
Vendors can declare secrets that are stored encrypted at rest and resolved per-invocation. Secrets are scoped to the organization and vendor, so each org has its own isolated set of credentials.
Declaring Secrets
Define secret requirements in defineExtension() using the secrets array:
export default defineExtension({
id: "acme",
name: "Acme",
version: "1.0.0",
logo: acmeIcon,
auth: { credentials: { apiKey: { type: "apiKey", label: "API Key", identityScope: "application" } }, default: "apiKey" },
secrets: [
{
key: "WEBHOOK_SIGNING_SECRET",
label: "Webhook Signing Secret",
description: "Used to verify incoming webhook payloads",
required: true,
},
{
key: "ENCRYPTION_KEY",
label: "Encryption Key",
description: "AES key for encrypting sensitive payloads",
required: false,
},
],
provides: { toolkits: [/* ... */] },
});Each SecretInput has the following shape:
interface SecretInput {
/** Unique key within the vendor — must match ^[a-zA-Z0-9_-]+$ */
key: string;
/** Human-readable label shown in the admin UI */
label: string;
/** Help text describing what this secret is used for */
description?: string;
/** Whether the extension requires this secret to function */
required: boolean;
}Secret keys must contain only alphanumeric characters, hyphens, and underscores (^[a-zA-Z0-9_-]+$). Keys that do not match this pattern are rejected at save time.
Secret Key Format
Secrets are stored using a structured key that scopes them to the organization and vendor:
extension:{orgId}:{extensionId}:{key}For example, a secret with key WEBHOOK_SIGNING_SECRET for vendor acme in organization org_abc123 is stored as:
extension:org_abc123:acme:WEBHOOK_SIGNING_SECRETThis ensures complete isolation between organizations and between vendors within the same organization.
Encryption at Rest
All secret values are encrypted with AES-256-GCM before being persisted. The encryption key strategy is configured per organisation — LOCAL, BYOK, or HYOK. Secret values are never stored in plaintext. See Security → Secrets management for the full model.
Accessing Secrets at Runtime
ctx.secrets is a flat Record<string, string> containing configured keys. In the governed
vendor-microVM path, each value is an invocation-scoped placeholder backed by a sealed broker grant:
import { defineTool } from "@scrydon/sdk-authoring/extensions/define";
import { z } from "zod";
export const verifyWebhook = defineTool({
id: "acme:webhooks:verify",
name: "Verify Webhook",
version: "1.0.0",
input: z.object({ payload: z.string(), signature: z.string() }),
output: z.object({ valid: z.boolean() }),
async execute(input, ctx) {
const signingSecret = ctx.secrets["WEBHOOK_SIGNING_SECRET"];
if (!signingSecret) {
return { success: false, output: { valid: false }, error: "Webhook signing secret not configured" };
}
const response = await fetch("https://api.acme.example/webhooks/verify", {
method: "POST",
headers: {
"content-type": "application/json",
"x-acme-signing-secret": signingSecret,
},
body: JSON.stringify(input),
});
return { success: true, output: { valid: response.ok } };
},
});Pass placeholder values unchanged to the global fetch. There is no ctx.fetch: inside the sandbox the global fetch is routed through the egress broker. The trusted broker substitutes the reusable value
after destination authorization and before outbound DLP scans the actual request. The guest,
filesystem, model, plan, logs, traces, and tool response never receive the reusable value.
The object contains only keys set by the organization admin; absent values remain absent. Local operations that must inspect or transform a reusable secret, such as HMAC signing inside non-Scrydon Agent tool code, are intentionally unavailable on this boundary.
Admin UI
Organization administrators configure secrets under Settings > Platform > Extensions, in the vendor's sheet. The UI displays the declared secret fields from the vendor's secrets array.
Secret values are write-only -- after a value is saved, it is never displayed again. The UI only shows which keys have been configured (with a masked indicator) and allows admins to update or delete them.
API Endpoints
The platform exposes three internal admin endpoints for secret management:
| Endpoint | Purpose |
|---|---|
/extensions/authoring/admin/archive-secret-set | Create or update a secret value |
/extensions/authoring/admin/archive-secret-delete | Delete a secret |
/extensions/authoring/admin/archive-secret-list | List configured secret keys (values are never returned) |
These endpoints are internal (service-to-service) and require the caller to be an authorized internal service. They are not directly accessible from client applications.
Build CLI
# Build an archive
sdk-authoring extension build [--entry src/index.ts] [--outDir dist]
# Test an archive
sdk-authoring extension test [--level static|sandbox|live] [--tool <id>]Build Output
my-vendor-1.0.0.archive.tar.gz
├── manifest.json # Auto-generated metadata (JSON Schemas, UI defs)
├── dist/
│ └── index.js # Compiled, minified ESM (all deps inlined via esbuild)
├── meta/
│ ├── sbom.cdx.json # CycloneDX 1.6 SBOM (auto-generated)
│ └── metafile.json # esbuild dependency graph
└── assets/ # Optional icons
└── icon.svgThe build CLI inlines all npm dependencies into the single dist/index.js file and auto-generates a CycloneDX 1.6 SBOM at meta/sbom.cdx.json. There is no runtime npm install — the archive is fully self-contained and deterministic.
Vendor ID Rules
- Lowercase alphanumeric characters and hyphens only
- Must start with a letter
- Pattern:
/^[a-z][a-z0-9-]*$/ - Examples:
acme,my-vendor,hello-world
Tool ID Convention
Tool IDs use the fully qualified format {extensionId}:{providedId}:{action} at runtime:
hello-world:hello-world:say-hellogoogle:sheets:readslack:messaging:send-message
Shorthand: In authoring code you can use just the action part (e.g. "say-hello", "read"). The SDK auto-qualifies short IDs by prepending {extensionId}:{providedId}: during defineExtension(). All first-party extensions use the short form. If you use the full form, the vendor/product prefix must match — the build CLI validates this.
Capability Helpers
Model capabilities are declared one MODEL FAMILY at a time with defineModels(); the family's runtime is a plain object typed by its capability config. The three capabilities that are not model families sit on a toolkit and each keeps a small tag wrapper so the manifest extractor can identify it.
| Helper | Declares | Runtime type (@scrydon/sdk-authoring/extensions/capabilities/<name>) |
|---|---|---|
defineModels({ family: "llm", runtime, models, defaultModel }) | Full LLM provider (chat, completion, streaming) | LLMCapabilityConfig (llm) |
defineModels({ family: "stt", runtime }) | Speech-to-text (batch and realtime) | STTCapabilityConfig (stt) |
defineModels({ family: "tts", runtime }) | Text-to-speech (batch and realtime) | TTSCapabilityConfig (tts) |
defineModels({ family: "embedding", runtime }) | Vector embedding generation | EmbeddingCapabilityConfig (embedding) |
defineModels({ family: "image", runtime }) | Image generation | ImageCapabilityConfig (image) |
defineModels({ family: "video", runtime }) | Video generation | VideoCapabilityConfig (video) |
defineModels({ family: "ocr", runtime }) | Document parsing / optical character recognition | OCRCapabilityConfig (ocr) |
defineModels({ family: "moderation", runtime }) | Content-safety classification | ModerationCapabilityConfig (moderation) |
defineWebSearch(config) — toolkit webSearch | Provider-scoped internet search | WebSearchCapabilityConfig (web-search) |
defineWebhooks(config) — toolkit webhooks | Vendor-specific webhook handling | WebhookCapabilityConfig (webhooks) |
defineDiscovery(config) — toolkit discovery | Resource browsing, selectors, access scopes | DiscoveryCapabilityConfig (discovery) |
All four helpers are imported from @scrydon/sdk-authoring/extensions/define:
import {
defineModels,
defineWebSearch,
defineWebhooks,
defineDiscovery,
} from "@scrydon/sdk-authoring/extensions/define";Each defineModels() result is one entry in provides.models; webSearch, webhooks and discovery are toolkit members instead. The manifest extractor converts runtime functions to boolean flags and extracts model metadata (dimensions, benchmarks, pricing) into the manifest's capabilityMeta field. See Capabilities for full usage examples of each family.
Manifest Schema
The build CLI auto-generates manifest.json from your defineExtension() call. The manifest is a JSON-serializable snapshot of your vendor's metadata, stripped of runtime functions. It follows the sole supported manifestVersion: 1 shape. Any other version is rejected.
Top-Level Structure
{
"$schema": "...", // Optional JSON Schema URL
"manifestVersion": 1, // Always 1
"version": "1.0.0", // Vendor version (from defineExtension)
"vendor": { ... }, // Vendor metadata
"auth": { ... }, // Authentication config
"products": [ ... ], // Product definitions
"protocols": [ ... ], // Protocol definitions (optional)
"llm": { ... } // LLM provider config (optional)
}Vendor Section
Contains display metadata. Runtime code is excluded.
| Field | Type | Required | Description |
|---|---|---|---|
id | string | Yes | Vendor ID (/^[a-z][a-z0-9-]*$/) |
name | string | Yes | Display name |
description | string | No | Short description |
color | string | No | Hex brand color |
icon | string | Yes | SVG icon as string |
website | string | No | Vendor website URL |
docsUrl | string | No | Documentation URL |
categories | string[] | No | Category tags |
defaultEnabled | boolean | No | Auto-enable for new orgs (default: false) |
supportsSystemMode | boolean | No | Supports client_credentials (default: false) |
connectivity | "cloud" | "local" | "hybrid" | No | Network requirements |
configFields | ConfigField[] | No | Admin-configurable fields |
secrets | Secret[] | No | Vendor-level secrets |
Product Section
Each product in the products array contains:
| Field | Type | Required | Description |
|---|---|---|---|
id | string | Yes | Product ID |
name | string | Yes | Display name |
capabilities.tools | Tool[] | No | Tool definitions with JSON Schema input/output |
capabilities.triggers | Trigger[] | No | Trigger definitions |
capabilities.runtimes | object | No | Boolean flags for every selectable runtime: llm, embedding, stt, tts, image, video, ocr, moderation, webSearch |
capabilities.capabilityMeta | object | No | Extracted model metadata (embedding dimensions, STT/TTS models, benchmarks) |
capabilities.webhooks | boolean | No | Whether the product handles webhooks |
capabilities.discovery | boolean | No | Whether the product supports resource discovery |
workflow.block | Block | No | Workflow editor block definition |
workflow.triggers | TriggerBlock[] | No | Workflow trigger blocks |
resources | Resource[] | No | Product resource catalog; never a product-wide action gate |
dependencies | Dependency[] | No | Declared NPM dependencies with reasons |
systemMode.permissions | string[] | No | App/system consent permissions |
How Runtime Code Becomes Manifest Data
The build CLI transforms runtime definitions into manifest-safe JSON:
| Source (defineToolkit) | Manifest Output |
|---|---|
runtimes.llm (with executeRequest function) | runtimes.llm: true |
runtimes.embedding (with embed function, models with dimension) | runtimes.embedding: true + capabilityMeta.embeddingModels |
runtimes.stt (with transcribe function, models) | runtimes.stt: true + capabilityMeta.sttModels |
runtimes.tts (with synthesize function, models) | runtimes.tts: true + capabilityMeta.ttsModels |
runtimes.image (with generation function and models) | runtimes.image: true + image model metadata |
runtimes.video (with generation function and models) | runtimes.video: true + video model metadata |
runtimes.ocr (with extraction function and models) | runtimes.ocr: true + OCR model metadata |
runtimes.moderation (with classification function and models) | runtimes.moderation: true + moderation model metadata |
runtimes.webSearch (with search function) | runtimes.webSearch: true; provider-scoped, with no model policy |
Tool input/output (Zod schemas) | JSON Schema objects |
Tool execute function | Omitted (not serializable) |
Block tools.toolSelector | Preserved as JSON-safe routing metadata |
Sub-block spreadIntoToolInput | Preserved; the platform merges that field's JSON object into the tool input at run time |
Block tools.config.tool / tools.config.params functions | Rejected: the build fails (static check S19) |
The manifest is what the platform reads at boot time to populate the extension catalog. It contains no executable code -- only metadata, JSON Schemas, and UI definitions. The executable archive is loaded lazily through the canonical backend. Capability/reviewed paths may use the Worker Thread compatibility backend; a static non-Scrydon tool selected by an isolated Agent requires the source-pinned, proof-gated vendor microVM.
Extension manifests cannot contain function closures. Use tools.toolSelector for routing, canonicalParamId to rename a field's input, a spreadIntoToolInput field for per-operation parameters, and typed tool input schemas — the platform converts field text to those types. The build and static-test commands fail on a tools.config.tool or tools.config.params function, and on a multi-tool block without a tools.toolSelector.
Continuous speech-to-text sessions
An STT extension that declares streaming models must implement both
createStreamingSession(request) and openStreamingSession(request, context).
The build validates this pairing. streaming.protocol is an extension-owned
identifier; adding a provider does not require a recording-app protocol adapter.
Import the session types from
@scrydon/sdk-authoring/extensions/capabilities/stt-streaming:
import type {
SttSession,
SttSessionContext,
SttStreamingSessionRequest,
} from "@scrydon/sdk-authoring/extensions/capabilities/stt-streaming";
// Signature of runtime.openStreamingSession:
type OpenSpeechSession = (
request: SttStreamingSessionRequest,
context: SttSessionContext,
) => Promise<SttSession>;SttSession provides sendAudio(Uint8Array), finalize(), close(),
onTranscript(handler), onError(handler), and onClose(handler).
Audio is continuous mono 16 kHz PCM16. The extension owns provider authentication,
resampling, framing, credential renewal, and SDK dependencies. Keep SDKs compatible
with the sandbox's brokered fetch and WebSocket; native networking modules
cannot bypass these controls. The sandbox exposes the web-standard globals
(fetch, WebSocket, crypto.subtle, CompressionStream,
DecompressionStream, TextEncoder, …) plus the Node builtins on the
bundler's allowlist (node:crypto, node:buffer, node:stream, node:util,
node:url, node:path, node:events, node:string_decoder,
node:querystring). node:zlib is not available — its runtime initialiser
reaches for process.binding, which the sandbox denies — so inflate with
DecompressionStream("gzip") or a pure-JS library such as fflate. Honor context.signal and close provider resources
when the session ends. context.auth, context.secrets, and
context.profileConfig use the canonical extension context, so authentication
is not restricted to API keys. finalize() must drain accepted audio before reporting
completion.
Emit { text, isFinal, startMs?, endMs?, confidence?, speakerId? }. Timestamps
are milliseconds from this session's audio origin. Omit unsupported timing or
attribution; do not invent it. A provider's speaker label identifies a speaker
only within the continuous session that produced it. It cannot identify an
organization member or establish identity across separate STT requests.
The platform returns a scrydon-stt-v1 connection grant to media clients. Provider
endpoints and credentials remain inside the extension runtime, including the
direct and proxied values returned by createStreamingSession.
Continuous sessions currently require reviewed extension code admitted to the worker execution backend. If organization policy requires a managed-process or microVM backend, streaming is refused before a connection ticket is issued; those backends currently support request/response execution. Batch transcription continues to use its normal supported execution tiers. Do not weaken organization isolation policy to make an unsupported streaming deployment appear available.
When upgrading, rebuild and reinstall streaming extension bundles with the updated
authoring SDK. Ticket-only bundles lack the required session hook. Deploy the
platform and media client together; the new client requires scrydon-stt-v1
and does not fall back to provider protocols in the recording app.
First-party OpenAI speech models
The OpenAI extension implements the same session contract as other extensions. Configure its OpenAI API-key connection and choose a model under the STT capability:
| Model | Input path | Timing and speakers |
|---|---|---|
gpt-live-transcribe | Native live session | Turn boundaries from server VAD; no word timestamps or provider speaker labels |
gpt-transcribe | File / utterance | Text and detected languages; no invented timestamps |
gpt-4o-transcribe, gpt-4o-mini-transcribe | File / utterance | Text; no word timestamps |
gpt-4o-transcribe-diarize | File / utterance | Timed segments with request-local speaker labels |
whisper-1 | File / utterance | Word or segment timestamps; English translation |
The existing default remains whisper-1. Live and file models are separate
selections; a live-only model cannot serve a final file-transcription request.
The extension uses the platform's brokered socket, converts 16 kHz PCM to 24 kHz
inside each session, and waits for accepted audio and pending final transcripts
before completing finalize(). It declares the provider's 60-minute session
ceiling for the existing rotation mechanism. Speaker tracking remains independent
of STT; batch speaker labels do not establish identity across requests.
File uploads preserve their audio container, including Recording Studio's FLAC
windows, and are limited to 25 MB. Diarization requests use diarized_json with
chunking_strategy=auto; unsupported prompts and word-timestamp options are omitted.
Model access still depends on the configured OpenAI account. Rebuild and reinstall
the OpenAI extension to expose the new models and live-session hook.
Provider references: file transcription, realtime transcription, and session lifecycle.