Connector runtime

Scrydon's connector runtime is one service that runs every data source's inbound acquisition (poll, push, stream, and a MAVLink telemetry link) and, where an organization has opted in, a source's declared outbound actions back toward the same provider or vehicle. It composes existing kernels — the poll scheduler, stream supervision, the push runtime, the durable ingest buffer, and (for MAVLink) a held UDP link — and adds nothing on top of them that duplicates what they already do.

This page is the map for two audiences: an operator deciding what to enable and how to read what happened, and an extension author deciding which ingest mode and which declared actions a data source should carry. For the field-level authoring reference, see Ingest modes and Data Sources.

Inbound: how data reaches a table

Every data source declares exactly one ingest.mode. Four of them run continuously on the connector runtime:

manual, on-demand and query sources are not continuous. A person triggers a manual upload or an on-demand fetch: the runtime parses and maps that one run and returns its rows, and the platform commits them. A query source does not run on the connector runtime at all; it reads a remote database through its governed connection and owns no managed table.

A MAVLink link is a stream source with transport: 'udp'. The runtime binds one UDP socket, sends outward to the peer to open the return path through NAT, and decodes HEARTBEAT, SYS_STATUS, SYSTEM_TIME and GLOBAL_POSITION_INT into a vehicle telemetry table — receive only, unless the source also declares an outbound action (below) and the link is signed: an unsigned link never sends a command. A link is signed (signing: 'signed-required') with the connection's linkKey credential, or left unsigned-allowed; a replayed or unsigned frame on a signed link is refused and never becomes a row. See A MAVLink link over UDP for the full manifest and the operator's private-network declaration it depends on.

Live positions and the committed record

The rows the table and the ontology read are committed about once a minute per source (STREAM_COMMIT_INTERVAL_MS), or sooner when a batch fills one commit. That commit is the record: it is what a governed read of the table sees and what an operator's "last sync" reflects. Before it, a WebSocket stream appends its frames to the durable ingest buffer within 250 ms of arrival (STREAM_BATCH_FLUSH_MS), so a crash between commits loses as little as possible. A MAVLink link keeps no such journal: what a link could not commit is lost with it, and its health window says so.

A source that also declares trackProjection sends positions to a governed ontology correlated_tracks stream — this is how a vehicle or AIS position reaches Horizon's map. Each commit sends its rows' positions. Between commits, a best-effort live tick at the same 250 ms cadence sends the newest position per entity, after the organization's data policy has scanned it; a tick is never written to the table and never counts as a sync. A row with no position is skipped, never failed. A row that carries its own markings is never projected: a track stream cannot enforce per-reader markings, so a marked row stays in the table, visible only to readers who hold its markings. See Data Sources for the trackProjection field reference.

Source health

A source reports one status a person reads — Healthy, Failed, Paused or Running — plus mode-specific detail: a stream's stream_disconnected / stream_silent / stream_set_aside, a poll's last_error and consecutive_failures, a push source's delivery warnings. The health check and digest daily alerts both read this same state; nothing bypasses it.

Markings, clearance and loss prevention on the way in

Every table a source writes has a markings column (an array of marking slugs) that every governed read filters on. The platform adds it when the table does not declare it, and a table that declares it must declare it as an array; a source cannot opt out of it. Before any row is committed, it passes the organization's data policy: a row the policy rewrites is written rewritten, a row it blocks is set aside and counted, and the rows beside it are written as they came. A row too large for one scan is set aside unscanned rather than let through unchecked. A scan that cannot answer blocks the commit entirely — rows are never written unscanned and never dropped silently. See Classification and clearance for the no-read-up rule that then gates who can read a classified or marked row back out.

Outbound: declared actions

A data source carries no sending capability by default. The only way one gets a write, invoke or publish capability is a declared action in its manifest — a closed input schema, units, preconditions, a classification, an expiry, and an explicit requiresApproval. There is no enabled field on the declaration itself: enablement is a separate, audited decision.

1. Enable. An organization administrator enables one declared action's exact definition (action.catalog.enable, admin-only). A request against a definition that is not enabled is refused before any outbound row is written.
2. Request. A caller's request reaches the catalog through API Platform's action.tasking.dispatch operation only; nothing else can queue a unit.
3. Approve (if required). A declared action with requiresApproval: true holds every request at the same exact-request approval gate every other kernel action uses — the approver may never be the kernel's verified requester. Once approved, the requester submits the same request again, and that submission consumes the approval: one approval admits one request. Approval and enablement are independent: both gates must open.
4. Outbox. Only once enablement and (if required) approval both admit the request does the ontology kernel write an outbox row (action_outbound_request) and queue exactly one unit.
5. Signed authority, bound to the lease. The kernel mints a short-lived (≤ 5 minutes), asymmetrically signed grant — the action's identity, target, destination, the control-lease epoch, and the approval evidence. Any holder of the public key can verify it; nothing it carries lets a verifier mint one. The dispatcher re-checks the grant against the link's current fence immediately before sending, and again on every retry, so a lease lost between issuance and the write sends nothing.
6. Send. For a link-mode destination, the dispatcher encodes and signs a MAVLink COMMAND_LONG and sends it over the exact link the grant named — and only if that command is on the allow-list (today, only MAV_CMD_SET_MESSAGE_INTERVAL). Every other command is refused before it is ever encoded.
7. Outcome. A COMMAND_ACK is correlated back to the action; a missing acknowledgement gets bounded, protocol-aware retries under the same grant. When retries are spent with no ack, the action settles as Outcome unknown — never silently Failed — and no later action on that same target is claimed for sending until the row is reconciled.

Related

• Ingest modes — the full per-mode authoring reference, including the MAVLink link manifest.
• Data Sources — declarative data sources and track projections.
• Operator runbook — outbound actions — enabling, approving, reading history, and reconciling.
• Classification and clearance — the no-read-up rule that gates what a reader sees on a classified or marked row.
• Horizon — the operator picture that consumes the runtime's committed rows and track stream.