Drone video stream demo
A synthetic MPEG-2 TS video stream, with real MISB ST 0601/0102 KLV telemetry, pushed one HLS segment at a time to a video/mp2t media data source, indexed for the M3U8 playlist route.
The drone-video-stream-v1 example data source (apps/docs/examples/drone-video/manifest.ts) is a sibling of drone-camera: that example pushes still frames and whole recorded clips; this one pushes an HLS-over-HTTP video stream, one MPEG-2 TS segment per push, each carrying real MISB ST 0601 KLV telemetry. Every byte is synthetic and built in-process by the uploader (uploader/push.ts) — never a third-party sample file of unknown licence, and no real sensor or drone involved.
What gets pushed, and what lands in the table
Each push is one TS segment (video/mp2t) plus a JSON metadata part naming the segment's stream, sequence, start, end and duration — the same segment field the push runtime reads independently of any row mapping (parseSegmentMetadata). The segment's own bytes are indexed into data_source_media_segment the moment the item itself is recorded, and every KLV packet decoded from the segment commits as its own row, under the ingest runtime's fixed KLV row vocabulary — capturedAt, sensorLat/sensorLon/sensorAltM, platformHeadingDeg, sensorAzimuthDeg/sensorElevationDeg/sensorRollDeg, frameCenterLat/frameCenterLon/frameCenterElevationM, footprint, unknownTags — plus mediaRef, the column every row carries pointing back at its own segment. The table declares exactly those columns, camelCase, because those rows bypass ingest.mapping entirely: it is empty here, since this source writes no row any other way.
A packet can carry MISB tags this decoder has no named mapping for; rather than guess at or silently drop them, every such tag rides along verbatim in unknownTags, one JSON array of {tag, data} per row, data base64-encoded — the same byte-crossing-JSON convention a stream's own set-aside messages already use. unknownTags is absent (not an empty array) on a row whose packet carried no unmapped tag.
The uploader's own generator (klvPacketBytes, from @scrydon/common's testing entrypoint, so this example reuses the exact fixture the ingest runtime's own tests already prove decodes correctly) only exercises a packet's timestamp and, optionally, its ST 0102 security tag — a decoded packet's sensor/position/angle/frame-center tags are decodeKlvPacket's own proven surface, not re-proven here, so every row this demo writes has those columns null. Every column but mediaRef and the primary key (capturedAt) is nullable for exactly that reason, the same way a heartbeat frame in drone-camera writes a detection row with every detection-specific field null rather than zero rows.
The ST 0102 → platform label mapping
A TS segment's decoded ST 0102 security set can carry a classification claim that only ever raises the label the segment (and any admitted KLV rows) commit under — never lowers it, and never below the source's own configured label. ST 0102's five-level enumeration maps onto the platform's own four-level scale:
| ST 0102 classification | Platform label |
|---|---|
unclassified | public |
restricted | confidential |
confidential | confidential |
secret | restricted |
top_secret | restricted |
restricted floors at confidential, not internal: every classified KLV tier lands at confidential or higher, so a claim is never mistaken for an unclassified one — raise-only, and the mapping errs toward the higher label whenever a tier has no exact platform equivalent (secret and top_secret both floor at restricted, the platform's own ceiling, since there is no tier above it to distinguish them with).
A security set a packet asserts at all is never read as no claim, even when its own classification byte cannot be mapped (missing, too short, or unrecognized): that raises to restricted, the same ceiling, fail-closed rather than silently waved through — counted (KLV_WITHHELD_REASON.SECURITY_UNRECOGNIZED) every time, whether or not the raise itself goes on to withhold any rows.
Why a raised claim can withhold KLV rows, never the video
When a segment's claim would raise its label above the installed source's own configured label, Platform's answer to the segment record withholds only that segment's KLV rows from the table — never the segment's bytes, which are kept (and served, and played) at the raised label regardless. A table holds one label for every row it has; a row whose own segment claims a higher label than the table's cannot land there without under-labelling it, so it is counted (KLV_WITHHELD_REASON.LABEL_RAISED) and left out, while the video itself is still exactly as available as any other segment at its new, raised label.
The 1000-row sample and the bad-packet counter
A segment's decoded KLV packets are never dropped from the FRONT when there are more of them than the commit batch has room for (SOURCE_BATCH_LIMITS.MAX_ROWS, 1000): they are sampled evenly across the whole segment, first row kept, so a track joined to the segment still sees its whole span, at a lower rate, rather than only its opening seconds. The rest are counted (KLV_WITHHELD_REASON.BATCH_LIMIT), never silently discarded. A packet that fails its own CRC-16 checksum is quarantined the same way — counted (KLV_WITHHELD_REASON.PACKET_UNREADABLE) and skipped — while the rest of the segment's packets, and the segment itself, are unaffected: a bad packet costs one row, never the clip.
Playing it back
Once drone-video-stream-v1 is installed and the uploader has pushed a few segments for a stream (cam-1, cam-2, …), the stream's window is served as an HLS media playlist:
- The M3U8 playlist route —
GET /api/data-sources/media-segments/:sourceId/:streamId/playlist.m3u8?from=…&to=…(a window of at most two hours) builds a standard HLS media playlist from the segments the signed-in reader may see. Each segment's URI is the governed per-item read route, so every segment's bytes are checked against the reader's clearance and audited when a player fetches them. Segments left out for the reader are only counted, in thex-media-hiddenresponse header, never described. TS segments only in v1 (an fMP4 window answers409 segment_format_not_playable— no init segment to play one with yet). - The hls.js player — a source's Recent media panel shows one player for each stream its recent TS segments belong to (the vendored, SRI-pinned
hls.jsbuild wherever the browser has Media Source Extensions, and native HLS only where it has none, as on iOS Safari). Each player opens on the ten minutes before that stream's newest listed segment, so a stream that stopped earlier still plays its last minutes. Images and recorded clips stay thumbnails.
The uploader's synthetic segments carry only the KLV metadata stream, no picture: the playlist and every segment read work end to end, but a player has no video track to render from them.
Running it
bun test --isolate apps/docs/examples/drone-video/manifest.test.ts
bun test --isolate apps/docs/examples/drone-video/uploader/push.test.ts# No network at all -- prints every segment's metadata and byte count.
cd apps/docs/examples/drone-video/uploader
bun run push.ts --dry-run --segments 2
# Against a real installed source:
DRONE_VIDEO_PUSH_SECRET=<the source's push secret> \
bun run push.ts --endpoint <push-source-url> --segments 5The manifest test validates the push source (the video/mp2t media binding, the item-rate sizing, the KLV row vocabulary's exact column names, the empty mapping). The uploader's test checks that the signed delivery verifies under the push runtime's own HMAC check, over the exact multipart body it sends, that two streams' segments never share a capturedAt (the table's own primary key depends on it), that sequence numbers never leave a gap within one uploader run, and that a run's segments end at the run's own clock (a sequence number is the segment's 6 s slot since the Unix epoch, so a second run continues the same streams).
Live from a DJI drone (phone screen + RTMP)
A DJI Neo has no direct downlink this demo can speak to, so the lowest-friction way to get real drone video in is the pilot's own phone: screen-record the DJI Fly app and restream that recording as RTMP, over the local Wi-Fi only, into a small relay (relay/) that turns it into the exact same signed HTTPS segment pushes uploader/push.ts already sends. Nothing here is a second ingest path — it is the same push contract, fed from a real recording instead of the synthetic generator.
What runs where:
- Phone: any RTMP-capable streaming app, pointed at this Mac's LAN address — screen-record (or mirror) DJI Fly's own flight view. Hide DJI Fly's on-screen telemetry/controls overlay in its own settings where the app offers it; the relay also crops it (see below) as a second line of defense.
- Mac/site box: MediaMTX (MIT-licensed;
brew install mediamtx, designed here againstv1.21.2) receives the RTMP stream, crops the DJI Fly overlay out withffmpeg, records the cropped feed as 6 s MPEG-TS segments, and pushes each finished segment throughuploader/push.ts's--filemode — the same HMAC-signed multipart delivery as every other push.ts caller. - Sauron sampling:
relay/stream_frame_source.pygrabs one JPEG frame every--interval-secondsfrom the relay's clean RTSP output (rtsp://127.0.0.1:8554/neo_cleanby default) for the drone-camera uploader's detector path. That wiring (a--stream-urlcapture source next to--watch-folder/--screen-region/--camera) belongs indrone-camera'supload.py, which lives on a different branch than this relay —stream_frame_source.pyis written standalone, with the exact samecapture() -> bytesseam, so it drops in without a second detection push path once that branch and this one meet.
RTMP never leaves the local network (issue #5842): MediaMTX's RTMP listener binds exactly this machine's own detected LAN IP (never a bare 0.0.0.0/:1935, which would also answer on a VPN/public interface this box happens to have) so a phone on the same Wi-Fi can reach it — nothing here opens an inbound port to the internet, and the only thing the relay ever sends off-box is the signed HTTPS segment push, exactly like the synthetic uploader.
Only one credential may publish to it. MediaMTX's own publish auth (authInternalUsers) is on: start.sh mints a fresh, random credential every run and prints it once alongside the rtmp:// URL — the same "handed to the operator once, never stored" shape the phone-camera capture page's own session key already uses. The phone's streaming app must supply that credential as RTMP URL query parameters (?user=…&pass=… — MediaMTX's own form for this, not a user:pass@host URL) to publish; anything else reaching the bound RTMP port is refused. Reads (both RTSP paths — the raw feed the republish step pulls back, and the clean feed Sauron samples from) are permitted only from 127.0.0.1/::1, never off this machine — verified against the real mediamtx v1.21.2 binary (see What this does not prove below for what that check did and did not cover).
Every other MediaMTX listener this relay never uses is turned off. MediaMTX's own defaults also start an HLS server, a WebRTC server, SRT and MoQ listeners, and RTP/RTCP UDP ports — every one of them bound to every interface on this machine, not just the LAN-only RTMP bind above, so a credentialed publish over one of them could reach this relay from off the Wi-Fi entirely, and an unauthenticated one could still collide with a port already in use. The rendered config turns all of them off (hls: false, webrtc: false, srt: false, moq: false, rtspTransports: [tcp]) — this relay speaks RTMP in and RTSP/TCP out, nothing else.
Running the relay:
bun test --isolate apps/docs/examples/drone-video/relay/config.test.ts
bun test --isolate apps/docs/examples/drone-video/relay/render-config.test.ts
bun test --isolate apps/docs/examples/drone-video/relay/backlog.test.ts
bun test --isolate apps/docs/examples/drone-video/relay/loss-counters.test.ts
python3 -m pytest apps/docs/examples/drone-video/relay/stream_frame_source_test.pyRELAY_PUSH_ENDPOINT=<push-source-url> \
DRONE_VIDEO_PUSH_SECRET=<the source's push secret> \
apps/docs/examples/drone-video/relay/start.shstart.sh checks for mediamtx/ffmpeg/bun/openssl on PATH, mints the session's RTMP publish credential (RELAY_RTMP_PUBLISH_USER/RELAY_RTMP_PUBLISH_PASS, generated unless the operator pinned their own), renders the MediaMTX config (relay/config.ts), prints the exact rtmp://<this-Mac's-LAN-IP>:1935/neo URL and that credential to type into the phone's app, and execs mediamtx — a Ctrl-C or kill on that process goes straight to MediaMTX, which owns and reaps its own runOnReady ffmpeg child, so stopping it never leaves an orphaned process behind.
Every setting is overridable through environment variables read by relay/render-config.ts (crop margins — RELAY_CROP_TOP_PX/RELAY_CROP_BOTTOM_PX/RELAY_CROP_LEFT_PX/RELAY_CROP_RIGHT_PX, default 120/160/0/0, a DJI Fly demo default and not a measurement of any one phone or orientation — ports, the RTMP/clean path names, the stream id, segment length, push retries, and the backlog cap below); see relay/config.ts's RelayOptions for the full list.
A bounded backlog, never an unbounded one. Each finished segment is kept on disk until its own push succeeds (uploader/push.ts's existing retry/backoff, now with a 10 s timeout per attempt so a hung request cannot block indefinitely); nothing previously capped how many kept segments could pile up during a sustained push outage. push.ts now enforces a bound on its own recording directory before every push — RELAY_MAX_BACKLOG_FILES (default 50, five minutes at the 6 s cadence) and RELAY_MAX_BACKLOG_BYTES (default 200 MiB) — evicting the oldest kept segments first, never the one about to be pushed and never its own bookkeeping files (the counters file, a .failed-attempt marker). A kept segment is not only ever evicted: each new segment's own push also drains (re-pushes) the single oldest OTHER kept segment first, so a transient outage recovers on its own without the operator intervening. Every eviction and every push that exhausts its retries is tallied in a small per-directory counters file (.relay-loss-counters.json) and logged loudly on stderr — each segment counted exactly once, whether it is eventually recovered, evicted, or both attempted and lost, never twice for the same segment.
What this backlog cap does not yet do: the running loss counts are local to the relay's own directory — they show up in its own logs, not (yet) in the installed source's health signals in Analytics. Surfacing them there would mean a source-level, not per-stream, count (the existing windowFacts/dataSourceSignals mechanism this PR's own review considered has no per-stream dimension, and is built around a lease-held connection a one-shot segment push never has) — left for a follow-up design, not silently done partway.
Latency and limits: end-to-end latency is roughly the 6 s segment length plus however long the push and MediaMTX's own buffering add — seconds, not the sub-second latency of a direct platform ingest; this is a phone screen-recording a controller app's own video preview, re-encoded twice (DJI Fly's own render, then the relay's crop/transcode), so expect a visible quality and compression-artefact hit versus the drone's native downlink. The overlay crop is a fixed pixel margin, not overlay-aware: a DJI Fly layout change, a different phone aspect ratio, or rotating the phone can put the HUD back in frame and needs a crop-margin adjustment.
What this does prove, live: the rendered config, run against the real mediamtx v1.21.2 binary on loopback-only alternate ports — a credentialed publish (ffmpeg, over RTMP) is accepted and a segment is recorded through the republish/crop step; a publish with no credential, and one with the wrong password, are both refused; only the two listeners this relay uses (RTMP, RTSP) ever bind a port.
What this still does not prove: no live RTMP stream from an actual phone, no DJI Fly screen-record, and start.sh itself were exercised (the check above ran the rendered config directly, not the script); the crop math, per-segment push/retry/drain/backlog behaviour, and the frame sampler remain unit-tested against fakes (no real ffmpeg for those, no real RTSP source); a credentialed publish over a protocol this relay turns off (SRT/WebRTC/MoQ) reaching from off the LAN was reasoned about (the listeners never start at all) but not itself attempted.
What this demo does not prove: a real drone's video downlink or a real MISB KLV encoder, a player rendering video (the synthetic segments carry no picture), or production ingest rates and volume.