Drone camera demo
An experimental ontology extension and marimo app for object detections from a ground camera watching for a DJI Neo, with platform detection, a declared camera position, a presence watch, and short clips around a sighting.
The drone-camera example extension declares an ImageDetection ontology type and two media-capable push data sources: a camera (a phone's capture page, or a laptop-side uploader) pushes one signed HTTPS multipart delivery per frame, plus short clips. Drones in those frames are found by the platform's own detector when the platform's detector is enabled by the operator (see Platform detection), or by the uploader's own detector. Download the operator notebook and the app permit template. The full-width notebook uses scrydon.apps components for a map at the declared camera position, a "Latest picture" panel, a drone-detections table with a clip player, and a critical-alerts panel.
Why a ground camera, not the drone itself
The demo targets a DJI Neo (135g): a small, phone-controlled drone (DJI Fly app). It has no HDMI out and does not push RTMP, and at 135g it may not broadcast Remote ID at all. So this demo is camera-based: a ground camera (the Mac's own webcam, or a phone's camera mirrored to the Mac) watches the room, and the uploader's detector finds the Neo in frame. The default detector is Sauron, a YOLOX-nano drone model run on the CPU; when its weights are not installed the uploader falls back to the open-vocabulary YOLO-World, told to look for ["drone", "quadcopter"], both mapped to the single reported class "drone". Detection happens in the uploader only: the platform runs no detector in this build. Indoors there is no GPS fix, so every ImageDetection carries a declared camera position (declaredLat, declaredLon, an optional declaredBearingDeg, positionQuality: "declared"), never a measured one. The map draws it with a dashed marker turned to its declared facing, so it never reads as a GPS fix.
Every pushed frame is a picture, not only a drone sighting
Besides a drone detection, the uploader pushes a plain still every --snapshot-every seconds (default 10s) and a heartbeat every --heartbeat-minutes so the source stays healthy. Both still write exactly one ImageDetection row — with every detection-specific field null rather than zero rows — so a steady picture stream shows in Horizon's "Latest picture" panel even when no drone is in view; "Drone detections" filters to the rows a drone was actually found in. Each of the two queries reads at most 500 rows and is unavailable as a whole past that: "Latest picture" goes unavailable once the table holds more than 500 frames, after about 83 minutes at the uploader's default 10 s cadence, or about 17 minutes of phone frames at one every 2 s.
A presence watch over a small declared zone
drone-in-view is a presence/signature watch, reusing mik-north-sea's weighted mission-signature mechanism (@scrydon/ontology-kernel's MISSION_ROLES) over a small declared zone around the camera. Each detection row is its own entity for the watch, and the watch alerts on the second sighting inside its window, not the first: one detection on its own can be a false alarm (a cluttered room, a reflection), so the threshold is a score of 2. The second sighting fires a critical detection at score 2; a third raises the score to 3 (capped by roleSaturation), crossing suppression.refireOnScoreDelta and firing again even inside the cooldown; a fourth plateaus at the same score. This required adding CAMERA_DRONE to the shared MISSION_ROLES enum, prefixed the same way the existing vessel roles are, since a vision-detector classification is a different domain from the airframe/vessel roles already there.
The watch window is 360 seconds, longer than the platform's five-minute watch sweep. Waking the watch when a frame arrives needs the event backbone, which is off by default in development, so the sweep may be the only evaluation; a shorter window would drop every sighting that ages out before the next sweep reads it.
Platform detection
The frames source declares the platform's own detector in its media binding: ingest.media.detector names sauron-drone-v1 from the platform's detector catalogue, with a minConfidence of 0.7. When the platform's detector is enabled by the operator, each frame the source records is also scored inside the platform, so a plain camera, such as a phone running the capture page, needs no detector of its own.
- Each box scoring at least 0.7 becomes a new
ImageDetectionrow of the frames table: itsclass,confidenceand box columns, the frame'smedia_refwith the box as its region, anddetectorset tosauron-drone-v1@sha256:<model hash>. Every other column (capture time, declared position, sensor,clip_id) is copied from the camera's own row for that frame. The camera's row is left as it is. - A frame with no box above the threshold adds no row; the camera's row still shows the picture.
- Detection rows carry the table's own label. A frame labelled above the table is refused, never written at a lower label.
- The model runs in an isolated sandbox with no network access, and the platform checks it against its pinned hash before it serves a frame. Detection only adds rows: what the watch does with them is raise an alert to an operator.
Until the platform's detector is enabled by the operator, a frame lands exactly as before, as the camera's row only, and declaring the detector changes nothing.
Why 0.7, not the catalogue's 0.5. This extension is a demo watched indoors, and the model scores an empty room at 0.57, so the catalogue's default of 0.5 would write a drone row for an empty frame. At 0.7 the empty room is dropped; recall is about 0.63, so hold the drone in clear view of the camera. A source watching open sky could sit nearer the default.
One detector at a time. When the platform's detector is enabled for this source, run the uploader with --detector none. With both on, one sighting is written twice, once by each, and the two rows alone would reach the watch's threshold.
The push sources and their media bindings
manifest.ts declares drone-camera-neo-v1 (frames) and drone-camera-neo-clips-v1 (clips), both with their media binding turned on. Each frame delivery carries an image and a JSON metadata part with one row per detection — or one null-fielded row for a plain picture. A clip is a short H.264 MP4 pushed as a separate media item (its own DroneClip row), only once the installed clips source's declared media binding names video/mp4 — the uploader never guesses. A camera stamps every frame it takes while a clip records with that clip's clipId, and DroneClipOfDetection joins a detection to its clip by it. A platform detection copies clip_id from its frame's row, so it links to the clip that covers it without the camera knowing the detection's id. A frame taken while no clip records links to none.
Signing matches the platform's generic inbound HMAC contract exactly: x-signature is sha256=<hex> over <x-timestamp>.<x-delivery-id>.<raw body>, verified against the same byte construction the platform's push ingress checks.
The uploader
uploader/upload.py is a small, uv-runnable Python script with no project setup: its dependencies (requests, pillow, numpy, onnxruntime, ultralytics and its CLIP text encoder) are pinned in its own script header, and each detector's weights are pinned by SHA-256; --camera uses OpenCV and clip encoding needs ffmpeg with libx264, both reached lazily. Three mutually exclusive frame sources — --watch-folder for copied photos, --screen-region x,y,w,h for a mirrored view, or --camera N|NAME for a local camera through OpenCV. --min-confidence drops anything below threshold before it is ever sent; it defaults to the chosen detector's own operating point (0.5 for Sauron, 0.03 for YOLO-World). ultralytics, which YOLO-World needs, is AGPL-3.0; Sauron's model and runtime are Apache-2.0 and MIT. The push secret is read from an environment variable and never logged. --dry-run builds every request exactly as it would be sent and writes it to a file instead of calling the network.
Use your iPhone as the camera
--list-cameras prints each camera's index and name; on macOS these come from AVFoundation itself (ffmpeg -f avfoundation -list_devices true -i "", or system_profiler SPCameraDataType -json as a fallback), under the assumption that AVFoundation, OpenCV's cv2.CAP_AVFOUNDATION backend, and that listing all enumerate the same camera list in the same order. --camera then accepts either a bare index or a case-insensitive name substring (e.g. --camera iPhone), resolved against that same listing, with a hard error listing the available cameras when a name matches zero or several of them.
This is what lets an iPhone stand in for a dedicated webcam through macOS Continuity Camera, with no app or cable: once the iPhone and the Mac share the same Apple ID, both have Wi-Fi and Bluetooth on, the iPhone is locked, in landscape orientation and near the Mac, and both run a reasonably recent macOS/iOS, the iPhone shows up as an ordinary AVFoundation camera and
uv run upload.py --camera iPhone --endpoint <push-source-url> --declared-lat 48.8566 --declared-lon 2.3522opens it directly, at 1920x1080 when the camera offers it. A camera-access denial (rather than a wrong name) is the usual reason the camera fails to open; the error names the fix (System Settings → Privacy & Security → Camera).
Publishing the app
The published app must pin the exact installed ontology revision and content hash. createDroneCameraAppPermit(revisionId, contentHash) validates that permit before it is entered in Analytics. Copy app.py into a marimo notebook, save it, and use the Analytics publication panel only after the drone-camera-neo-v1 source has been installed. See app release manifests and marimo app components.
Phone camera (no laptop)
Instead of the laptop-side uploader, an org admin can turn a phone into the camera directly: open the drone-camera-neo-v1 push source in Analytics, open its Settings and choose its Phone camera section, and enter the camera's position (latitude and longitude are required). Analytics shows a QR code holding a one-time link, valid for 10 minutes. Scanning it opens a capture page, served by Analytics with its own headers, that asks for the camera and pushes frames through the same signed multipart delivery the uploader uses. A short-lived, revocable session key stands in for the source's real secret, so the phone never holds it. The link is built on the deployment's public Analytics URL (PUBLIC_ANALYTICS_APP_URL). The session key is derived from the source's API key, so the push source's connection must hold one; a source connected another way is refused at issue.
The phone is a plain camera: it runs no detector, and the platform runs none either in this build. It sends every frame it takes, one every 2 s (the frames binding allows 30 a minute), whatever is in view. Each frame lands as an ImageDetection row with a null class, no box and a null detector, so a phone-only setup fills "Latest picture" but never "Drone detections", and the drone-in-view watch never fires for it. Drone detections come only from the laptop-side uploader. With no detection to trigger a clip, the page records a fixed 8 s clip every 60 s while it runs, each with a fresh clipId and no detectionId.
The session expires on its own (2 h by default, 8 h at most). Stop on the phone turns the camera off and keeps the session open, so Start camera resumes it. End session on the phone, leaving the capture page, and Revoke in Analytics each end it at once.
The phone camera needs the default subpath routing mode, where the capture page reaches its API on its own origin. In subdomain mode the section is not shown; see routing modes.
What the example's tests cover
The manifest tests validate both push sources (media bindings, the frames source's detector against its table, nullable detection columns, clip_id, the clips source's video/mp4 allow-list), the ontology bindings, the link joins, and the permit fixture. watch.test.ts evaluates drone-in-view against the real kernel planner (planWatchDetection), checks it against the manifest's own watch, and proves the no-alert/fire/refire/plateau sequence described above. The uploader's tests check the HMAC signing byte-for-byte, the metadata JSON shape for both push sources, the open-vocabulary class mapping and confidence filtering, the snapshot/heartbeat cadence, the rolling frame buffer and clip assembly, and an end-to-end run with a fake camera, detector and clip encoder, including a dry run that never imports the network client. No network call, no model download and no camera runs in any of this; the real ffmpeg encoder runs once when it is on PATH.
Safety note: fly the Neo only in a clear, open room away from people and fragile objects, or simply hold it up with its motors off, aimed at the camera — the demo only needs the Neo visible in frame, never airborne.
What this demo does not prove: platform detection on a live stack (it is tested piece by piece, not yet run end to end), real DJI Neo hardware and a real camera (webcam or mirrored phone), the YoloWorldDetector's real detections, a delivered clip actually being accepted and played, production ingest rates, or behavior at production volume. See apps/docs/examples/drone-camera/README.md for the full runbook.