Air-Gapped Deployment

Use this guide if your Kubernetes cluster has no outbound internet access — for example, classified government networks, isolated industrial environments, or strict compliance environments. If your cluster can reach the internet, see Helm instead.

What is Zarf?

Zarf is an open-source tool (Apache 2.0) purpose-built for delivering Kubernetes applications into air-gapped and disconnected environments. Scrydon uses Zarf to bundle all container images, Helm charts, and manifests into a single self-contained .tar.zst archive that can be transferred physically (USB, DVD, cross-domain guard) or over a one-way data diode.

You do not need Harbor, Artifactory, or any external registry for the artifacts included in the package.

The package includes the pinned valkey:8.1-alpine image used by the default-on durable event backbone. No public Docker Hub pull is required when the chart starts its managed broker.

It also includes all six Dapr delivery images: five control-plane components and the daprd runtime sidecar injected into application pods.

Prerequisites

• Kubernetes 1.28+ cluster (air-gapped)
• zarf CLI installed — a single static binary, available at zarf.dev/releases or delivered alongside your package
• License bundle ({ jwt, publicKey } JSON file) received via secure channel — see Licensing. Bare .jwt files are no longer accepted; the bundle ships the matching RSA public key so the platform can verify signatures without a network call.

Runtime isolation is packaged, but not automatically accredited. The stock package includes runtime-plane, its workload images, and managed-execution-actor-host. Its air-gap overlay intentionally keeps runtimePlane.enabled: false and the default executionFabric.allowedTiers: [microvm] until the target proves a schedulable Kata/KVM RuntimeClass, internal registry/DNS reachability, workload admission, a distinct guest kernel, governed egress, and teardown using the imported release. Packaged images do not by themselves prove a microVM.

Step 1: Receive the Package

Scrydon ships air-gapped packages as a signed .tar.zst archive, published to an OCI registry:

# Pull the Zarf package from the OCI registry (on a connected machine)
zarf package pull oci://ghcr.io/scrydon/scrydon:<version>

# Output: zarf-package-scrydon-amd64-<version>.tar.zst

Transfer this file to your air-gapped environment via approved means (encrypted USB, DVD, cross-domain guard, one-way data diode). The package is a standard tar archive and is compatible with most cross-domain guard appliances.

Step 2: Verify the Package

Before deploying, verify the package's integrity and authenticity.

Verify the Cosign Signature

zarf package inspect zarf-package-scrydon-amd64-<version>.tar.zst \
  --key scrydon-cosign-public-key.pem

The Scrydon public key (scrydon-cosign-public-key.pem) is delivered alongside the package. Contact your Scrydon account representative if you need a fresh copy.

Verify the SHA-256 Checksum

sha256sum -c SHA256SUMS

The SHA256SUMS file is included in the delivery bundle. Both checks must pass before deploying.

Inspect the SBOM

zarf package inspect zarf-package-scrydon-amd64-<version>.tar.zst --sbom

This opens the built-in SBOM viewer, listing all software components and their licenses included in the package.

Step 3: Deploy the Package

zarf package deploy zarf-package-scrydon-amd64-<version>.tar.zst --confirm

Zarf will:

1. Create an in-cluster OCI registry (if one is not already present)
2. Load all container images from the archive into the in-cluster registry
3. Deploy a mutating webhook that rewrites image references to point to the in-cluster registry
4. Run helm install with the Helm chart and values bundled in the package
5. Wait for all deployments to become ready

The chart creates one events-bootstrap Job to ensure and migrate the events database. Api-platform waits on the resulting schema before becoming ready. The managed Valkey password is generated on a live first install and reused from its Kubernetes Secret on upgrades.

To pass custom configuration during deployment:

zarf package deploy zarf-package-scrydon-amd64-<version>.tar.zst \
  --confirm \
  --set DOMAIN=scrydon.yourdomain.mil

Step 4: Apply the License Bundle

After deployment, apply your license bundle. The bundle is a JSON file delivered separately via secure channel ({ "jwt": "…", "publicKey": "-----BEGIN PUBLIC KEY-----…" }) and is NOT included in the Zarf package. There is no scrydon-license Kubernetes Secret — the bundle is stored in the platform's platform_config DB row after first use, and rotated through the platform UI.

You have two ways to apply the bundle on a fresh install:

Option A — /setup wizard (default). Open https://app.yourdomain.com/platform/setup (or /setup if you mounted platform at the root) in a browser and paste/drop the bundle into Step 1. The wizard parses the bundle, verifies the JWT against the public key in the bundle, and persists both on Step 2.

Option B — pre-seed via Helm values. Set auth.secrets.LICENSE (the bundle's jwt) and auth.secrets.LICENSE_PUBLIC_KEY (the bundle's publicKey) when deploying the Zarf package, then go straight to the Admin step in /setup:

zarf package deploy zarf-package-scrydon-amd64-<version>.tar.zst \
  --confirm \
  --set LICENSE_JWT="$(jq -r .jwt license.bundle.json)" \
  --set LICENSE_PUBLIC_KEY="$(jq -r .publicKey license.bundle.json)"

Whichever path you pick, the license is persisted to the DB on first read and the values entries are no longer consulted afterwards.

Step 5: Verify the Deployment

Check Pod Status

The default chart layout puts every service in scrydon-platform. Multi-namespace installs are opt-in via namespaces.* overrides.

kubectl get pods -n scrydon-platform
# If you split namespaces via namespaces.agentic / namespaces.analytics,
# also check those namespaces.

All pods should reach Running 2/2 (app container + Dapr sidecar). If pods stay at 1/1, Dapr injection failed — check kubectl get pod -l app=dapr-sidecar-injector -A.

If a pod is stuck in Init:0/1, the wait-for-db init container hasn't completed — confirm the bundled Postgres (kubectl get pod -l app=db -n scrydon-platform) is Running.

If api-platform reports wait-for-events-schema, inspect kubectl get job -A -l app=events-bootstrap and verify the external events database/Secret when using BYO PostgreSQL.

Check Ingress

kubectl get ingress -n scrydon-platform

All options

The Zarf package wraps the same Helm chart. Any chart value can be set at deploy time with --set <KEY>=<VALUE> (as shown for DOMAIN and the license above), or baked into the bundle's values overlay. A value that enables an image-backed component works only when that image is present in the package. Every value the chart exposes — routing, BYO database, StarRocks/SeaweedFS, scheduling, low-resource trims — is documented in the Helm reference. Air-gapped installs additionally set license.mode: offline so license validation never phones home. Keep the packaged runtime-plane disabled until the target-specific proof above is complete; then enable it with the imported image references and the proven RuntimeClass settings.

Alternative: your own private registry (without Zarf)

If you already run an internal registry (Harbor, Artifactory, a customer ACR) you can skip Zarf and install the chart directly. Import every image the chart pulls, then set both registry flags:

helm install scrydon ./helm/scrydon \
  -f helm/scrydon/values.yaml \
  -f helm/scrydon/values.airgapped.yaml \
  --set global.imageRegistry=harbor.internal \
  --set dapr.global.registry=harbor.internal/dapr

Both are required. global.imageRegistry covers every chart-managed image; dapr.global.registry covers five Dapr control-plane images plus the injected daprd runtime sidecar. Those six images come from the Dapr subchart and are unreachable from the parent's global (Helm subchart values are static YAML). Missing the second flag is not a partial failure — the injector either never becomes Ready or application pods cannot start their sidecar. helm install prints a warning when only one is set.

The images to import are listed in the header of values.airgapped.yaml. To produce the exact list for the version you are deploying:

helm template scrydon ./helm/scrydon \
  -f helm/scrydon/values.yaml -f helm/scrydon/values.airgapped.yaml \
  | grep -oE 'image: .*' | sed 's/image: //' | tr -d '"' | sort -u

Individual third-party images can be pulled from a different scope with the per-image registry key (infra.starrocks.image.registry, opa.image.registry, …) when your mirror does not carry them all under one path.

GitOps and offline manifest rendering

Anything that renders the chart with helm template instead of installing it live — ArgoCD's default renderer, Kustomize's Helm inflator, or a helm template | kubectl apply pipeline across the air gap — must set global.requireExplicitSecrets: true and supply the secret set from values.gitops.example.yaml.

Helm's lookup returns empty without a live API connection, so every chart-managed secret would be regenerated on each render — 38 values, including the Dapr crypto master key (all secrets at rest become undecryptable), the Lakekeeper encryption key, AUTH_SECRET, and the object-storage keys. With the flag set, a missing value is a render-time failure naming the values path instead of a silent credential rotation. Full detail: Rendering without a live cluster.

A direct live Helm install, zarf package deploy, and Flux's HelmRelease all run a real helm upgrade --install and need none of this.

Security Considerations

Air-gapped environments have additional security requirements:

• Offline licensing: License validation makes no outbound call. The license JWT is validated locally using the delivered public key. Other features use only destinations permitted by the air-gapped network and organization egress policy.
• No telemetry: In air-gapped deployments, no telemetry or analytics data leaves the cluster.
• Image integrity: Verify the signed package and SHA256SUMS, record the imported image digests, and use those exact artifacts for runtime-plane accreditation. Zarf rewrites packaged references to the in-cluster registry; rewriting is not a substitute for retaining digest evidence.
• Secrets: The license bundle is persisted in Scrydon's platform database after first use; deployment credentials use the platform's normal in-cluster secret stores. Zarf does not transmit these values out of the disconnected environment.

Next: See Licensing for details on license validation, grace periods, and renewal.