Pack Sources

Pack sources let your developers push packs to infrastructure they already control — a Git repository or an OCI registry — and have your Scrydon installation pick them up automatically on a schedule. No inbound traffic, no manual uploads for every release.

How It Works

Your Scrydon installation runs a reconciler that polls each registered source on a configurable interval (default 5 minutes, minimum 30 seconds). On each tick it:

1. Fetches the scrydon.yaml catalog file from your source.
2. Compares it against the previously installed versions.
3. For each new or changed entry: fetches the artifact, verifies its signature, and installs it through the same pipeline as a manual upload.
4. Records the result — success or detailed failure reason — in the source's run history.

The reconciler is pull-only. Your CI pipeline never needs inbound access to your Scrydon cluster.

The source catalog wins

The scrydon.yaml catalog file is the source of truth for which packs appear in your organization's catalog. If an administrator retires a source-managed pack from the catalog UI, the next sync restores it as long as it is still listed in scrydon.yaml — the same way re-uploading a retired pack manually brings it back. To remove a source-managed pack permanently:

• remove its entry from the source's scrydon.yaml, or
• disable (or delete) the source itself.

Retiring is still useful for manually uploaded packs (no source manages them, so they stay retired) and as a temporary hide for source-managed packs between syncs.

Choosing a Source Type

You can register multiple sources of either type per organization.

The scrydon.yaml Catalog File

Every source is described by a single scrydon.yaml file at a configurable path in your repository or registry. This file is the source of truth for which packs should be installed and at what version.

apiVersion: scrydon.io/v1
kind: PackCatalog
packs:
  - id: example.sample
    version: 1.0.0
    artifact:
      kind: git-tar
      ref: main
      path: packs/sample
    signature:
      kind: none

  - id: example.fraud
    version: 1.2.0
    artifact:
      kind: oci
      ref: ghcr.io/example/scrydon-fraud@sha256:abc123def456...
    signature:
      kind: cosign-keyless
      certificateIdentityRegexp: "https://github.com/example/.*"
      certificateOidcIssuer: "https://token.actions.githubusercontent.com"

Pack catalog fields

Publishing from CI

Git source layout

For a Git source, your repository contains the scrydon.yaml catalog alongside your pack directories. The reconciler packs the declared subdirectory at the configured ref into a .scrydon-pack.tar.gz on the fly.

Minimal layout:

my-packs/
├── scrydon.yaml          # Pack catalog
├── packs/
│   └── fraud/            # Pack directory — each has a pack.json
│       ├── pack.json
│       └── ontology/
│           └── manifest.json
└── .github/
    └── workflows/
        └── validate.yml  # Validates scrydon.yaml on every PR

Validate scrydon.yaml on pull requests so catalog errors are caught before they reach your Scrydon installation:

# .github/workflows/validate.yml
name: validate-catalog
on:
  pull_request:
    paths: ['scrydon.yaml', 'packs/**']
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v2
      - name: Validate catalog schema
        run: |
          bunx @scrydon/sdk-authoring sources validate scrydon.yaml

OCI source layout

For an OCI source, your CI pipeline builds a .scrydon-pack.tar.gz for each pack, pushes it as an OCI artifact, signs it with cosign, and updates the scrydon.yaml catalog (also pushed as an OCI artifact).

- uses: sigstore/cosign-installer@v3
- uses: oras-project/setup-oras@v1

bunx @scrydon/sdk-authoring pack build packs/fraud \
  --out dist/fraud-${{ github.ref_name }}.tar.gz

oras push ghcr.io/${{ github.repository_owner }}/scrydon-fraud:${{ github.ref_name }} \
  --artifact-type application/vnd.scrydon.pack.v1.tar.gzip \
  dist/fraud-${{ github.ref_name }}.tar.gz:application/vnd.scrydon.pack.v1.tar.gzip

COSIGN_EXPERIMENTAL=1 cosign sign --yes \
  ghcr.io/${{ github.repository_owner }}/scrydon-fraud:${{ github.ref_name }}

DIGEST=$(oras manifest fetch --output - \
  ghcr.io/${{ github.repository_owner }}/scrydon-fraud:${{ github.ref_name }} \
  | jq -r '.config.digest // .digest')
sed -i "s|REPLACE_WITH_DIGEST|$DIGEST|" catalog/scrydon.yaml

oras push ghcr.io/${{ github.repository_owner }}/scrydon-catalog:catalog \
  --artifact-type application/vnd.scrydon.catalog.v1+yaml \
  catalog/scrydon.yaml:application/vnd.scrydon.catalog.v1+yaml

Full working examples are in the examples/pack-sources/ directory of the Scrydon SDK repository.

Registering a Source via the Settings UI

Authenticating a private source

Private Git repositories and private OCI registries need a credential. Paste it straight into the source's Access credential field — Scrydon saves it as an encrypted organization secret for you and wires up the reference automatically. You never manage a separate secret by hand.

Delivering custom integrations

A pack can include an integration content entry in its pack.json. When such a pack is installed, the platform registers the custom integration alongside any other content kinds (ontology, process-flow, etc.) — no separate upload step.

Sync only places packs in your org catalog — installation is an explicit second step. The Settings → Platform → Integrations → Add Integration marketplace is where custom integrations are discovered and first installed. The Settings → Platform → Packs → Catalog tab is where you track install state across all content kinds and apply updates.

In the marketplace, synced custom integrations appear under From your pack catalog:

• Not yet installed — shows an Add button; clicking it installs and activates the integration immediately.
• Already installed and current — shows an Up to date indicator and remains in the pack catalog group. Clicking the card opens the vendor configuration view.
• Update available — shows an Update badge. Clicking the action deep-links to Settings → Platform → Packs so you can review workflow impact and apply the update there. See Pack version management.

Installing a whole pack (including its non-integration content kinds such as ontology, workflows, and data sources) is done via the Install row action under Settings → Platform → Packs → Catalog.

Adding an integration from the marketplace activates it immediately — the explicit admin action is the review. Signature policy gates non-interactive flows:

Pack size limit for integration artifacts: 8 MB per bundle.tar.gz. Larger integrations must use the Manual upload door (50 MB). See the pack authoring guide for the full subdir layout and pack.json declaration.

Signature Policies

Every pack artifact must pass signature verification before it is installed. The signature policy is set per source at registration time.

cosign-keyless (recommended)

signature:
  kind: cosign-keyless
  certificateIdentityRegexp: "https://github.com/acme/scrydon-packs/.*"
  certificateOidcIssuer: "https://token.actions.githubusercontent.com"

The reconciler calls cosign verify with these constraints. The OIDC issuer and certificate identity must match exactly — a pack signed in a different repository or by a different CI system will fail verification.

For this to work, your publish workflow must request id-token: write permission and use sigstore/cosign-installer@v3.

cosign-key

signature:
  kind: cosign-key
  publicKey: |
    -----BEGIN PUBLIC KEY-----
    MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE...
    -----END PUBLIC KEY-----

Generate a key pair with cosign generate-key-pair and paste the public key above. Keep the private key in a CI secret (COSIGN_PRIVATE_KEY) and sign with cosign sign --key env://COSIGN_PRIVATE_KEY.

unsigned-allowed

Use only for local development or air-gapped lab environments where running a signing infrastructure is impractical. In scrydon.yaml, entries under a unsigned-allowed source use signature: { kind: none }.

Troubleshooting

verify_failed

Signature verification failed for one or more pack artifacts.

• Check that your publish workflow has id-token: write permission (for keyless).
• Confirm the certificateIdentityRegexp matches the GitHub Actions run URL for your repository (e.g. https://github.com/acme/scrydon-packs/.*).
• Confirm the certificateOidcIssuer matches the OIDC provider your CI uses (https://token.actions.githubusercontent.com for GitHub Actions).
• For cosign-key: verify the public key in the source configuration matches the private key used to sign the artifact.
• Check that the pack was signed after being pushed to the registry — signing a different tag or digest than the one referenced in scrydon.yaml will fail.

fetch_failed

The reconciler could not reach your source.

• For Git sources: verify the URL is an HTTPS clone URL and that any credentials (PAT or deploy key) have read scope on the repository.
• For OCI sources: verify the registry reference is correct and that the pull credentials have read access to the repository.
• Verify the source is reachable from your Scrydon cluster's network. The reconciler makes outbound HTTPS requests — ensure egress is not blocked.
• Click Test connection in Settings to get a more detailed error.

parse_failed

scrydon.yaml failed schema validation.

• Run the validator locally: bunx @scrydon/sdk-authoring sources validate scrydon.yaml
• Common causes: unknown keys (typos), missing required fields, invalid pack ID format (must be dotted lowercase, e.g. acme.fraud), invalid semver, artifact ref missing for OCI entries.
• Confirm the file is valid YAML before checking the schema.

install_failed

The pack was fetched and verified but failed to install.

• The pack bundle itself may be malformed. Test it locally: bunx @scrydon/sdk-authoring pack test packs/fraud
• Check the run history error detail in Settings > Platform > Packs > Sources tab — click a source row (or its View history action) to open the run-history panel — for the specific pack.json validation error.
• Confirm the pack.json package.id matches the dotted-lowercase pattern and that installOrder matches the kinds in contents.

Source shows "managed by chart" and is read-only

This source was seeded via packSources: in your Helm values. Edit the chart values to change it — do not create a duplicate UI-managed source with the same name.

Sync stops after repeated failures

After three consecutive failed runs, the reconciler backs off to double the configured interval (capped at one hour). Fix the underlying error — the backoff resets automatically on the next successful sync. Clicking Sync now in the UI resets the counter immediately.

A retired pack came back / a pack is missing from the catalog

Source-managed packs follow the source catalog: a retired pack that is still listed in scrydon.yaml is restored on the next sync (see The source catalog wins). Conversely, if a pack you expect is missing, check that it is listed in the source's scrydon.yaml, that the latest run in the source's history succeeded, and that the Sync now toast reports it as installed — then retire/restore state follows the source automatically.