BYO Database

Scope: replacing the in-cluster Postgres deployed by infra.db with a customer-managed instance. Covers AWS RDS, GCP Cloud SQL, Azure Database for PostgreSQL (Flexible Server), Supabase, Neon, CloudNativePG, the Zalando Postgres Operator, or any other Postgres ≥ 15 with pgvector.

Related: Backup & Restore (the bundled backup CronJob is scoped to the in-cluster DB only — managed-DB users should rely on the provider's native PITR).

Why

The bundled infra.db is a single-pod Postgres backed by a PVC — fine for proofs of concept and single-node clusters, but it lacks point-in-time recovery, synchronous replication, automated failover, and regional snapshot replication. Production customers and most compliance audits (SOC 2, ISO 27001 A.8.13, SecNumCloud 3.2, NIST CP-9/CP-10) expect a managed service or a dedicated DBA-operated cluster.

Prerequisites

Pre-create databases and extension

Run once, from any client that can reach your managed DB. The default chart enables every component, which maps to five databases; drop the lines for any service you've disabled in your values file.

psql "$ADMIN_URL" <<'SQL'
CREATE DATABASE auth;
CREATE DATABASE agentic;
CREATE DATABASE analytics;   -- only if analytics.enabled (default true)
CREATE DATABASE cortex;      -- only if cortex.enabled (default true)
CREATE DATABASE ontology;    -- only if apiOntology.enabled (default true)

\c auth
CREATE EXTENSION IF NOT EXISTS vector;

\c agentic
CREATE EXTENSION IF NOT EXISTS vector;

\c cortex
CREATE EXTENSION IF NOT EXISTS vector;
SQL

The bundled chart creates these automatically via an initdb script on the in-cluster Postgres; for managed DBs you must do it yourself. analytics and ontology do not require pgvector; auth, agentic, and cortex do.

Why this must happen before helm install: the chart runs per-service Drizzle migration Jobs as post-install hooks against DATABASE_URL directly — drizzle-kit migrate does not create the database itself. A missing database surfaces as auth-migration / agentic-migration / cortex-migration / api-ontology-migration Jobs in CrashLoopBackOff shortly after install.

Configuration

The chart supports two modes. Pick one per app — when both are set for an app, infra.db.external.existingSecrets.<app> wins over infra.db.external.<app>Url.

Mode A — external Secrets (recommended)

Keep DSNs out of values files and CI logs. Pre-create one Secret per app, each in the app's target namespace, each carrying a single DSN under the key that app expects. The chart adds each as a second envFrom on the corresponding deployments.

1. Create one Secret per enabled component (five with chart defaults):

NS=scrydon-platform   # adjust if you use split namespaces

# auth → key DATABASE_URL → auth DB
kubectl create secret generic scrydon-db-auth -n "$NS" \
  --from-literal=DATABASE_URL='postgres://scrydon:REDACTED@pg.example.com:5432/auth?sslmode=require'

# agentic → key DATABASE_URL → agentic DB
kubectl create secret generic scrydon-db-agentic -n "$NS" \
  --from-literal=DATABASE_URL='postgres://scrydon:REDACTED@pg.example.com:5432/agentic?sslmode=require'

# analytics → key DATABASE_URL_ANALYTICS → analytics DB
kubectl create secret generic scrydon-db-analytics -n "$NS" \
  --from-literal=DATABASE_URL_ANALYTICS='postgres://scrydon:REDACTED@pg.example.com:5432/analytics?sslmode=require'

# cortex → key DATABASE_URL_CORTEX → cortex DB
kubectl create secret generic scrydon-db-cortex -n "$NS" \
  --from-literal=DATABASE_URL_CORTEX='postgres://scrydon:REDACTED@pg.example.com:5432/cortex?sslmode=require'

# ontology → key DATABASE_URL_ONTOLOGY → ontology DB
kubectl create secret generic scrydon-db-ontology -n "$NS" \
  --from-literal=DATABASE_URL_ONTOLOGY='postgres://scrydon:REDACTED@pg.example.com:5432/ontology?sslmode=require'

Why one Secret per app: each app reads its DSN env var (DATABASE_URL, DATABASE_URL_ANALYTICS, DATABASE_URL_CORTEX, DATABASE_URL_ONTOLOGY) via envFrom, which merges every key of the referenced Secret into the container env. Sharing one Secret would force apps that read DATABASE_URL (auth, agentic) to see the same value and point at the same database — wrong. Per-app Secrets keep each app pinned to its own database while allowing all apps to share a namespace.

2. Install the chart:

# values.customer.yaml
infra:
  db:
    enabled: false
    tls:
      enabled: false
    backup:
      enabled: false
    external:
      existingSecrets:
        auth: scrydon-db-auth
        agentic: scrydon-db-agentic
        analytics: scrydon-db-analytics
        cortex: scrydon-db-cortex
        ontology: scrydon-db-ontology

helm upgrade --install scrydon oci://scrydonops.azurecr.io/scrydon/charts/scrydon \
  --version <version> \
  --namespace scrydon-platform \
  -f values.customer.yaml

If you manage secrets via External Secrets Operator, Sealed Secrets, or SOPS, make sure the resulting Secret has the correct key (DATABASE_URL or DATABASE_URL_ANALYTICS) in the right namespace — the chart never inspects how the Secret got there.

You may also mix modes per app (e.g. existingSecrets.auth for auth, agenticUrl for agentic) — Mode A wins per app when both are set.

Mode B — inline DSNs

Simpler — one values block, no pre-created Secrets. DSNs end up in the chart-managed secrets-<app> Secret inside the cluster. Fine for staging; avoid in production (DSNs sit in values files, CI logs, and helm get values output).

# values.customer.yaml
infra:
  db:
    enabled: false
    tls:
      enabled: false
    backup:
      enabled: false
    external:
      authUrl:      "postgres://scrydon:REDACTED@pg.example.com:5432/auth"
      agenticUrl:   "postgres://scrydon:REDACTED@pg.example.com:5432/agentic"
      analyticsUrl: "postgres://scrydon:REDACTED@pg.example.com:5432/analytics"
      cortexUrl:    "postgres://scrydon:REDACTED@pg.example.com:5432/cortex"
      ontologyUrl:  "postgres://scrydon:REDACTED@pg.example.com:5432/ontology"
      sslMode: require   # appended as ?sslmode=require to each DSN

What still needs to stay in-cluster

Migrations

The chart runs one Helm-hook Job per enabled component — auth-migration, agentic-migration, and (when enabled) analytics-migration, cortex-migration, api-ontology-migration — as post-install / pre-upgrade hooks. Each one sources its DSN from the same Secret the runtime uses, so they apply against your managed DB.

The wait-for-db init container that ping-tests the in-cluster service FQDN is skipped when infra.db.enabled=false — external DB readiness is the provider's responsibility.

To re-run all migrations manually after a values change:

kubectl delete job -l 'app in (auth-migration,agentic-migration,analytics-migration,cortex-migration,api-ontology-migration)' -n scrydon-platform
helm upgrade scrydon oci://scrydonops.azurecr.io/scrydon/charts/scrydon \
  --version <version> \
  --namespace scrydon-platform \
  -f values.customer.yaml

Verification

# Apps should be Ready with Dapr sidecars injected (2/2). The chart defaults
# every service to scrydon-platform; add your namespaces.* targets if you split.
kubectl get pods -n scrydon-platform

# No in-cluster Postgres should exist
kubectl get deploy db -n scrydon-platform
# Expected: Error from server (NotFound)

# Migration Jobs should have run successfully
kubectl get jobs -n scrydon-platform
# auth-migration-<rev>, agentic-migration-<rev>, and (when enabled)
# analytics-migration-<rev>, cortex-migration-<rev>, api-ontology-migration-<rev> — Complete

# A running app pod should have DATABASE_URL pointing at the managed host
kubectl exec deploy/api-platform -n scrydon-platform -c api-platform -- \
  sh -c 'echo "$DATABASE_URL" | sed -E "s#//[^@]+@#//REDACTED@#"'

Provider notes

AWS RDS / Aurora

• Parameter group: set rds.force_ssl=1.
• Extension: pgvector available from Postgres 15.3 on RDS and Aurora. Install via CREATE EXTENSION vector; from an rds_superuser-owned role.
• Role: create a least-privilege role (CREATE ROLE scrydon LOGIN PASSWORD '…' CREATEDB;) and grant it per-DB ownership.
• IAM auth: not supported by the chart today — use password auth.

GCP Cloud SQL

• cloudsql.iam_authentication=on flag is optional; chart uses password auth.
• pgvector enabled by default on Postgres 15+ Enterprise Plus; on standard tiers run CREATE EXTENSION vector; once.
• Private IP: run the cluster in the same VPC as your Cloud SQL instance, or front it with a Cloud SQL Auth Proxy sidecar.

Azure Database for PostgreSQL (Flexible Server)

• Server parameter: add VECTOR to azure.extensions before the first CREATE EXTENSION.
• Networking: private DNS zone + VNet peering is recommended; alternatively open firewall rules for the AKS egress IP.
• CA bundle: Azure publishes a combined CA bundle. Mount it and use sslmode=verify-full&sslrootcert=/etc/ssl/azure-ca.pem for full verification.

Supabase / Neon

• pgvector enabled by default.
• Connection string format (pooled, transaction mode): postgres://postgres.<project>:<pass>@aws-0-<region>.pooler.supabase.com:6543/postgres. Verify your migrations run via the pooled connection; some DDL may require the direct (5432) connection.
• Neon's auto-suspend will freeze idle branches — set a keepalive or use a paid tier for platforms that idle between workflow runs.

Restore onto managed Postgres

To restore a pg_dump --format=custom archive produced by the bundled backup CronJob (see Backup & Restore):

# decompress if you used gzip
gunzip -k /backups/auth-20260416T020000Z.sql.gz

# restore into the managed DB (run from any client with network access)
pg_restore -d "$DATABASE_URL" --no-owner --no-privileges \
  /backups/auth-20260416T020000Z.sql

--no-owner --no-privileges is important — role names on the managed DB usually differ from the bundled Postgres (postgres vs scrydon / rds_superuser / cloudsqlsuperuser).

After restore, roll the app pods so they pick up fresh connections:

kubectl rollout restart deployment -n scrydon-platform

Troubleshooting

CREATE EXTENSION vector fails — the managed provider does not have pgvector on your current Postgres version / tier. Upgrade the instance; no workaround.

App pods CrashLoopBackOff with ECONNREFUSED or SSL required — DSN host/port wrong, TLS mandatory but sslmode missing, or network policy blocks egress to the DB. kubectl logs the pod, check the DSN value with the kubectl exec command under Verification.

Migrations hang — the wait-for-db init container is skipped, so the Job goes straight to migrating. If it hangs, the DSN is wrong or the role lacks CREATE privileges. Inspect kubectl logs job/auth-migration-<rev>.

Dapr workflow state table not found — the Dapr Workflows component auto-creates its schema on first start. If you are replacing the bundled DB with an existing managed DB that was used elsewhere, ensure the tablePrefix values (dapr_workflow_state, analytics_dapr_workflow_state) do not collide.

Compliance mapping

• ISO 27001:2022 A.8.13 (information backup) — managed provider's native backup satisfies this; disable infra.db.backup.enabled.
• ISO 27001:2022 A.8.24 (TLS in transit) — sslmode=require minimum, verify-full preferred.
• NIST CP-9 / CP-10 (backup & restore) — managed PITR coverage; document the provider's RPO/RTO in your BCP.
• SecNumCloud 3.2 (continuity) — requires regional or multi-AZ replication, which the bundled in-cluster Postgres cannot deliver. BYO managed Postgres is the supported path.