Reference link join rules

Background: how edges are normally found

A reference link type infers edges by value-join: a property on the source object must equal a column on the target object. By default the engine auto-discovers which column to join on using a heuristic: it looks for shared token overlap between the source column name and the target's identity columns (the columns declared in defineIdentityRule) and the target binding's non-identity columnMap columns — identity columns are ranked first. This probing was broadened in this release to cover columnMap columns in addition to identity columns.

This works well when source and target column names share clear token overlap, regardless of whether the target column is an identity column or not. It does not work when:

• The column names don't overlap even though the values match semantically — the heuristic ranks candidates by token overlap, so columns with no shared tokens score zero and are not probed.
• The values are stored in different formats on each side (e.g. ASN as "AS3215" vs 3215, cveId with different casing) — the heuristic always uses raw equality.

In these cases the auto-discovery produces either 0 edges or the wrong edges — silently, with no error.

Declaring an explicit join rule

Add a join object to defineLinkType(...). The engine uses it deterministically and skips auto-discovery entirely:

import { defineOntology, defineLinkType } from '@scrydon/sdk-authoring/ontologies'

defineLinkType({
  slug: 'VulnAffectsVendor',
  displayName: 'Vulnerability affects vendor',
  sourceTypes: ['Vulnerability'],
  targetTypes: ['NetworkOperatorVendor'],
  // No kind needed — "reference" is the default.
  join: {
    sourceColumn: 'vendorProject',   // column on the Vulnerability side
    targetColumn: 'vendorName',      // column on the NetworkOperatorVendor side
  },
})

vendorName is not an identity column of NetworkOperatorVendor, so auto-discovery would miss it. Declaring join pins the engine to exactly this column-pair.

Why this example matters

CISA KEV records store the affected vendor as vendorProject. The NetworkOperatorVendor object type holds its canonical identifier in vendorName — a non-identity property. Without an explicit join rule the heuristic probes both identity and non-identity columns (token-ranked), but results depend on name overlap and may be non-deterministic when multiple columns are candidates. Declaring join pins the engine to exactly this column-pair and makes the edge-inference deterministic.

Value normalization

If the same logical value is stored in different formats on each side, use normalize to make them comparable at join time:

defineLinkType({
  slug: 'AsPathUsesAsn',
  displayName: 'AS-path entry uses ASN',
  sourceTypes: ['AsPathEntry'],
  targetTypes: ['AutonomousSystem'],
  join: {
    sourceColumn: 'asnRaw',     // stored as "AS3215" (string with prefix)
    targetColumn: 'asNumber',   // stored as 3215 (integer or plain number string)
    normalize: 'digits',        // strip all non-digit characters before comparing
  },
})

Normalization is applied to both sides before comparing. The join is a standard equality after normalization.

CVE ID casing example

Some feeds store CVE identifiers as CVE-2024-12345 and others as cve-2024-12345. Normalize to lowercase to treat them as equal:

defineLinkType({
  slug: 'ExploitTargetsCve',
  displayName: 'Exploit targets CVE',
  sourceTypes: ['Exploit'],
  targetTypes: ['CveRecord'],
  join: {
    sourceColumn: 'cveRef',
    targetColumn: 'cveId',
    normalize: 'lower',
  },
})

When to declare join

Declare an explicit join rule when:

• Column names share no token overlap. Auto-discovery (broadened in this release to probe both identity and non-identity columnMap columns) ranks candidates by shared tokens between source and target column names — if no tokens overlap, the pair scores zero and is not probed, producing 0 edges silently.
• The values are stored in different formats. The heuristic always uses raw equality, so format differences (e.g. "AS3215" vs 3215, CVE identifier casing) require an explicit join with normalize.
• The auto-discovered join is ambiguous. When multiple target columns share tokens with the source column, declare join to make the choice explicit.

You do not need join when:

• The source column name shares clear token overlap with a target column (identity or non-identity columnMap) and both sides store values in the same format.
• Both sides use the same column name and the same format.

Prefer canonicalizing at ingest when you can

normalize is intended for cases where you do not control the source format — for example, when two external feeds publish ASNs in different formats.

If you do control how values are ingested (e.g. your own ETL pipeline populates the table), the better approach is to store values in a canonical format on both sides at ingest time. This is faster at query time, easier to understand, and avoids hidden normalization logic in the pack.

Rule of thumb: fix the data at ingest when you own it; use normalize only when you don't.

Full schema reference

join?: {
  sourceColumn: string   // Property slug on the source object type
  targetColumn: string   // Property slug on the target object type
  normalize?: 'trim' | 'lower' | 'digits'
}

Proximity links are unaffected

This page is about reference link types. Proximity link types (kind: "proximity") use a completely different edge-inference mechanism — haversine distance — and do not use join at all. See the Proximity link types guide.

Where to next