How Cortex re-fits data

Data does not arrive finished. Rules get written after the rows they should have applied to. Entities that mattered in March stop being mentioned in August. This page is about the two mechanisms that keep an already-ingested graph fitted to what you know now.

The spine

A source row is persisted, embedded, enriched into entities and edges, written into the graph, and served by retrieval. Each stage reads the stage before it, and the enrich stage is the one that can be run again later.

Enrichers are pure functions

An enricher takes a row and returns an EnrichmentResult. It writes nothing. A single Writer owns every database write and every conflict rule, which is what makes the following properties hold rather than mostly hold. All three are enforced by tests.

Property	What it means for you
Idempotent	Running enrichment on the same row again produces the same graph. Re-runs are safe, and they are the normal case, not an exception.
Atomic per row	A row's enrichment commits as a unit. If the commit happened, everything for that row landed. There is no half-enriched row to detect.
Cite-able	edge_sources and node_sources accumulate evidence across re-runs, so an edge asserted by three separate rows records all three.

One enricher failing is logged and counted. It never aborts the row, and it never takes the other enrichers down with it.

Every fact records how it was derived

Every membership and every edge is tagged with the mechanism that produced it. This is the part that lets an agent rank one claim above another, and it is the whole point of the design: a declared field and a guess are both in the graph, but they are never indistinguishable.

The seven rungs, in descending order of trust:

Rung	Kind of evidence
structural	A declared field. The row said so. Gold.
lexical_canonical	An exact match after normalization.
lexical_fuzzy	An approximate string match.
embedding_similarity	Geometric closeness in embedding space.
cluster_centroid	Membership derived from a cluster rather than a pair.
temporal_adjacency	Things that happened close together in time.
llm_extraction	A model read the text and asserted it. Probabilistic.

Structural facts are gold. Similarity is geometric. LLM extraction is probabilistic. Treat them differently, because the graph already does.

Re-enrichment on rule change

A feed's extraction rules can change. When they do, the feed's rulesVersion is bumped, and a backlog drainer re-runs rows that were already embedded against the new rules.

The practical consequence: old data gets the benefit of a rule written later. You do not have to have known, at ingest time, everything you would eventually want to extract. Nothing is re-fetched from the source and nothing is re-embedded; the enrich stage runs again over rows that are already there.

The consolidation cycle

Separately, a maintenance pass runs over the graph. Think of it as sleep-time work rather than request-time work. It does three things:

1. Re-score every entity

   salience = evidence x recency x confidence. An entity that keeps being reaffirmed by new rows stays high; one that stopped being mentioned decays.

2. Re-apply the junk gate

   The same quality gate that runs at extraction time is applied again, so entities that should never have been promoted get caught on a later pass.

3. Invalidate the dead weight

   An entity is invalidated only when it is both below a salience floor and unreaffirmed past a TTL. Both conditions are required. This is deliberately conservative: a quiet but well-evidenced entity survives, and so does a weak but recently reaffirmed one.

Where to go next