# How Cortex re-fits data

> The derivation ladder, re-enrichment on rule change, and the consolidation cycle.

Source: https://mitosislabs.ai/developers/concepts/enrichment

Data does not arrive finished. Rules get written after the rows they should have
applied to. Entities that mattered in March stop being mentioned in August. This
page is about the two mechanisms that keep an already-ingested graph fitted to
what you know now.

## The spine

<DiagramSpine />

A source row is persisted, embedded, enriched into entities and edges, written
into the graph, and served by retrieval. Each stage reads the stage before it,
and the enrich stage is the one that can be run again later.

## Enrichers are pure functions

An enricher takes a row and returns an `EnrichmentResult`. It writes nothing. A
single **Writer** owns every database write and every conflict rule, which is
what makes the following properties hold rather than mostly hold. All three are
enforced by tests.

| Property | What it means for you |
| --- | --- |
| **Idempotent** | Running enrichment on the same row again produces the same graph. Re-runs are safe, and they are the normal case, not an exception. |
| **Atomic per row** | A row's enrichment commits as a unit. If the commit happened, everything for that row landed. There is no half-enriched row to detect. |
| **Cite-able** | `edge_sources` and `node_sources` accumulate evidence across re-runs, so an edge asserted by three separate rows records all three. |

One enricher failing is logged and counted. It never aborts the row, and it never
takes the other enrichers down with it.

## Every fact records how it was derived

Every membership and every edge is tagged with the mechanism that produced it.
This is the part that lets an agent rank one claim above another, and it is the
whole point of the design: a declared field and a guess are both in the graph,
but they are never indistinguishable.

The seven rungs, in descending order of trust:

| Rung | Kind of evidence |
| --- | --- |
| `structural` | A declared field. The row said so. Gold. |
| `lexical_canonical` | An exact match after normalization. |
| `lexical_fuzzy` | An approximate string match. |
| `embedding_similarity` | Geometric closeness in embedding space. |
| `cluster_centroid` | Membership derived from a cluster rather than a pair. |
| `temporal_adjacency` | Things that happened close together in time. |
| `llm_extraction` | A model read the text and asserted it. Probabilistic. |

Structural facts are gold. Similarity is geometric. LLM extraction is
probabilistic. Treat them differently, because the graph already does.

<Tip>
  When an answer surprises you, the derivation kind is usually the explanation.
  A confident-looking edge sitting on `embedding_similarity` is telling you the
  two things are near each other, not that one is a fact about the other.
</Tip>

## Re-enrichment on rule change

A feed's extraction rules can change. When they do, the feed's `rulesVersion` is
bumped, and a backlog drainer re-runs rows that were **already embedded** against
the new rules.

The practical consequence: old data gets the benefit of a rule written later. You
do not have to have known, at ingest time, everything you would eventually want
to extract. Nothing is re-fetched from the source and nothing is re-embedded; the
enrich stage runs again over rows that are already there.

## The consolidation cycle

Separately, a maintenance pass runs over the graph. Think of it as sleep-time
work rather than request-time work. It does three things:

<Steps>
  <Step title="Re-score every entity">
    `salience = evidence x recency x confidence`. An entity that keeps being
    reaffirmed by new rows stays high; one that stopped being mentioned decays.
  </Step>
  <Step title="Re-apply the junk gate">
    The same quality gate that runs at extraction time is applied again, so
    entities that should never have been promoted get caught on a later pass.
  </Step>
  <Step title="Invalidate the dead weight">
    An entity is invalidated only when it is **both** below a salience floor
    **and** unreaffirmed past a TTL. Both conditions are required. This is
    deliberately conservative: a quiet but well-evidenced entity survives, and so
    does a weak but recently reaffirmed one.
  </Step>
</Steps>

<Warning>
  Entities are **invalidated, not deleted**. The graph is append-only: every
  enrich write is an upsert, and nothing is destroyed. Invalidation changes what
  retrieval will surface, not what the graph holds.
</Warning>

## Where to go next

<CardGroup cols={2}>
  <Card title="Typed entities" href="/developers/concepts/typed-entities">
    What extraction rules declare, and how a row becomes a node.
  </Card>
  <Card title="What Cortex is" href="/developers/concepts/what-is-cortex">
    The ingest-once, read-many shape this all serves.
  </Card>
</CardGroup>