Context and inference cost

Every retrieval system decides where the semantic work happens. Cortex puts it at ingest. Most per-query RAG stacks put it at query time. The consequences of that one choice run through cost, latency, and which models you can actually use.

Where the work happens

	Per-query stack	Cortex
Embedding	At ingest.	At ingest.
Understanding what an item means	On every query, by re-reading the retrieved chunks with a model.	Once, at ingest, into typed entities and edges.
Cost of the second agent reading the same item	The same as the first.	Nothing. The item was already understood.
Cost of the same agent asking again tomorrow	The same as today.	Nothing.
What the query returns	Text, for a model to read.	Scored, typed, cited evidence.

Enrichment is paid once per item. It does not scale with how many times the item is read, or with how many agents read it. A per-query stack re-derives meaning on every call, for every agent, forever.

That is an architectural argument, and it is the honest form of the claim. We do not publish a dollar figure, because the number depends entirely on your model, your traffic and your corpus.

There is no LLM at query time

answer() runs vector kNN, full-text search, and a one-hop graph expansion, fused with reciprocal rank fusion. No model is called, which is why retrieval is measured in milliseconds rather than seconds: the published median answer latency is 477ms.

What comes back is evidence, not prose:

const answer = await cortex.answer(officeId, { query, limit: 5 });

for (const r of answer.results) {
  console.log(r.score, r.title, r.universal_id, r.source_url);
}

Your agent does the reasoning. That means you choose the model, per call, with full knowledge of how much evidence it has to chew on.

What your model actually receives

The second half of the cost story is context. A pile of retrieved documents consumes tokens in proportion to how much text was retrieved, and most of those tokens are haystack. A small set of scored, typed, cited results consumes tokens in proportion to how many facts you asked for.

const memory = client.cortex(officeId).forAgent(officeId, 'atlas');
const promptBlock = await memory.contextFor('what did pricing move to?', { maxItems: 5 });

contextFor renders a citation-formatted block sized by maxItems, so the prompt you build has a shape you control rather than one your retrieval happened to produce.

Local embeddings

Ingest-time work does not have to leave your infrastructure. An office can be configured to use a local Ollama embedding pool running mxbai-embed-large.

To be precise about what that moves: it is the embedding model that runs locally. It is the component that turns text into vectors during ingest, not a generation model and not something in the query path, because the query path does not call a model at all.

Where to go next