AI applications need better primitives than text generation
A production LLM call is often a latent program wearing a generate() interface. Classification, extraction, linking, ranking, normalization, and arithmetic have different output spaces and different ways to fail, and the useful layer sits under the prompt: name the operation, recover a candidate program from traffic, then choose an implementation per step.
TypeSafe's Jev is a real correction for one of those shapes. A category, an ordered score, or a probability over a yes-or-no question is a better machine interface than a paragraph. Decisions are the start of the vocabulary. They are not the whole vocabulary. Naming the operation is what lets a runtime stop treating the original call as atomic.
One call, already a program
"Read this annual report and return every customer–supplier relationship as {customer, supplier, evidence}" sounds like general reasoning. The computation is a classical information-extraction stack. Named-entity recognition, normalization, candidate generation, entity linking, relation extraction, schema validation. The frontier model belongs on the slice that falls outside the distribution those operators were built for.
A support ticket has the same shape, with a different chain. Language, intent, and sentiment are classifications. Priority is an ordered score downstream of those judgments. Account resolution is an entity link. The record is a schema check. The API records one inference. The chain records a dependence: severity is a function of what the ticket is about, so priority sits after intent and sentiment. Dropping that dependence would be a different program.
The dashed bypass is the part worth keeping when you factor a call. The original email still lands on the triage record. Structured fields do not replace the evidence they were read from.
The frontier model earns its place at the beginning of this process. It lets a team ship behavior while the ontology, the schema, and the exceptions are still unclear. After enough traffic, the runs are a set of examples: inputs, outputs, corrections, and the cases that broke. Finite examples do not pin down a unique program. Many programs agree on the traffic you have and diverge on the next request. What the examples support is a search, inside a vocabulary small enough to mean something. That search is programming by example. The traces are the spec. The families below are the language the spec is allowed to use.
Decisions start the vocabulary
PII detection wants a probability over a boolean. Department routing wants a choice from a fixed set. Severity wants an ordered score. A writing interface discards structure the application already had. Turning the probability into an action still takes a threshold, chosen for what a wrong decision costs.
Classification can be stretched further. Retrieval can be posed as choosing among documents, span extraction as choosing among substrings, entity resolution as a same-entity decision over pairs. Those reductions are valid once someone has already built a finite candidate set — and that step is the structure the reduction erases. Without it, a softmax over a corpus is intractable, pairwise resolution is quadratic, and classifying every substring is quadratic where sequence labeling is linear.
A primitive should keep the structure an optimizer can use. A searchable candidate space. Span locations. Canonical identities. Exact arithmetic wherever an exact function exists.
A working taxonomy
This is how we are thinking about the solution — but it is by no means the only taxonomy or even the optimal one.
Also, maybe fine-tuned small LLMs.
Potentially, different teams and systems may have a proprietary taxonomy of tasks that fits them better.
disposition
The line under each family is one implementation that often fits, not a law. A small classifier for a stable label set. A lookup index for a bounded catalog. A formula engine for arithmetic and date math. Span labeling is frequently a sequence model or a fine-tune; the margin note on the figure is the reminder that a small language model is a physical strategy, sitting alongside rules and indexes. The hatched cell is the residue: calls that stay on a frontier model because no operator in the vocabulary reproduces them.
This is the cut we use to describe repeated workloads — seven families, forty-one subgroups in the v1 registry. It is a catalog of shapes, with a sketch of how they compose. It becomes an instruction set once each operator has a semantics, a way to compose, and a cost. Categories will be re-cut. Legal work will care about clauses and citations. Finance will care about security resolution and corporate actions. Healthcare and logistics will bring their own. The point of publishing a taxonomy is to make computational shape explicit enough that a runtime can select an implementation, and to leave room for a cut that fits a domain better than ours.
The operation and the implementation are different layers
ENTITY_MATCH(company_mention, company_master) can run as an exact lookup, a fuzzy rule, an embedding index, a gradient-boosted model, a cross-encoder, a decision model, a small language model, or a frontier model. Those are execution strategies. The operation's meaning is the match.
Databases made this split decades ago. SQL states the join. The planner picks a hash join, a merge join, or an index lookup, using equivalences that are theorems of the relational model. A learned operator does not inherit those theorems. CLASSIFY(email, INTENT) denotes an estimator tied to a label set, a threshold, and a training distribution. Swapping a small model for a frontier model preserves the interface and changes the function. You can select implementations. Agreement with held-out traces is how you tell whether the new plan still does the job. That is a measurement on a distribution, not a proof that two plans compute the same relation.
The application can still name the steps:
language = CLASSIFY(email, LANGUAGE) intent = CLASSIFY(email, INTENT) sentiment = SCORE(email, SENTIMENT) priority = RANK(email, PRIORITY) account = ENTITY_LINK(email.sender, CUSTOMER_MASTER) record = VALIDATE(TriageRecord)
Routing picks a model for an indivisible request. Compilation can change the request. Compilation starts once a candidate program exists. Recovering that program from traffic is the synthesis step. Lowering each operator is the step after.
What the plan changes
One frontier call can lower to the chains in the figures: deterministic preprocessing, a small classifier, an index, a matcher, a normalizer, a validator, and a frontier fallback behind a novelty gate. The gate is Chow's old idea — answer when you are inside the region you know, defer when you are not. It saves money when most of the probability mass sits where the cheap operators already agree with the original call. If the defer rate is high, or the confidence is miscalibrated, the cascade costs more and the latency tail gets worse. The diagram is a plan you can measure. It is not a proof of savings.
Explicit types give the surrounding system a place to put the evidence. Retrieval can return references. Extraction can keep spans. Linking can expose canonical ids. A score field is not yet a calibrated probability; calibration is a further property you have to establish before a threshold is safe to automate. A deterministic node is exact relative to the policy it states — a calendar, a timezone, a tokenizer — and not by virtue of being drawn as a box.
Generation was the bootstrap. It let teams build programs before they understood them. Production traffic is now dense enough to search for the program inside a declared vocabulary and, where the measurements hold, run that program on cheaper machinery. The layer that does this represents what the application is doing independently of which model first did it. A production LLM call becomes a program with a plan.
The decisive infrastructure sits below the prompt and above the model: a vocabulary of operations, a search over traces, and a runtime that picks an implementation per step.
Open beta
Route your traffic through Seldon and watch recurring work turn into cheaper pipelines.