How it works
The cluster is the spec. The trace is the test.
Seldon is a restricted, example-driven form of program synthesis over a typed library of classical NLP, IR, and ML operators — made trustworthy by verifying every candidate against your own traffic before it serves a single request. Here is what that means, and why it holds up.
The thesis
Most LLM calls are under-specified data workflows.
A large share of production LLM traffic is not open-ended reasoning. It is a small, well-established set of classical NLP, information-retrieval, and machine-learning tasks — classification, extraction, deduplication, ranking — dressed up as prompts. Each of those tasks had a cheaper, more deterministic solution long before LLMs existed.
Repeated traces reveal the business process, ontology, and data model that should have existed all along. Seldon's job is to recover that latent structure and compile the recurring work onto the traditional pipeline for it — calling frontier models only where they genuinely earn their cost.
Ontology
We don't compile traces to code. We recover the structure underneath them.
Going straight from prompts to a pipeline would bake in every accident of phrasing. So the compiler first infers three linked views — jointly, not as a rigid waterfall — and only then synthesizes against them. This is the layer where Seldon takes a position: an LLM workflow is a shadow cast by a real domain model, and the ontology is the object casting it.
Business process
The workflow actually being performed — sales opportunity management, support triage, earnings monitoring, invoice reconciliation. The recurring shape of the work, recovered from what the traffic does.
Ontology
The domain entities and their relationships — Account, Opportunity, Meeting, Risk; “Meeting discusses Opportunity”, “Risk affects Opportunity”. The compiler induces the schema the workflow implies but never wrote down.
Data model
The ontology made operational: tables, columns, types, keys, valid states, freshness expectations, validation rules, lineage, and confidence scores. This is what a pipeline can actually target.
These three views are lowered into a Seldon Intermediate Representation (SIR): a typed operator graph where every node carries an input/output contract, quality constraints, cost and latency estimates, and a fallback policy. The SIR — not the prompt — is what gets compiled, reviewed, and versioned. It is the durable artifact that turns a non-deterministic prompt chain into a data contract you can audit.
Fig.1 — Trace to pipeline
Program synthesis
Restricted, example-driven, neurosymbolic synthesis over a typed operator library.
Naming the theory keeps us honest and keeps us from over-claiming. Seldon draws on four mature lineages: programming-by-example (the trace cluster is the specification — real input/output pairs, not a hand-written contract), type-directed component synthesis (search a fixed, typed library of operators, with type contracts pruning the space), neural-guided search (a frontier model proposes decompositions at compile time as a learned prior), and library learning (verified plans are promoted into a growing catalog of reusable composition patterns).
The search space is deliberately small: the seven atomic task families below. Each is registered as a typed operator with a machine-checkable signature, so composition becomes a typing problem — operator A may feed operator B only if A's output contract satisfies B's input contract. The result is a shallow directed graph of whole NLP/ML operators — no loops, no recursion. That coarse granularity is the single biggest reason the problem is tractable.
Classification
text → label ∈ a small fixed set
Sequence labeling
output spans ⊂ the input text
Structured extraction
output is a stable JSON schema
Retrieval / matching
output references a bounded catalog
Similarity / clustering
pairwise comparison or grouping
Normalization / transform
deterministic function of input
Numeric / analytical
a number derivable by computation
Synthesis tries three strategies, cheapest first: known composition patterns (curated classical decompositions like detect → link or extract → normalize → validate → write), then general type-directed enumeration backward from the output type, and only then a frontier model as a planner. The expensive model runs once, at compile time; the grounded plan is cached, and every future request uses the cheap compiled path. That is the compiler bargain — costly optimization once, cheap execution forever after.
Why you can trust it
Verified against your own traffic before it serves a request.
The oracle is the arbiter, not the proposer.
A candidate pipeline is accepted only if, replayed against your held-out historical traces, it reproduces the model's outputs above threshold on each operator's native metric. A bad decomposition — whatever proposed it — simply fails replay and is discarded.
Soundness is empirical, never a proof.
We make no claim of formal correctness. Every compiled path is shadow-tested against live traffic and promoted only once its accuracy and coverage clear a threshold, with a human in the loop on promotion.
Compilation is partial, not all-or-nothing.
Per-operator confidences compose into a per-output confidence. The compiled path serves the confident majority; low-confidence items fall back to the frontier model per request. A workflow has a coverage fraction, not a binary verdict.
Frontier fallback is always retained.
Genuine reasoning, open-ended generation, and agentic work stay on the frontier model — and are labeled non-compilable so your coverage reporting stays honest. Seldon compiles what it can prove, not everything.
See it on your own traffic.
Point your base URL at Seldon, keep shipping, and watch recurring work turn into cheaper, deterministic pipelines — with frontier fallback always retained.