Program synthesis, and why compiling LLM calls into ETL is one

Seldon starts life as a drop-in LLM router, but its long game is a compiler: it turns the repeated LLM calls flowing through it into cheaper, deterministic data pipelines. That is not a metaphor. It is a restricted, example-driven form of a well-studied problem in computer science called program synthesis. This post explains what program synthesis is, and why compiling recurring LLM work into ETL/data pipelines is one honest instance of it.

What is program synthesis?

Program synthesis is the task of automatically constructing a program that satisfies a specification. Instead of a human writing the code, a search procedure finds a program — drawn from some space of allowed programs — that provably or empirically meets what was asked for. The field splits along three questions: how you specify intent, what space of programs you search, and how you verify a candidate is correct.

The most practical branch is programming by example (PBE): the spec is just a set of input/output pairs. Excel's FlashFill is the canonical success story — you type a couple of examples of the string transformation you want and it synthesizes the underlying program. PBE works because a few representative examples, plus a constrained language of operations, pin down the intended behavior tightly enough to search for it.

• Specification — a formal contract, a logical property, or (in PBE) input/output examples.
• Search space — a domain-specific language (DSL) or a typed library of components the synthesizer is allowed to compose.
• Verification — a check that a candidate satisfies the spec, by proof, by type-checking, or by testing against the examples.

Repeated LLM calls are an under-specified data workflow

Here is the observation Seldon is built on: a large share of production LLM calls are not open-ended reasoning. They are the same shape, run thousands of times a day — classify this ticket, extract these fields from this document, normalize this address, link this company name to a ticker. The prompt is doing a data-processing job that was never written down as a program. It is an under-specified workflow that happens to be implemented by a frontier model.

But every one of those calls leaves a trace: an input, and the model's output. Cluster the traces by intent and you have exactly what PBE needs — a pile of input/output examples describing a recurring task. The trace cluster is the specification. Nobody had to author it; the traffic wrote it for you.

Seldon's compiler, framed as synthesis

Seldon represents a candidate pipeline as a Seldon Intermediate Representation (SIR): a typed graph of atomic operators — classification, extraction, sequence labeling, retrieval/matching, normalization, and a few more — glued by deterministic code. Synthesis is the search for an SIR graph whose behavior reproduces the trace cluster. Mapping it onto the three questions above:

• Specification — the clustered traces (input/output pairs) produced by the LLM. This is PBE / inductive synthesis: the examples are the spec.
• Search space — a fixed, typed library of atomic operators. Restricting the space to a known component set is what makes the search tractable; a frontier LLM proposes candidate decompositions at compile time as a learned prior over programs (neural-guided, type-directed, component-based synthesis).
• Verification — the trace oracle is the arbiter. A candidate is accepted only if, replayed against held-out historical traces, it reproduces the LLM's outputs above a quality threshold using each operator's native metric.

A synthesized SIR graph for "link company mentions to tickers" might lower to a pipeline like this — each stage a cheap, typed operator, with the frontier model retained only for the hard tail:

Input(article text)
  → Detect mention spans        (sequence labeling)
  → Normalize surface form      (deterministic transform)
  → Generate candidates         (retrieval against a ticker catalog)
  → Disambiguate / link         (matching + classification)
  → Validate + emit tuples      (schema check)
  ↳ low confidence? → fall back to the frontier LLM

Three honest departures from classical synthesis

Calling this "program synthesis" would be over-claiming without naming where it departs from the textbook version. There are three, and they are deliberate.

1. Acceptance is statistical, not a proof

Classical synthesis wants a program that provably satisfies the spec. Seldon accepts a program that matches the oracle above a threshold on held-out examples, and routes the uncertain remainder back to the LLM per request. Soundness is empirical — it comes from testing and shadow-mode replay, never from a proof.

2. The oracle is fallible

The examples come from an LLM that is itself sometimes wrong, so Seldon is synthesizing a cheaper program to imitate a noisy teacher. That makes this as much knowledge distillation as synthesis. On genuinely deterministic sub-tasks the compiled pipeline can even be more correct than the oracle — an inversion classical synthesis does not have.

3. The spec is inferred, not given

Nobody hands Seldon a contract. It first has to infer the I/O contract — which fields, which types, which task family — from the clustered traces before it can search. There is a spec-inference step bolted in front of the program search.

Knowing when not to compile

A synthesizer is only trustworthy if it can say "no." Seldon runs a cascade of compilability gates, cheapest first, and declares a task non-compilable if it trips any of them:

• Determinism screen — if the output carries information not derivable from the input plus available reference data, no composition of operators can manufacture it. Genuine generation and open-world reasoning fail here.
• Type reachability — if no operator or pattern can produce the target output type from the input type, the task is out of the language.
• Functional-relation test — if near-identical inputs map to wildly different outputs, the input→output relation is not a stable function to approximate.
• Verification threshold — the decisive, empirical gate: if no candidate clears the accuracy bar on held-out replay at any useful coverage, the task is non-compilable for now.

Crucially, compilation is partial, not all-or-nothing. Per-operator confidences compose into a per-output confidence; a threshold sends the confident majority down the compiled path and defers the hard tail to the frontier model. So the output of the decision procedure is not a boolean but a tuple — best plan, coverage fraction, accuracy, projected savings — and a pipeline ships only when that clears an economic bar. The frontier LLM is always retained as the fallback, so partial coverage is safe.

Why this matters

Framing the compiler as synthesis is not academic housekeeping. It buys three things. It keeps us honest — the oracle, not an eager planner, decides what ships, so a wrong pipeline fails replay before it ever serves traffic. It makes "we couldn't compile this" a first-class, logged outcome rather than a silent failure. And because non-compilability only ever shrinks as the operator library grows, coverage ratchets upward over time: the same traffic gets cheaper as Seldon learns more shapes.

You still think you are calling an LLM. Increasingly, the answer is produced by a compiled program that Seldon synthesized from your own traffic — and, when it matters, you can take that program with you.