How Functions work

A function is a contract around an LLM call or a short workflow you already have: fixed instructions, declared variable inputs and an output schema. Seldon searches for cheaper or faster ways to fulfil that contract, measures them against your original call, and reports the trade-offs. You decide what to activate.

The reference#

Your original call is the reference. Seldon never edits it. Every alternative is measured against it on the same inputs, and the reference remains the fallback a pipeline can use when it detects a failure.

Candidate pipelines#

Alternatives range from the same prompt on a cheaper model to composed pipelines that combine smaller models, specialized extraction or decision components and deterministic steps such as parsing, lookup and arithmetic. A specialized winner is never required; the best alternative may simply be a different model.

Synthetic evaluation#

Seldon generates evaluation inputs from your frozen instructions and input contract, covering ordinary, empty, ambiguous, negative and conflicting cases. Generation, search and the final held-out evaluation use separate data.

• Held-out data is independent of search. It is still synthetic, not a sample of your traffic.
• New datasets don't include prompt-injection cases, so results don't establish prompt-injection robustness.
• Evaluation is frozen when optimization starts. Later edits affect future runs only.

See Evaluation for how outputs are judged.

What the measurements mean#

Pass rates, cost and latency describe the synthetic evaluation workload under the stated timing conditions. They are not a guarantee for each request or a production service level. Load, provider behavior, your input distribution and fallback can all change production results.

The report keeps limitations visible: an unresolved judgment is never a pass, a missing metric is never zero, and search observations are never labeled held-out.

Why activation is your decision#

Quality, cost and latency inform the choice but do not block it. A cheaper pipeline with a slightly lower pass rate can be the right trade-off for one application and the wrong one for another. Seldon only requires that an activatable pipeline actually ran: valid bindings, supported pinned models and at least one schema-valid primary execution.

Activation makes a revision callable. It never edits your application, intercepts other requests or moves traffic on its own.

Fallback#

At call time, a pipeline that detects a failure can run the frozen original workflow within the same deadline and spending limits, and the response reports fallback_used. Fallback catches detected failures. It is not an oracle for silently wrong answers.

What is not included#

• Automatic discovery of what to optimize from your gateway traffic.
• Gateway substitution, where matching requests would be redirected to a function without changing the model field.
• Streaming, tool execution, open-ended conversation and multimodal input in Functions.