Concept
What is Inference?
Also known as: model inference, generation
Inference is the act of running a trained model to produce output — the request-time work of turning a prompt into a response, as opposed to the training that created the model.
Training and inference are two different jobs on the same weights. Training happens once, costs millions, and produces the model. Inference happens every time anyone uses it: the prompt is tokenised, run through the network, and the output generated one token at a time, each conditioned on all the ones before. That sequential generation is why output tokens cost more and take longer than input tokens — the input can be processed in parallel in one pass, the output cannot. Inference is what you actually pay for, per call, forever.
An agent is many inferences, not one. A multi-step Velaris run reasons, searches the capability mesh, calls a tool, reads the result and reasons again — each of those a separate inference with its own cost and its own delay, and they add up along the run. That's what model routing is for: sending each step to a model that fits it, so a cheap step doesn't pay frontier prices. It's also why context engineering has teeth — every token you carry is a token re-processed at every inference in the loop.