Model router & inference

Every model call goes through Vetta’s model router — the ones an agent makes inside a session, and the ones you make yourself through the model proxy. The router picks the inference backend for each call based on two inputs — the model and the completion window — meters the tokens against the caller’s balance before the call runs, and returns the result. Inside a session the harness decides when to call the model; the router — part of the runtime — decides where the call goes and what it costs. You never address a backend directly. You name a model and a window; the router does the rest. There are two ways to name them:

On an agent — model and window are fields of the agent, overridable per session. Every turn the harness runs goes through the router with them.
Directly — one call, no agent and no session, through the proxy in the Messages or Chat Completions format.

Both use the same catalogue, the same lanes and the same meter; only who pays differs. Images and video are not model calls: they have their own catalogue and their own pages — Images and Video.

Two inference lanes

Vetta federates two inference backends behind one interface:

Aggregated inference network — a broad catalogue of models available at interactive latency. This is the default lane and serves every immediate request. On this lane the router asks the network for the fastest provider that hosts the model, not the cheapest — immediate is the pay-for-speed window, and the default price-ordered routing was measured putting a cold first token tens of seconds out on a slow node. One consequence is worth stating plainly: because a model is hosted by several providers at different prices, two identical immediate calls to the same model id can cost different amounts depending on who served them. You are billed what the call actually cost, so the difference is real rather than an estimate — see What a model call costs. The priority and loose windows are unaffected.
Completion-window pool — specialized open-weights hosting that offers genuine reduced tariffs for the priority and loose windows at reduced latency cost. This lane serves priority and loose requests, and it only hosts a specific set of window-supported models.

 model call  (model + window)
        │
        ├─ window = immediate ─────────────▶ Aggregated inference network
        │                                     any model in the catalogue · immediate tariff
        │
        └─ window = priority | loose ──────▶ Completion-window pool
                       │                      priority / loose tariff
                       │
                       └─ model NOT window-supported ─▶ 400, refused before any spend

The routing rule

The window determines the lane, and the lane constrains the model:

Window	Lane	Model requirement
`immediate`	Aggregated network	Any model in the catalogue, including `vetta/auto`.
`priority`	Completion-window pool	Must be a window-supported model.
`loose`	Completion-window pool	Must be a window-supported model.

A non-default window (priority or loose) with a model that the completion-window pool does not host is refused with a typed error — window_unavailable, HTTP 400 — before any inference runs and before any spend. immediate always works, on any catalogued model. See Errors.

This is a deliberate fail-closed: rather than silently downgrading a loose request to the interactive tariff (and quietly overcharging you), the router rejects the combination so you fix it explicitly — either pick a window-supported model, or drop to immediate.

The catalogue is live

There is no curated list. The catalogue is read from the inference network at request time and cached briefly, then filtered down to the models Vetta can actually drive an agent with:

text out — a model that returns images or audio is not an agent’s model;
tool calling — the harness cannot run a loop with a model that cannot call a tool;
a published per-token price for both prompt and completion, because a model that cannot be quoted cannot be metered, and metering before the call is what makes spend fail closed.

That is hundreds of models, and it moves on its own: a model the network publishes today is runnable today, with no release of ours. Two consequences follow for anything that reads it — the listing is paged, and it is searchable. Nothing should assume one call returns the whole catalogue, and nothing should hard-code a model id it has not checked.

Discovering models

GET /v1/models is the catalogue; GET /v1/models/{id} is one entry, for the id you already hold.

vetta models list --search glm --limit 5
vetta models list --window priority          # only what that window can serve
vetta models get zai-org/GLM-5.2-FP8

curl -fsSL "https://api.vetta.sh/v1/models?search=glm&limit=5" \
  -H "authorization: Bearer sk_live_..."

# Model ids contain a slash — percent-encode the path segment.
curl -fsSL "https://api.vetta.sh/v1/models/zai-org%2FGLM-5.2-FP8" \
  -H "authorization: Bearer sk_live_..."

const page = await vetta.models.list({ search: "glm", limit: 5 });
const more = await vetta.models.list({ after: page.next_cursor });
const m = await vetta.models.retrieve("zai-org/GLM-5.2-FP8");
console.log(m.supported_windows); // ["immediate","priority","loose"]

Query	What it does
`window`	Narrow to the models that window can actually be served in — `immediate`, `priority` or `loose`.
`search`	Free text, matched against the model’s id and name.
`limit`	Page size, 1–100. Defaults to 20.
`after`	The previous page’s `next_cursor`.

The listing is cursor-paged like every other list on the API: has_more and next_cursor are real, not constants. An id the deploy does not serve is a not_found (404) from the by-id route — which makes it the cheapest way to validate a model id before you run on it.

{
  "object": "model",
  "id": "zai-org/GLM-5.2-FP8",
  "context_window": 1048576,
  "max_output_tokens": 32768,
  "supported_windows": ["immediate", "priority", "loose"],
  "efforts": ["low", "medium", "high"]
}

The model object publishes no prices — the same line GET /v1/media/models draws. What you spend is bounded by the agent’s budget before the call, and read back as actuals from vetta agent spend. A published rate card would be a number to reconcile against; the ledger is the number that is true.

?window= narrows to what a window can serve — the same derivation the router refuses on, so a model listed for a window is never rejected for it.

`max_output_tokens`

Every entry publishes max_output_tokens: the longest reply that model may produce, taken from what the model itself advertises and clamped to a platform ceiling. It is per model, not one number for the fleet — a model that can write 131 072 tokens and one that can write 8 192 are not bounded the same way. It is also what the pre-flight quote is bounded by. Before a call runs, the router reserves the worst case: every input token at the input rate, plus max_output_tokens at the output rate. The call then settles at what it actually used and the remainder is released. So max_output_tokens sets how much of a budget one in-flight call reserves, not what it costs — a long-output model holds more credit while it is running, and returns the difference when it is done.

Vetta Auto

vetta/auto is a model id like any other, and it picks the model per request: you name the task, the router picks the model that fits it, call by call. It is useful when a workload is uneven — a mix of trivial and hard turns — and you would rather not pin one model expensive enough for the worst of them.

TypeScript

const agent = await vetta.agents.create({
  name: "Triage",
  model: "vetta/auto",
  budget: { capMicroUsd: "25000000", maxTaskMicroUsd: "3000000", period: "month" },
});

Two things are specific to it:

immediate only. It routes across the aggregated network, so it never runs in the completion-window pool. priority or loose with vetta/auto is refused with window_unavailable, exactly like any other unsupported pair.
It is priced as a ceiling, then settled at the model that answered. Because the model is not known until the call is routed, there is no rate card to quote from. So the request carries a hard price cap — $5 per million input tokens and $25 per million output tokens — which the network enforces: a call it cannot serve inside the cap is refused rather than routed to something dearer. The pre-flight hold is taken at exactly those cap rates, so it is a genuine upper bound. The debit is then settled at the rate of the model that actually answered, which is normally well below the cap. You are never billed above the ceiling you were quoted.

vetta/auto selects a different model for different requests by design. Pin a specific id instead when a run has to be reproducible, or when a prompt is tuned to one model’s behaviour.

Calling the router directly

You do not need an agent to use a model. The model proxy is the router with nothing in front of it: one request in, one reply out, billed to the organization. It exists for the call a session is the wrong shape for — a classifier in your own pipeline, a one-off summary, an existing client you would rather repoint than rewrite. Two doors, one router:

Door	Route	Format
Messages	`POST /v1/proxy/anthropic/v1/messages`	The Messages request and response, `system` and `tools` included, streamed or not.
Chat Completions	`POST /v1/proxy/openai/v1/chat/completions`	The Chat Completions envelope — `choices`, `finish_reason`, `usage`, `data:` chunks when streamed.

Any id the catalogue publishes goes in model, vetta/auto included; the window travels as the Vetta-Window header because neither format has a field for it, and unset means immediate.

const reply = await vetta.proxy.messages(
  { model: "vetta/auto", max_tokens: 512, messages: [{ role: "user", content: "Is this ticket urgent? …" }] },
);

const chat = await vetta.proxy.completions(
  { model: "zai-org/GLM-5.2-FP8", messages: [{ role: "user", content: "Summarise this invoice." }] },
  "priority",
);

vetta proxy message --model vetta/auto --text "Is this ticket urgent? …"
vetta proxy url        # the base URL to point an existing Messages-format client at

curl -fsSL https://api.vetta.sh/v1/proxy/openai/v1/chat/completions \
  -H "authorization: Bearer sk_live_..." \
  -H "vetta-window: priority" \
  -H "content-type: application/json" \
  -d '{ "model": "zai-org/GLM-5.2-FP8", "messages": [{ "role": "user", "content": "Summarise this invoice." }] }'

What a direct call shares with an agent’s, and what it does not:

	Agent turn	Direct call
Catalogue, lanes, `window_unavailable`	same	same
Pre-flight quote → stream → settle at actual usage	same	same
Who pays	the agent’s budget, then the org	the org balance directly — `insufficient_credits` when the quote does not fit
Ledger entry	carries `session_id` and `agent_id`	carries the request id (`x-request-id`) and neither
Tools	run by the harness inside the sandbox	returned to you as `tool_use` / `tool_calls`; you run them and send the result back
State between calls	the session’s durable log	none — send the whole conversation every time
Scope	`sessions:write`	`proxy:write` (or `sessions:write`), see authentication

A direct call carries text, tools and streaming — and refuses, as validation_failed, what it cannot carry: sampling and decoding controls (top_p, top_k, stop, seed, n, logprobs, penalties), structured output (response_format), tool_choice, extended-thinking blocks, and image or document content parts. Nothing is dropped on the floor; a setting the router will not honour is a call you should not pay for. The per-field detail is on the proxy reference.

Choosing which model answers

vetta/auto chooses inside the call, from the whole network. There is also an optional add-on that chooses from your shortlist, with your descriptions: name two to sixteen catalogue ids, say in a line what each is for, and an evaluation model reads the conversation and says which one should answer it. It comes in two shapes. The first hands you the id, so you can log it or branch on it:

const pick = await vetta.models.autoSelect({
  options: {
    "anthropic/claude-sonnet-4.5": "Routine coding, edits and tool use",
    "openai/gpt-5.6-sol": "Hard multi-step reasoning or architecture",
    "google/gemini-3-flash": "Short factual answers where speed matters",
  },
  messages,
  default: "anthropic/claude-sonnet-4.5",
  min_confidence: 0.5,
});
// pick.model is a concrete catalogue id — pass it as `model` on the next call.

The second is the same policy riding on the call itself, so the router makes the pick and the model call in one step — per call on either proxy door, or per turn on an agent:

const reply = await vetta.proxy.messages({
  model: "anthropic/claude-sonnet-4.5",              // the default; must be one of the options
  model_selection: { options: { /* as above */ }, min_confidence: 0.5 },
  max_tokens: 1024,
  messages,
});
reply.model; // the id that answered

Everything after the pick is the ordinary path above: the chosen id is resolved, quoted from its rate card for the request’s window, admitted, routed and settled. It is an add-on, not a change to routing — vetta/auto, pinned ids, windows and the catalogue behave exactly as described on this page whether or not you use it. Full detail — the option rules, what default and min_confidence actually do, what a selection costs and adds in latency, and how it behaves when the evaluator is unavailable — is on Model auto-selection.

What a model call costs

Model spend — an agent’s or a direct call’s — is metered per token across a five-tier ledger: each class of token is priced separately because the backends bill them separately. The request’s completion window tariff sets the rate the call is quoted and reserved at; where the backend reports what the completion actually cost, that figure is what you are billed from, so the amount tracks the provider that served the call rather than a list price for the model id.

Token tier	Field	What it is
Input	`input`	Fresh prompt tokens sent to the model.
Cache write	`cache_write`	Prompt tokens written into the provider’s prompt cache.
Cache read	`cache_read`	Prompt tokens served from cache — billed at a fraction of the input rate.
Output	`output`	Tokens the model generates.
Reasoning	`reasoning`	Internal thinking tokens, when the model and `effort` produce them.

These five fields are the canonical token ledger: every inference debit records all five, and they are the components a session’s token_usage and the billing line items are built from.

Cache-read tokens are the reason list-price rate cards mislead. A card that bills every input token at the full input rate over-states real cost by 1.017×–4.176× depending on harness and window (see Benchmarks). Vetta meters each tier at its real rate, so your bill tracks what the backend actually charged — not a rate-card fiction.

Because every call is priced against these five tiers before it runs, a call that would breach your budget — or, for a direct call, your balance — is refused rather than discovered on an invoice. Model spend shows up as the model line item in the agent’s spend breakdown:

CLI

vetta agent spend Refunder --by component

{ "by_component": { "model": 11902000, "computer": 481000, "search": 6474, "media": 390000 } }

Amounts are integer micro-USD (11902000 is $11.902). The model component is the sum of the five token tiers above; computer, search and media are the other components a debit can carry, and a component with no spend is absent rather than zero.

Effort

Some models accept an effort level that trades latency and reasoning-token spend for quality. Effort is independent of the completion window: the window sets the tariff and latency lane, effort sets how hard the model thinks within it. Which levels a model accepts is published per model in efforts, read from what the model itself advertises and narrowed to the three wire values. An empty efforts means the model takes no effort setting at all — most do not — so check the entry before you pin one.

TypeScript

const agent = await vetta.agents.create({
  name: "Deep",
  model: { id: "zai-org/GLM-5.2-FP8", effort: "high" },
  window: "priority",
  budget: { capMicroUsd: "25000000", maxTaskMicroUsd: "3000000", period: "month" }, // $25 / $3
});

Configuration reference

string | object

required

A model ID (e.g. zai-org/GLM-5.2-FP8, or vetta/auto to pick per request) or an object { id, effort }. Set on the agent and overridable per session. Any id GET /v1/models publishes is legal; anything else is validation_failed.

Show model object

string

required

The model identifier.

string

low, medium, or high, for models that support it.

string

default:"immediate"

The completion window. immediate routes to the aggregated network; priority and loose route to the completion-window pool and require a window-supported model.

Read-only model fields

integer

Maximum context length in tokens.

integer

The longest reply this model may produce, and what the pre-flight quote is bounded by.

string[]

Which windows the model can run in. immediate is always present; priority/loose appear only for pool-hosted models.

string[]

Effort levels the model accepts — low, medium, high, or empty when it accepts none.

Next: completion window

The three-price model, and why you choose it per request.

Agent

Running an agent

Agent capabilities

Computer

Identity

Team

Model router & inference

Two inference lanes

The routing rule

The catalogue is live

Discovering models

`max_output_tokens`

Vetta Auto

Calling the router directly

Choosing which model answers

What a model call costs

Effort

Configuration reference

Read-only model fields

Next: completion window

​Two inference lanes

​The routing rule

​The catalogue is live

​Discovering models

​max_output_tokens

​Vetta Auto

​Calling the router directly

​Choosing which model answers

​What a model call costs

​Effort

​Configuration reference

​Read-only model fields

Next: completion window

Two inference lanes

The routing rule

The catalogue is live

Discovering models

`max_output_tokens`

Vetta Auto

Calling the router directly

Choosing which model answers

What a model call costs

Effort

Configuration reference

Read-only model fields