Pool node

A pool node holds several models and decides, on every request, which one to try first. The first model that answers wins; if all of them fail, the route takes the pool’s fallback edge. A model node with fallbacks is a fixed order you wrote down when you built the route. A pool re-ranks on numbers that track what providers are doing right now.

​

Add a pool

Click the + on an edge and choose Add Pool. A pool can go at the end of a path, in an empty branch, or on another node’s fallback edge — anywhere an answer is expected. A new pool has two rows to fill, because two is the smallest set that can be ordered; Add model takes it up to ten. Each row is a model picker that offers individual provider endpoints only — groq/gpt-oss-120b, not the bare gpt-oss-120b — since a bare name is already a pool of its own, and a pool ranks single providers. A model another row already holds is not offered again.

​

Choose the order

The three bars under the chips are the weights, and each chip is a preset for them: Cheapest first is cost at 100 and nothing else. Drag a second bar up and the order becomes Weighted by itself. Only the ratios matter. Each metric is scaled against a typical value first, so “cost 50, latency 25” means price counts twice as much as latency. A weight of zero drops that metric from the blend, so a model with no latency number still ranks on a blend that only weighs cost. Price is published, so every model ranks on it from the start. Latency and throughput are measured from recent traffic across Opper, not just yours; a model that has seen little of it has no number yet and sorts last until it does. A pool that is new to you ranks fully on Cheapest first from day one.

​

The score beside each model

Each row shows the model’s score for the chosen order right now: the sort key evaluated for that model, lower tried first, from the same live numbers a request uses. The rows stay in the order you added them — the score tells you the order the runtime will use. A model the order cannot score reads no score and sorts last. Hover the chip for the numbers behind it: list price, time to first token, throughput and how many calls were measured.

​

Failover

The ranking is a preference, not a filter. A model that sorts last is still tried when everything ahead of it fails. When every model in the pool has failed, the route takes the pool’s fallback edge — a fallback model, or End, in which case the caller gets the last error. The trace of each call records the order the pool tried, and the response’s meta.routing names the model that answered.

​

The same order as a routing rule

The Routing rule applies this ordering to the pool behind a bare model name — every provider serving gpt-oss-120b, with no graph involved. A pool node applies it to the models you list. Both use the same vocabulary and the same ranking, so a rule and a pool that ask for the same order rank identically on identical numbers.