What a model pool is
Many models in the catalog are served by more than one provider. Asking for a model by its bare name —gpt-oss-120b rather than cerebras/gpt-oss-120b —
reaches the whole pool of providers that serve it. Opper tries one of them
first and falls over to the rest if it fails.
With no routing rule, the pool is tried in round robin: every call goes
to the next provider in turn, which spreads load evenly but ignores everything
the providers differ on — price, time to first token, throughput. A routing
rule states the preference once, and every pooled call in its scope follows
it.
Pinned calls are unaffected. A call that names a specific provider’s model
gets exactly that model.
Choose an order
Price is published, so every provider ranks on it from the start. Latency
and throughput are measured from recent traffic across Opper, not just
yours, so a newly added provider has no number until it has served some
calls; it sorts last until then and stays in the pool for failover.
Set the weights
The three bars are always drawn, and each order chip is a preset for them: Cheapest first is cost at 100 and nothing else. Drag a second bar up and the order becomes Weighted by itself; a blend that collapses to one bar reads as that metric’s preset again. Only the ratios matter. Each metric is scaled against a typical value first — a dollar of price, a second of latency and a rate of throughput are not comparable otherwise — so “cost 50, latency 25” means what it says: price counts twice as much as latency, and throughput does not count at all. A weight of zero drops the metric from the blend entirely. A provider with no latency number still ranks on a blend that only weighs cost.Override the order for a project
Select Override for a project to add an override. A project’s own order replaces the organization’s, so a project can choose round robin to switch an inherited order off, or pick a different order for a different priority.Where you see the result
Every pooled call’s response reports how it was routed in theopper.routing metadata: served names the provider and model that ran,
pool_size how many could have, and pool_rank where the served one
stood in the order Opper computed for that call. pool_source says what
decided the head — rule, affinity, or round_robin — and pool_order
names the rule’s order when one applied.
A conversation that has already been served by one provider keeps going to
it while its prompt cache is warm, which outranks the routing rule. A rule
change therefore shows up on new conversations first; an existing one
follows within an hour.