Fallbacks
Name the models to try when a provider is overloaded, and nothing else.
Naming alternatives
Add models to try alternatives in order. There is no implicit fallback: we will never spend your credit on a model you did not name.
{ "model": "anthropic/claude-sonnet-5", "models": ["openai/gpt-4o", "openai/gpt-4o-mini"], "messages": [{"role": "user", "content": "Hello"}] }
Check x-vatan-model on the response to see which one answered, and filter your request log by route to see how often it happens.
Which failures are retried
Only failures that would not repeat elsewhere: a 429 or a 5xx. Never a 400, a 401 or an unknown model, because those fail identically at every provider and retrying is pure latency and pure cost.
429 is included deliberately. Rate limits are per provider, so the whole point of holding a second one is having somewhere to go when the first is saturated.
Streaming
On a streamed request, fallback stays available until the first token reaches you and stops permanently after it. Splicing a second model's continuation onto the first model's opening produces text that no model wrote, and failing honestly beats inventing a reply.