Failover when a GPU or a provider stops answering.

Failover means a request that cannot be served where it landed is served somewhere that can. On denv that somewhere can be another replica at the same provider, or a replica at a different one. One endpoint either way.

The word is the behavior. Traffic moves to healthy replicas. The concepts page describes the objects. This page is the operator view.

What is checked

Health checks watch the signals that actually take GPUs down:

• XID errors, including XID 79, “GPU fell off the bus”
• ECC and NVLink errors
• Thermal throttling
• Silent nodes and GPUs that get slow while still looking up

A failed check cordons the node. It takes no new work. Running replicas drain inside the recipe’s disruption budget, so a repair cannot take every replica at once.

Where the request goes

1. Prefer healthy replicas in the same site. That keeps the prefix cache.
2. If the site cannot serve, send the request to another site you allowed on the environment.
3. Do not retry a response that already started streaming. The caller gets an error and retries. A silent retry would be a different answer.

That second step is multi-provider failover. It only works because the replicas are in one cluster, not in a pile of clusters you have to federate yourself.

What failover is not

It is not a second copy of your weights in a vendor’s token API. It is not serverless cold start. The replicas have to exist. A warm minimum at more than one site is what makes the failover fast. Scaling from an empty cache at a new provider takes minutes, because of image pull, weights and compile. The 213 tok/s note records one cold start: 12 minutes 46 seconds down to 99 seconds once the cache is warm.

Failover also does not replace the machine. That is auto-repair. Failover keeps the endpoint answering while repair happens.

denv does not sell or resell GPUs. You pay the provider that still has healthy replicas. The product is early.