One Kubernetes cluster across many GPU clouds.

Most teams serving models do not have a single GPU problem. They have GPUs in more than one place: a hyperscaler, a neocloud, maybe a rack they already own. Each place has its own API, its own capacity this week, and its own way of failing.

The usual fix is a Kubernetes cluster per provider, plus glue. The glue is where networking, scheduling and 3am repairs go to die.

denv takes the other shape: one cluster, many sites. Adapted from Anup’s note on X.

Why inference can stretch and training usually cannot

A training job wants every GPU on one fabric. An inference replica does not. One replica is one model on the GPUs it needs, inside one site. The request lands on one replica and finishes there. Replicas do not talk to each other.

So the rule is small:

• Keep each replica inside one site, on that site’s network.
• Let replicas live at any site you allow.
• Send a request across sites only to choose a site, or to fail over.

Pods of one replica stay in one site. Traffic to a model can span many. That is already how a site works.

How a node joins

Each machine dials out to the control plane. No inbound ports and no VPN. That is what lets a Hetzner box, a colo server, or a cloud VM join the same cluster. The catalog today is Lambda, Nebius, AWS, Hetzner and your own servers. Aggregators are marked coming.

GPU clouds become providers on denv. Adding one is adding a site, not a second platform.

Routing

A prefix cache is why “spray traffic evenly” is a bad default. The replica that saw the last turn of a long prompt can skip most of the prefill. Sending the next turn to another cloud throws that away.

denv’s standard serving profile uses llm-d, which routes with cache and load in mind. See llm-d. Globally, a request prefers a healthy site and sticks when it can. It moves when that site cannot answer. Mid-stream, a request that already started tokens is not silently replayed somewhere else. The caller sees an error and retries. That is a product choice, written down in the failover page.

When a site dies

Health checks are per GPU, not per cloud. Repair handles a bad card. If many nodes at one site fail together, that is a provider incident: stop repairing them one by one and fail over. “No capacity”, “over quota” and “bad credentials” are different errors. The first means try the next provider. The second needs a person. The third means stop touching that account.

When not to do this

If every GPU you have is in one cloud and you already run that cloud’s Kubernetes well, a second control plane is extra parts. If you are training, keep the job on one fabric. If one replica must span providers over the network, this design is the wrong tool. If data cannot leave a region, the site is a compliance boundary first and a capacity pool second.

denv does not sell the GPUs. You pay each provider directly. The product is early. Write to anup@denv.ai if you already run in more than one place and want to see what is live.