denv is not serverless GPU inference.

Serverless GPU means the platform starts a GPU when a request arrives and tries not to bill you when it does not. RunPod, Modal, Beam and the hyperscalers all have a version of this. Search sends a lot of “just give me an endpoint” traffic there. Some of that traffic should stay there.

denv does not scale to zero. A model replica holds weights in GPU memory. Loading them, and compiling kernels, is slow. One fresh H100 start we measured took 12 minutes 46 seconds. A restart with the cache warm took 99 seconds. Those numbers are in the 213 tok/s post. An endpoint that has to answer interactive traffic cannot pay that on the first request of the afternoon.

Use serverless if

Requests are rare, latency on the first call can be bad, or the work is a batch job that can wait. You will spend less on idle GPUs. denv’s pricing, a fee per GPU under management plus the provider’s bill, assumes the GPU is in the cluster.

Use denv if

The endpoint has to be warm, the GPUs are in accounts you already have, and a dead GPU or a dead site should fail over instead of cold-starting somewhere new. Health checks and auto-repair run while the replica is up. They are useless if the replica does not exist yet.

You can still use a hosted API as overflow. The inference guide says that directly: keep an API as the safety valve, and put steady traffic on GPUs you run. denv is the second half. Voice is the clearest case: we have not published a speech latency, and a cold start measured on one fresh H100 was 12 minutes 46 seconds. It is early, and it is not a serverless bill.