Use cases

Voice needs a GPU that is already awake.

Speech workloads care about a warm GPU and a dead node not dropping the next turn. denv has not published a voice latency number.

A voice product loses the conversation if the first token waits on a cold start. The model has to be loaded, and a replacement GPU has to exist before the one you are on dies.

What we have measured, and what we have not

We have not published a time-to-first-token for speech, and we have not published a tokens-per-second figure for a voice model. Those numbers are not on this site. Do not reuse the text benchmarks as if they were.

The only published first-token figures are for Qwen3.8-27B text, one request at a time:

  • 0.61 seconds in the 23-way test. The H100 was in Europe and the laptop was in the US.
  • 221 ms median, 234 ms at the 90th percentile, in the earlier H100 run. That endpoint was called directly. The other providers in that run were reached through OpenRouter, which adds a hop to their first token.

A cold start on a fresh H100 in that same work took 12 minutes 46 seconds. With weights and compiled kernels already on the machine, a restart took 99 seconds. That is why serverless is the wrong default for a live call. The GPU is billing you while it is up. That is the trade.

What denv does about it

Keep a warm replica. The recipe sets a minimum. Health checks take a bad GPU out. Failover sends the next request to a healthy replica, including one at another provider, if you allowed that site. A response that already started streaming is not silently replayed somewhere else. The caller hears an error and retries.

The GPUs are yours: Lambda, Nebius, AWS, Hetzner, or a machine you join. One cluster, one endpoint. Adding capacity is adding a site, not a second platform. See multi-cloud.

GPU driver install happens when the node joins. Repair can reinstall the driver. A voice replica that needs a whole GPU should take the whole GPU. A small side model can take a MIG slice. We have not published how a voice model behaves on a slice.

The product is early. If you want a speech model tested on a GPU type you already have, write to anup@denv.ai. Until that recipe exists, this page will not invent its latency.