Engines

vLLM hosting, without babysitting the cluster.

denv runs vLLM from a tested recipe on your GPUs: llm-d routing, dashboards, health checks and repair. You pay the GPU provider.

vLLM is the engine most of these recipes start from. denv does not fork it into a private server. A recipe pins a vLLM version by image digest, plus the precision, the arguments, and the serving profile for that GPU type.

You still get an OpenAI-compatible endpoint. You do not get a second bill for tokens. The GPU bill is yours.

What denv adds on top of the engine

  • A recipe tested on the GPU type before it serves. The 50.8 → 213 tok/s run was vLLM on one H100, pinned to a digest, with FP8 weights and a draft model.
  • llm-d when the profile is more than a single replica: cache-aware routing, and prefill/decode split for large models.
  • Dashboards from the first request: time to first token, throughput, queue depth, KV-cache use, GPU health.
  • Health checks and auto-repair, including GPU driver reinstall.
  • Failover across providers in the same cluster.

vLLM on Kubernetes

“vLLM on Kubernetes” is the default here, not a side project. The cluster is real Kubernetes. You can kubectl it. The engine is a workload on that cluster, with a disruption budget so a node repair does not take every replica.

Cold start is part of running vLLM, not a footnote. One measured start on a fresh H100 took 12 minutes 46 seconds, and 99 seconds once weights and compiled kernels were cached. That number is from the same H100 work, written up in the throughput post.

SGLang vs vLLM

SGLang is the other engine in the catalog. Use vLLM when you want the widest model coverage and the path those public H100 numbers were measured on. Use SGLang when the recipe for that model on that GPU is the SGLang one, often for prefix-heavy or structured generation. denv does not make you pick a company. The recipe picks the engine. Both sit behind the same endpoint shape.

What we will not claim

There is no published tok/s for every model on every GPU. The public figures are the Qwen3.8-27B runs on one H100. A recipe page says so when a number exists, and stays quiet when it does not.

denv is early. To deploy vLLM on a GPU you already have, write to anup@denv.ai or follow the quickstart as it rolls out.