Choose a model and GPU

Every benchmark result is a recipe measured on one workload: a model, an engine version, a GPU and a topology. Choosing a setup means choosing a result, then checking it fits the GPUs you can get.

1. Rank results for your workload

denv bench --workload agent --latency p99<1.5s

#  MODEL                GPU          RECIPE                           TTFT P99  TOK/S  TOK/S/GPU  $/M OUT
1  deepseek-v4.1-flash  gb200-nvl72  deepseek-v4.1-flash/deepinfer@3  …         …      …          …
2  deepseek-v4.1-flash  h200         deepseek-v4.1-flash/sglang@2     …         …      …          …
3  glm-5.3              mi355x       glm-5.3/vllm@2                   …         …      …          …

Flag	Use it to
--workload	Match your traffic shape: chat, agent, rag, coding or batch.
--latency	Set the target every ranked result must meet, such as p99<1.5s for time to first token.
--model	Compare setups for one model family, or leave it out to compare models.
--hardware	Limit to GPUs you have or can get, such as --hardware h100.
--rank	cost (default), throughput or speed.

Results that miss the latency target are listed after the ranked ones, marked. Benchmarks measure speed, throughput and cost only; how they are computed is in How benchmarks work.

2. Read the result

denv bench get 1

Check:

Field	Question to ask
speed	Does p99 time to first token and output speed hold at the concurrency you expect?
throughput	How many GPUs will your peak load need at this throughput per GPU?
cost	Which GPU price did it assume? On your own hardware or reserved capacity, yours is different.
recipe	Which engine version, topology and arguments produced the numbers?

3. Check it fits your fleets

denv fleet list

FLEET        PROVIDER  HARDWARE      DOMAIN   FREE GPUS
gb200-rack   own       gb200-nvl72   72       144
h200-use1    aws       h200          8        32

The HARDWARE column is what recipes match on (Hardware). A recipe's topology must fit your domain size: a 64-GPU decode pool needs a 72-GPU domain, not an 8-GPU server. If you have no fleet with that hardware, denv offers --hardware <id> shows where you can get it.

4. Confirm on your own GPUs

Published results use public GPU prices and standard workload shapes. Re-run the top candidates on your fleet, with your price and your traffic shape:

denv bench run --recipe deepseek-v4.1-flash/sglang@2 --target h200-use1 --traffic shape.jsonl

5. Deploy it

denv deploy --from-bench 1 --target gb200-rack

Or reference the recipe in a release you manage in denv.yaml:

release:
  model: ds-flash
  weights: hf://deepseek-ai/DeepSeek-V4.1-Flash-1017@c41a7e2
  recipe: deepseek-v4.1-flash/deepinfer@3

Override a few fields

If one or two things differ, such as a newer engine patch or more prefill replicas, override them in the release. The recipe stays the same; the release records the difference.

release:
  recipe: deepseek-v4.1-flash/deepinfer@3
  overrides:
    engine: { version: 0.1.3 }
  topology:
    mode: disaggregated
    prefill: { gpus: 4, replicas: 3 }
    decode: { gpus: 64, replicas: 1 }

An override changes what was measured, so benchmark the release on your fleet before relying on the published numbers.

Serve the same weights on different hardware

A stage can serve from fleets with different hardware by giving each fleet its own recipe, for example the top result on your racks and the second on rented H200s:

stages:
  - name: prod-us
    fleets: [gb200-rack, h200-use1]
    recipes: { h200-use1: deepseek-v4.1-flash/sglang@2 }

Two engines serving the same weights should give the same answers; parity checks confirm it at deploy time.

When to write your own recipe

• No result covers your hardware or your model.
• You serve your own fine-tune with a different chat template.
• You have tuned arguments you want to reuse across releases.

See Write a recipe, then benchmark it with denv bench run. For what each engine supports, see Engines.