Deploy Llama 3.1 70B on H100.

Llama 3.1 70B is the size people mean when they say they have outgrown a single small GPU. denv deploys it as a recipe on H100: engine, precision, and how many GPUs that build needs, tested together.

Memory, before anything else

Weights are about parameters times bytes. At 16-bit that is about 140 GB for a 70B model, before KV cache and runtime overhead. At 8-bit it is about 70 GB. One H100 does not hold 140 GB of weights. Whether one H100 holds the 8-bit build depends on the KV cache you also have to leave room for. That rule of thumb is from the inference guide. The recipe is the tested answer for a specific checkpoint. This page will not guess a GPU count beyond that.

Where it runs

H100 on Lambda, Nebius, EC2, or your own server. If the recipe uses more than one GPU, those GPUs stay in one site. A replica does not stretch across providers. A second site is for another replica and for failover, not for one replica’s tensor parallel group.

Speed

Nothing on denv.ai measures Llama 3.1 70B in tokens per second. The public H100 results are Qwen3.8-27B only. Quote those if you are talking about that model. Do not quote them here.

vLLM is the engine those Qwen runs used. A 70B recipe may pin vLLM or SGLang. Driver install and auto-repair are part of the node, not of the model card.

The product is early. If you need this pair, write to anup@denv.ai.