Deploy Llama 3.1 8B on an H100.

Llama 3.1 8B Instruct is the model name in the homepage code sample: llama-3.1-8b-instruct. That sample is how you call an endpoint. It is not a speed test.

How a deploy works

You pick the recipe for this model on H100, attach a site that has H100s, and set replica bounds. denv caches the weights at the site, starts the replicas, and waits for health checks. The endpoint is OpenAI-compatible. The call in the sample points base_url at your cluster and model at the recipe’s name.

H100 is available through Lambda, Nebius, EC2, or a server you join. You pay that provider. denv does not.

Fit

An 8B model does not need a full H100. Putting it alone on the card works, and it is wasteful if you have other small models. NVIDIA MIG and Kubernetes DRA are how a recipe takes a slice instead. The slice has less memory bandwidth than the card. Do not expect the H100 Qwen numbers on an 8B slice. Those numbers are a different model.

What is not published

No tokens-per-second figure for Llama 3.1 8B on H100 appears in the public tests. The tests were Qwen3.8-27B. Using 213 tok/s here would be invented. The engine is vLLM or SGLang, whichever the tested recipe pins.

Repair and failover apply once you run more than a single replica, or a second site. The product is early. Write to anup@denv.ai if this recipe is not in the catalog yet.