Docs · reference

Hardware

Hardware ids, scale-up domains, and what denv detects on each host.

Recipes name the hardware they are tested on. Fleets report the hardware their hosts have. denv places a release only where the two match.

Hardware ids

Id Vendor Scale-up domain Typical sources
h100 NVIDIA One 8-GPU NVLink server Hyperscalers, GPU clouds, own servers
h200 NVIDIA One 8-GPU NVLink server Hyperscalers, GPU clouds, own servers
b200 NVIDIA One 8-GPU NVLink server Hyperscalers, GPU clouds, own servers
b300 NVIDIA One 8-GPU NVLink server GPU clouds, own servers
gb200-nvl72 NVIDIA One 72-GPU rack GPU clouds, own racks
gb300-nvl72 NVIDIA One 72-GPU rack GPU clouds, own racks
mi300x AMD One 8-GPU server GPU clouds, own servers
mi355x AMD One 8-GPU server GPU clouds, own servers
ascend-950 Huawei Per the vendor's system design Own racks
l40s NVIDIA One server; GPUs connected over PCIe Hyperscalers, GPU clouds, own servers
cpu Any One host Anywhere; for embeddings and small models

denv offers --hardware <id> lists where you can get each one across your connected providers.

Scale-up domains

A scale-up domain is a set of GPUs joined by a fast fabric (NVLink and NVSwitch, or the vendor's equivalent), so that tensor, expert and data parallelism and KV transfer can run between them at high bandwidth.

  • On an 8-GPU server, the domain is the server.
  • On an NVL72 rack, the domain is the rack: 72 GPUs across 18 compute trays, each tray a host to denv. denvd groups the trays into one domain by their shared NVLink fabric identity.

Domains have ids such as gb200-rack/d1. A degraded domain (a tray down, a link failing) reports fewer usable GPUs and refuses releases whose units need the full domain.

Large disaggregated releases use topology.placement: same-domain, which keeps a serving unit's prefill and decode pools inside one domain. DeepInfer's published benchmarks, for example, used 4 GPUs for prefill and 64 for decode, each within one scale-up domain. A fleet reports how many domains it has and how many GPUs are free in each, so denv can tell whether a unit fits before it starts a deployment.

See Serving topology.

What denv detects

When a host joins, denvd reports:

Detected Used for
GPU model and count Matching recipes' hardware.
Driver and runtime versions Choosing engine images that run on the host.
Fabric membership (NVLink/NVSwitch domain id, peers) Grouping hosts into scale-up domains.
Network interfaces suitable for KV transfer between hosts Disaggregated units that span hosts in a domain.
Local NVMe Weight cache and the nvme KV cache tier.
CPU and memory The host-memory KV cache tier and CPU-only models.

Detection is repeated after reboots and driver changes. A host whose hardware no longer matches its fleet is taken out of rotation and reported.

Where to get hardware

One stage can serve from several of these at once. Use per-fleet recipes on the stage when the hardware differs.