Kubernetes

Share a GPU with MIG and Kubernetes DRA.

NVIDIA MIG slices a GPU. Kubernetes Dynamic Resource Allocation requests the slice. denv uses both so a small model need not take a whole H100.

A small model on a whole H100 is a quiet way to waste money. NVIDIA MIG (Multi-Instance GPU) partitions one physical GPU into isolated instances. Each instance has its own memory and compute slice. Kubernetes DRA, Dynamic Resource Allocation, is how a pod asks for one of those slices instead of the only older choice: a whole GPU.

denv uses both. A recipe can ask for a full GPU or a share. The cluster places it. You do not write a separate scheduler per cloud.

What this is for

  • A few small models on one large GPU: an 8B chat model, a draft model next to a bigger one, a side evaluation.
  • Teams that already pay for H100, H200 or B200 and do not have a full card of traffic for every model.
  • The same recipe catalog. The engine is still vLLM or SGLang. The endpoint is still OpenAI-compatible.

MIG is isolation, not a free lunch. A slice has less memory and less bandwidth than the card. A 70B model that needs the card should take the card. The recipe is where that choice is pinned, per GPU type.

What denv does not pretend

This page does not publish MIG profiles, DRA resource class names, or a tok/s number for a slice. Those belong in the recipe you actually deploy, tested on that GPU type. If you need the profile list for a cluster you are about to run, write to anup@denv.ai. The product is early, and a profile we have not tested is not a profile we will imply.

Time-slicing, the older way of sharing a GPU by taking turns, is a different tool. It does not isolate memory. MIG does. For inference, isolation is the point: one model’s KV cache should not be able to knock the neighbor over.

Drivers

GPU driver management sits next to sharing. A node joins with the driver installed. Auto-repair can reinstall that driver. MIG and DRA only help if the driver and the Kubernetes device stack on the node match what the recipe expects. denv treats that as part of bringing the node up, not as a wiki page you maintain per provider.

See also H100, where a whole card is often more than an 8B model needs, and the concepts page.