One H100, 213 tokens a second: faster than every GPU provider on OpenRouter

Also on YouTube.

This morning we recorded our first end-to-end demo: Qwen3.8-27B on one H100 in a Nebius account, deployed by denv, called with the OpenAI client. It answered at 50.8 tokens a second.

So we opened OpenRouter, looked at who serves the same model fastest, and set out to beat them on that same kind of GPU. By the evening the H100 was doing 213 tokens a second, faster than every GPU provider we raced.

The result

Qwen3.8-27B, 24 requests per target, one request at a time, measured back to back on October 29, 2026.

Provider	Hardware	Output speed, median	10th percentile	First token, median	90th percentile
Cerebras	wafer-scale chip	1,092 tok/s	781 tok/s	227 ms	689 ms
denv	1× H100, FP8 + DFlash2	213 tok/s	139 tok/s	221 ms	234 ms
ModelRun (Modular)	GPU, fp4	191 tok/s	84 tok/s	393 ms	1,103 ms
Modal	GPU	125 tok/s	89 tok/s	394 ms	653 ms
Parasail	GPU, fp8	73 tok/s	43 tok/s	509 ms	1,589 ms

Cerebras is in a different class: a wafer-scale chip, not a GPU. Among GPU providers, one H100 running denv's recipe was the fastest at the median. Its first token was also the quickest we measured, and the steadiest: 234 ms at the 90th percentile (the providers' figures include one extra network hop; see below).

Where we started

OpenRouter lists 21 providers for Qwen3.8-27B and publishes each one's speed on live traffic. When we looked, the leaders were Cerebras (532 tok/s), ModelRun (142) and Modal (125). Our demo's 50.8 tok/s put us around twelfth.

The reason is simple arithmetic. With one request in flight, every new token reads all of the model's weights from GPU memory once. Qwen3.8-27B in bf16 is 55.6 GB, and an H100 reads about 3.35 TB/s, so the ceiling is roughly 60 tokens a second. We were already at 82% of it. Going faster meant changing what each step reads, or how many tokens each step produces.

Four recipes, one GPU type

Each line is a denv recipe deployed to a 1× H100 on Nebius and raced with the same script.

Recipe	Output speed, median	What changed
bf16, no speculation	50.8 tok/s	this morning's demo (one call)
FP8 + MTP, 3 draft tokens	164 tok/s	Qwen's official FP8 checkpoint halves the bytes; the model's own multi-token-prediction head drafts 3 tokens a step
INT4 + MTP, 3 draft tokens	179 tok/s	a 4-bit weight build reads less again, at some cost in quality
FP8 + DFlash2, 7 draft tokens	213 tok/s	an external block-diffusion drafter proposes 7 tokens a step; more of them are accepted

The winner is not the 4-bit build. Speculative decoding verifies every drafted token against the target model, so the output is the FP8 model's own output: the drafter changes the speed, not the answer.

The recipe

Any team can create the same recipe and deploy it to their own cloud:

denv recipe create qwen3.8-27b/fp8-dflash2 \
  --model Qwen/Qwen3.8-27B-FP8 --engine vllm --max-model-len 32768 \
  --arg --max-num-seqs=256 \
  --arg '--speculative-config={"method":"dflash","model":"incoai/Qwen3.8-27B-DFlash2","revision":"015e795645c74b1a0eeef3b570031fb62e769bc5","num_speculative_tokens":7}'

denv pins it at creation: the weights to commit 017b9c7, the drafter to its revision, and vLLM 0.32.0 to its image digest. A deploy is accepted on a GPU type only after the recipe has come up healthy there.

On a fresh machine the first start took 12 minutes 46 seconds (image 3m43s, weights 3m56s, engine 5m07s). With the weights already on the machine, a restart took 99 seconds.

How we measured

• The same six prompts (code, writing, a summary, a comparison, an email), four rounds: 24 requests per target.
• One request at a time, up to 512 tokens, thinking off, temperature 0.7 and top-p 0.8 for everyone.
• Each provider pinned through OpenRouter's provider routing, with fallbacks off.
• Output speed is the completion tokens after the first, divided by the time from the first token to the last. Time to first token is measured at the client.
• Everything ran from one laptop in one session, back to back.

The script is about a hundred lines of Python with the OpenAI client: race.py. The results it wrote are in race-final.json. Run it against any endpoint you like.

What this does not show

• Concurrency. We sent one request at a time to a GPU that served only us. The providers were serving other customers at the same time. Per-request speed on a dedicated GPU drops as you add concurrent requests; we did not measure that here.
• Price per token. At one request at a time, a $4.50/hour H100 producing 213 tok/s costs about $5.87 per million output tokens, more than ModelRun's $2.50. A dedicated GPU pays for itself with steady traffic, or when the data has to stay in your own account.
• The network. Providers were reached through OpenRouter, which adds a hop to their time to first token but not to their output speed. Our endpoint was called directly.
• Precision differs. ModelRun serves fp4, Parasail fp8, Cerebras fp16; Modal does not say. We served FP8.
• One day, one place, 24 requests each. Speculative decoding is faster on predictable text such as code than on open-ended prose, so other prompts will move the numbers.

Try it

denv deploys any open model, on any engine, to GPUs in your own cloud account, and you pay your cloud directly. If you want this recipe on your GPUs, book a call or start at denv.ai.