Document work is long context, then a queue.

Document work is a long prompt: a contract, a filing, a pile of pages. The KV cache grows with that prompt and with how many of them you run at once. It often runs out of GPU memory before the card runs out of compute. Batch jobs can wait. A person waiting on one document cannot.

We have not published tokens per second, time to first token, or cost per document for this workload. The H100 figures on the benchmark and the 213 tok/s note are short Qwen prompts, one request at a time. Using them here would be a guess. The memory rule of thumb in the inference guide is the part that does transfer: weights are parameters times bytes, and the cache sits on top. A 70B model at 16-bit is about 140 GB of weights before that cache. The recipe, not this page, decides precision and GPU count.

What denv is for on this job

A tested recipe on a GPU you already pay for. vLLM or SGLang. One endpoint. If the traffic is a nightly batch, a cold start of many minutes can be acceptable. The measured cold start, on one fresh H100, was 12 minutes 46 seconds, then 99 seconds once the cache was warm. That was not a document model. It is the shape of the cost. If someone is waiting, keep a minimum replica up. Serverless starts from empty. That is a different product.

When the GPU faults, repair takes it out and failover moves the next document to a healthy replica. A second site is how you ride out a provider with no capacity this week. One cluster is the point. You do not build a second stack per cloud.

denv does not sell the GPU hour and does not sell the token. A dedicated card only pays for itself when it is busy. We have not published the utilization at which a document workload crosses that line.

The product is early. There is no document-AI recipe with a published speed. Write to anup@denv.ai if you want one tested on your GPU type.