Docs · how to

Deploy a custom model

Serve your own fine-tune from private storage, with LoRA adapters, a recipe that extends a catalog one, and golden prompts.

You have fine-tuned weights based on an open model. This guide serves them through a canary stage.

1. Make the weights reachable and pinned

Where Reference Access
Private Hugging Face repo hf://acme/support-70b@4f2a9c1 weights-auth secret with a read token
S3 s3://acme-models/support-70b@<versionId> Read access on the bucket for the provider role
GCS gs://acme-models/support-70b#<generation> Read access for the provider's service account
Upload to denv denv://support-70b/<version> denv weights upload

For Hugging Face:

denv secret set hf-token --value $HF_TOKEN

Or upload directly:

denv weights upload ./checkpoints/support-70b --model support-70b

2. Pick a recipe and extend it

Start from the catalog recipe for the base model's family and hardware:

denv recipe list --model qwen3 --hardware h200

Extend it when your fine-tune needs different arguments, such as a longer context:

recipes:
  - name: support-70b-h200
    version: 1
    extends: qwen3-235b/sglang@5
    hardware: [h200]
    args: { context-length: 65536 }

denv recipe validate --dry-run
denv recipe validate

See Choose a recipe.

3. Write golden prompts

Golden prompts are checks a replica must pass before it takes traffic. Keep them few and specific to what your fine-tune must get right.

{"messages": [{"role": "user", "content": "Order 4471 status?"}], "expect": {"contains": "lookup_order"}}
{"messages": [{"role": "user", "content": "Reply with the word ok."}], "expect": {"equals": "ok"}}
{"adapter": "billing", "prompt": "Refund policy in one line:", "expect": {"regex": "30 days"}}

Each case runs at temperature 0. See Rollout policy.

4. Describe the release

release:
  model: support-70b
  weights: hf://acme/support-70b@4f2a9c1
  weights-auth: hf-token
  recipe: support-70b-h200@1
  topology: { mode: aggregated, gpus: 8 }
  adapters:
    - name: billing
      weights: s3://acme-models/lora/billing@3Hk2v
  api: [chat-completions, responses]
  health:
    warmup: 32
    golden: tests/golden.jsonl

Callers select an adapter with "model": "support-70b:billing".

5. Dry run, then release

denv release --dry-run

This checks that the weights resolve to a fixed revision, the engine version in the recipe supports the hardware and arguments, and every stage's fleets fit the topology. Then:

denv release

support-70b@v1  sha256:7d0e…a31c

6. Deploy to canary

denv env apply
denv deploy support-70b/canary --release support-70b@v1
denv deployment watch <deployment-id>

Weights are prefetched to the target hosts, the engine starts, warmup runs, then golden prompts. A replica that fails any of them never takes traffic, and the deployment rolls back.

7. Benchmark it

A fine-tune keeps its base model's architecture, so the base model's published results are a starting point, not a measurement of your release. Measure speed, throughput and cost on your own fleet and workload:

denv bench run --recipe support-70b-h200@1 --target <your-h200-fleet> --workload chat

A model with no official API

Parity compares against a reference. For a fine-tune, the reference is usually your own earlier release or a trusted deployment of it, not a public API. See Parity.

Next