Why the requirement changes by runtime
H3 combines a 33B dense omni-transformer, the full Qwen3-VL-32B encoder, a visual VAE, and an audio VAE. Keeping every component resident needs far more accelerator memory than loading the encoder once, moving it away, and streaming transformer or decoder blocks from system memory. Quantization reduces weight memory again, but it changes the precision path.
- Resolution and frame count grow activation memory.
- Ref2VA adds reference tokens and can be heavier than text-only generation.
- Offloading lowers VRAM but increases PCIe or unified-memory traffic.
- Downloading only FL2VA or Ref2VA reduces disk use, not the memory of a loaded task.
Measured configurations, not universal minimums
| Configuration | Measured profile | Reported peak |
|---|---|---|
| 4 x H100 | SGLang, BF16/FP32, 5 s at 1344 x 768 | 49.80 to 66.04 GB per GPU |
| 4 x H200 | SGLang, BF16/FP32, 5 s at 1344 x 768 | 63,490 to 94,290 MB per GPU |
| 8 x B300 | SGLang BF16 or online FP8 | about 52 to 124 GB per GPU by profile |
| 2 x RTX 5090 | SGLang layerwise offload, 5 s at 1344 x 768 | 26.3 GiB per GPU |
| 1 x DGX Spark | Community Diffusers INT8 plus group offload | Fits 128 GB unified memory |
| RTX 4090 Laptop 16 GB | Community ComfyUI repack, 5 s at 960 x 540 | Dynamic loading; 16 GB VRAM |
Plan system RAM and storage with VRAM
Low-VRAM workflows rely on system memory. A machine that technically loads the diffusion model can still stall, swap, or fail during text encoding and video decode. The L40S community benchmark reported 64 GB system RAM as comfortable for its optimized raw-weight path. The DGX Spark guide allocated roughly 144 GB to selected FL2VA files and advised 180 GiB of free storage.
- Use fast local NVMe for model caches and outputs.
- Keep at least tens of gigabytes beyond selected model files for containers, caches, and videos.
- Avoid downloading Ref2VA when the machine will only serve text and keyframe jobs.
- Measure peak host RAM through a complete decode, not only model load.
What to buy for each goal
Do not choose hardware from parameter count alone. Benchmark the exact task family, duration, resolution, references, steps, precision, attention backend, and runtime that will ship.
| Goal | Practical hardware direction |
|---|---|
| Explore locally | A high-memory NVIDIA workstation with ComfyUI repacks and patience |
| Preserve original BF16 path | Use a current measured multi-GPU SGLang recipe |
| Serve several users | Datacenter GPUs, fast interconnect, and measured batching |
| Test one exact machine | Rent that SKU before purchasing and run a fixed canary |