Read the workload before the number
A generation-time claim is incomplete without its workload. At minimum, record the output dimensions, exact frame count, denoising steps, task family, reference inputs, weight precision, attention backend, offload policy, warm or cold state, and whether model loading and decode are included.
Published end-to-end results
| Hardware and runtime | Workload | Reported time |
|---|---|---|
| 8 x B300 / SGLang BF16 FL2VA | 5.17 s, 1344 x 768, 50 steps, warm | 19.04 s latency |
| 4 x H200 / SGLang Ulysses4 | 5 s, 1344 x 768, 50 steps, warm resolution | 74.38 s E2E |
| 4 x H100 / SGLang TP2 + Ulysses2 | Published topology comparison | 13.25 s pipeline latency |
| 2 x RTX 5090 / SGLang offload | 5 s, 1344 x 768, 50 steps | 559.67 s E2E |
| DGX Spark / Diffusers INT8 offload | 5.17 s, 768 x 576, 10 grid points | about 543 s optimized average |
| L40S / optimized raw weights | 10 s, 960 x 544 | about 7.2 min |
| RTX 4090 Laptop / ComfyUI repack | 5 s, 960 x 540, 20 steps | 182 s E2E |
The largest performance levers
The L40S report is a useful example: keeping the text encoder resident starved the denoiser of memory, while streaming the video VAE leaf by leaf made decode much slower than necessary. Better component placement mattered more than a nominally faster card.
- Canvas size and frame count: video token count dominates quickly.
- Component residency: moving the text encoder away before denoising can free enough VRAM to stop transformer blocks streaming every step.
- Video VAE placement: decode can dominate a run when its modules repeatedly cross the bus.
- Parallel topology: tensor, Ulysses, Ring, and FSDP trade memory against communication and latency.
- Approximation: INT8, FP8, few-step LoRAs, caching, and alternate attention need output-quality validation.
Build a benchmark that answers your decision
Use three fixed prompts: one text-only motion shot, one first-frame identity shot, and one reference-heavy shot. Run a preview canvas and the 768p target. Repeat a warm request at least three times, report the median, and inspect the same frames and audio timestamps.
- Separate load, encode, denoise, video decode, audio decode, and mux time.
- Record peak VRAM and host RAM.
- Publish the model revision and all non-default flags.
- Keep quality-changing optimizations in separate rows.