Benchmark
Table with columns: Checkpoint, Latency, Max VRAM| Checkpoint | Latency | Max VRAM |
|---|
| Converted Diffusers Nunchaku Lite NVFP4 r32 + BNB4 T5 | 1.34 s (stdev 0.03 s) | 16.73 GiB |
| Original Nunchaku NVFP4 r32 + BF16 T5 | 0.81 s (stdev 0.00 s) | 20.99 GiB |
RTX 5090, 1024×1024, 4 steps, guidance scale 0.0, one warmup and three measured runs, full GPU placement. VRAM is peak total device usage sampled with nvidia-smi, including allocations outside PyTorch's caching allocator. The native row uses the base model's BF16 T5 encoder.
Output Comparison
Both images use the same prompt, seed 0, scheduler, resolution, and step count. The native and converted NVFP4 images use identical inputs.
Run
Requires the Hugging Face kernels package and a Blackwell NVIDIA GPU for NVFP4 kernels.
import torch
from diffusers import FluxPipeline
pipe = FluxPipeline.from_pretrained(
"lite-infer/flux.1-schnell-nunchaku-lite-nvfp4_r32-bnb4-text-encoder",
torch_dtype=torch.bfloat16,
).to("cuda")
image = pipe(
prompt='A cinematic photograph of a red fox standing in a misty forest at sunrise, detailed fur, volumetric light',
generator=torch.Generator("cuda").manual_seed(0),
width=1024,
height=1024,
num_inference_steps=4,
guidance_scale=0.0,
).images[0]
image.save("output.png")