Benchmark
Table with columns: Checkpoint, Latency, Max VRAM| Checkpoint | Latency | Max VRAM |
|---|
| Converted Diffusers Nunchaku Lite INT4 r32 + BNB4 T5 | 4.13 s (stdev 0.20 s) | 16.40 GiB |
RTX 5090, 1024×1024, 4 steps, guidance scale 0.0, one warmup and three measured runs, full GPU placement. VRAM is peak total device usage sampled with nvidia-smi, including allocations outside PyTorch's caching allocator.
Output Comparison
Both images use the same prompt, seed 0, scheduler, resolution, and step count. Native Nunchaku 1.x refuses INT4 checkpoints on Blackwell GPUs, so a same-precision native benchmark was unavailable on the RTX 5090.
Run
Requires the Hugging Face kernels package and a Turing, Ampere, Ada, or Blackwell NVIDIA GPU; Hopper is unsupported for INT4 kernels.
import torch
from diffusers import FluxPipeline
pipe = FluxPipeline.from_pretrained(
"lite-infer/flux.1-schnell-nunchaku-lite-int4_r32-bnb4-text-encoder",
torch_dtype=torch.bfloat16,
).to("cuda")
image = pipe(
prompt='A cinematic photograph of a red fox standing in a misty forest at sunrise, detailed fur, volumetric light',
generator=torch.Generator("cuda").manual_seed(0),
width=1024,
height=1024,
num_inference_steps=4,
guidance_scale=0.0,
).images[0]
image.save("output.png")