GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
lite-infer
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
lite-infer
Model Tree
black-forest-labs/FLUX.1-schnell
Input Modalities
Output Modalities
Supported Functionality
| Checkpoint | Latency | Max VRAM |
|---|---|---|
| Converted Diffusers Nunchaku Lite INT4 r32 + BNB4 T5 | 4.13 s (stdev 0.20 s) | 16.40 GiB |
RTX 5090, 1024×1024, 4 steps, guidance scale 0.0, one warmup and three measured runs, full GPU placement. VRAM is peak total device usage sampled with nvidia-smi, including allocations outside PyTorch's caching allocator.
Both images use the same prompt, seed 0, scheduler, resolution, and step count. Native Nunchaku 1.x refuses INT4 checkpoints on Blackwell GPUs, so a same-precision native benchmark was unavailable on the RTX 5090.
Requires the Hugging Face kernels package and a Turing, Ampere, Ada, or Blackwell NVIDIA GPU; Hopper is unsupported for INT4 kernels.
python
import torchfrom diffusers import FluxPipeline pipe = FluxPipeline.from_pretrained( "lite-infer/flux.1-schnell-nunchaku-lite-int4_r32-bnb4-text-encoder", torch_dtype=torch.bfloat16,).to("cuda") image = pipe( prompt='A cinematic photograph of a red fox standing in a misty forest at sunrise, detailed fur, volumetric light', generator=torch.Generator("cuda").manual_seed(0), width=1024, height=1024, num_inference_steps=4, guidance_scale=0.0,).images[0]image.save("output.png")