Usage
You can find ready-to-use scripts in the diffusers-recipes repository. For 16 GB GPUs and up, use sdnq_prompt_upsampler_model_cpu_offload.py; for 12 GB and 8 GB GPUs, use sdnq_prompt_upsampler_two_stage.py, which writes the caption first and then generates the image.
The upsampler replaces the prompt_upsampler component of the FLUX 3 Image Klein pipeline. Import sdnq before loading so the quantized weights are registered. Decoding is slower than with the bf16 upsampler (about 22 vs 70 tokens/s on an RTX 5090).
For the system prompts, layout boxes and image editing, see the original model card.
Sample image
The INT4 image above was generated with the following request and settings (seed 7):
import sdnq # noqa: F401import torchfrom diffusers import ModularPipelinefrom transformers import AutoProcessor, Qwen3_5ForConditionalGeneration upsampler_id = "OzzyGT/Flux_3_Image_Klein_pup_sdnq_dynamic_4bit" pipe = ModularPipeline.from_pretrained("black-forest-labs/flux-3-image-klein")pipe.update_components( prompt_upsampler=Qwen3_5ForConditionalGeneration.from_pretrained(upsampler_id, dtype=torch.bfloat16), prompt_upsampler_processor=AutoProcessor.from_pretrained(upsampler_id),)pipe.load_components(dtype=torch.bfloat16)pipe.to("cuda") prompt = "A small neighborhood pet shop: a tabby cat on the counter, a golden retriever in a wicker basket and a scarlet macaw perched above an aquarium. The sign over the door reads WHISKERS & WAGS." output = pipe( prompt=prompt, height=1152, width=2048, generator=torch.Generator().manual_seed(7), output=["images", "prompt"],)print(output["prompt"]) # the caption written by the upsampleroutput["images"][0].save("flux3_klein_pup_int4.png")
License
This model is released under the Apache License 2.0, the same as the original. It is derived from Qwen3.5-9B by the Qwen team at Alibaba Cloud.