GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
cuio
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
cuio
Model Tree
black-forest-labs/FLUX.1-dev
Input Modalities
Output Modalities
Supported Functionality
ComfyUI/custom_nodes folder!!! You're encouraged to adjust the eta parameter to get better results 🌟!ComfyUI/custom_nodes folder!!!Hyper-SD Scribble demo host on 🤗 scribble
Hyper-SDXL One-step Text-to-Image demo host on 🤗 T2I
Hyper-SD is one of the new State-of-the-Art diffusion model acceleration techniques. In this repository, we release the models distilled from FLUX.1-dev, SD3-Medium, SDXL Base 1.0 and Stable-Diffusion v1-5。
Hyper-FLUX.1-dev-Nsteps-lora.safetensors: Lora checkpoint, for FLUX.1-dev-related models.Hyper-SD3-Nsteps-CFG-lora.safetensors: Lora checkpoint, for SD3-related models.Hyper-SDXL-Nstep-lora.safetensors: Lora checkpoint, for SDXL-related models.Hyper-SD15-Nstep-lora.safetensors: Lora checkpoint, for SD1.5-related models.Hyper-SDXL-1step-unet.safetensors: Unet checkpoint distilled from SDXL-Base.python
import torchfrom diffusers import FluxPipelinefrom huggingface_hub import hf_hub_downloadbase_model_id = "black-forest-labs/FLUX.1-dev"repo_name = "ByteDance/Hyper-SD"# Take 8-steps lora as an exampleckpt_name = "Hyper-FLUX.1-dev-8steps-lora.safetensors"# Load model, please fill in your access tokens since FLUX.1-dev repo is a gated model.pipe = FluxPipeline.from_pretrained(base_model_id, token="xxx")pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))pipe.fuse_lora(lora_scale=0.125)pipe.to("cuda", dtype=torch.float16)image=pipe(prompt="a photo of a cat", num_inference_steps=8, guidance_scale=3.5).images[0]image.save("output.png")
python
import torchfrom diffusers import StableDiffusion3Pipelinefrom huggingface_hub import hf_hub_downloadbase_model_id = "stabilityai/stable-diffusion-3-medium-diffusers"repo_name = "ByteDance/Hyper-SD"# Take 8-steps lora as an exampleckpt_name = "Hyper-SD3-8steps-CFG-lora.safetensors"# Load model, please fill in your access tokens since SD3 repo is a gated model.pipe = StableDiffusion3Pipeline.from_pretrained(base_model_id, token="xxx")pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))pipe.fuse_lora(lora_scale=0.125)pipe.to("cuda", dtype=torch.float16)image=pipe(prompt="a photo of a cat", num_inference_steps=8, guidance_scale=5.0).images[0]image.save("output.png")
Take the 2-steps LoRA as an example, you can also use other LoRAs for the corresponding inference steps setting.
python
import torchfrom diffusers import DiffusionPipeline, DDIMSchedulerfrom huggingface_hub import hf_hub_downloadbase_model_id = "stabilityai/stable-diffusion-xl-base-1.0"repo_name = "ByteDance/Hyper-SD"# Take 2-steps lora as an exampleckpt_name = "Hyper-SDXL-2steps-lora.safetensors"# Load model.pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16, variant="fp16").to("cuda")pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))pipe.fuse_lora()# Ensure ddim scheduler timestep spacing set as trailing !!!pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")# lower eta results in more detailprompt="a photo of a cat"image=pipe(prompt=prompt, num_inference_steps=2, guidance_scale=0).images[0]
You can flexibly adjust the number of inference steps and eta value to achieve best performance.
python
import torchfrom diffusers import DiffusionPipeline, TCDSchedulerfrom huggingface_hub import hf_hub_downloadbase_model_id = "stabilityai/stable-diffusion-xl-base-1.0"repo_name = "ByteDance/Hyper-SD"ckpt_name = "Hyper-SDXL-1step-lora.safetensors"# Load model.pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16, variant="fp16").to("cuda")pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))pipe.fuse_lora()# Use TCD scheduler to achieve better image qualitypipe.scheduler = TCDScheduler.from_config(pipe.scheduler.config)# Lower eta results in more detail for multi-steps inferenceeta=1.0prompt="a photo of a cat"image=pipe(prompt=prompt, num_inference_steps=1, guidance_scale=0, eta=eta).images[0]
Only for the single step inference.
python
import torchfrom diffusers import DiffusionPipeline, UNet2DConditionModel, LCMSchedulerfrom huggingface_hub import hf_hub_downloadfrom safetensors.torch import load_filebase_model_id = "stabilityai/stable-diffusion-xl-base-1.0"repo_name = "ByteDance/Hyper-SD"ckpt_name = "Hyper-SDXL-1step-Unet.safetensors"# Load model.unet = UNet2DConditionModel.from_config(base_model_id, subfolder="unet").to("cuda", torch.float16)unet.load_state_dict(load_file(hf_hub_download(repo_name, ckpt_name), device="cuda"))pipe = DiffusionPipeline.from_pretrained(base_model_id, unet=unet, torch_dtype=torch.float16, variant="fp16").to("cuda")# Use LCM scheduler instead of ddim scheduler to support specific timestep number inputspipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config)# Set start timesteps to 800 in the one-step inference to get better resultsprompt="a photo of a cat"image=pipe(prompt=prompt, num_inference_steps=1, guidance_scale=0, timesteps=[800]).images[0]
Take the 2-steps LoRA as an example, you can also use other LoRAs for the corresponding inference steps setting.
python
import torchfrom diffusers import DiffusionPipeline, DDIMSchedulerfrom huggingface_hub import hf_hub_downloadbase_model_id = "runwayml/stable-diffusion-v1-5"repo_name = "ByteDance/Hyper-SD"# Take 2-steps lora as an exampleckpt_name = "Hyper-SD15-2steps-lora.safetensors"# Load model.pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16, variant="fp16").to("cuda")pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))pipe.fuse_lora()# Ensure ddim scheduler timestep spacing set as trailing !!!pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")prompt="a photo of a cat"image=pipe(prompt=prompt, num_inference_steps=2, guidance_scale=0).images[0]
You can flexibly adjust the number of inference steps and eta value to achieve best performance.
python
import torchfrom diffusers import DiffusionPipeline, TCDSchedulerfrom huggingface_hub import hf_hub_downloadbase_model_id = "runwayml/stable-diffusion-v1-5"repo_name = "ByteDance/Hyper-SD"ckpt_name = "Hyper-SD15-1step-lora.safetensors"# Load model.pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16, variant="fp16").to("cuda")pipe.load_lora_weights(hf_hub_download(repo_name, ckpt_name))pipe.fuse_lora()# Use TCD scheduler to achieve better image qualitypipe.scheduler = TCDScheduler.from_config(pipe.scheduler.config)# Lower eta results in more detail for multi-steps inferenceeta=1.0prompt="a photo of a cat"image=pipe(prompt=prompt, num_inference_steps=1, guidance_scale=0, eta=eta).images[0]
Take Canny Controlnet and 2-steps inference as an example:
python
import torchfrom diffusers.utils import load_imageimport numpy as npimport cv2from PIL import Imagefrom diffusers import ControlNetModel, StableDiffusionXLControlNetPipeline, AutoencoderKL, DDIMSchedulerfrom huggingface_hub import hf_hub_download# Load original imageimage = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/sd_controlnet/hf-logo.png")image = np.array(image)# Prepare Canny Control Imagelow_threshold = 100high_threshold = 200image = cv2.Canny(image, low_threshold, high_threshold)image = image[:, :, None]image = np.concatenate([image, image, image], axis=2)control_image = Image.fromarray(image)control_image.save("control.png")control_weight = 0.5 # recommended for good generalization# Initialize pipelinecontrolnet = ControlNetModel.from_pretrained("diffusers/controlnet-canny-sdxl-1.0",torch_dtype=torch.float16)vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)pipe = StableDiffusionXLControlNetPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, vae=vae, torch_dtype=torch.float16).to("cuda")pipe.load_lora_weights(hf_hub_download("ByteDance/Hyper-SD", "Hyper-SDXL-2steps-lora.safetensors"))# Ensure ddim scheduler timestep spacing set as trailing !!!pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")pipe.fuse_lora()image = pipe("A chocolate cookie", num_inference_steps=2, image=control_image, guidance_scale=0, controlnet_conditioning_scale=control_weight).images[0]image.save('image_out.png')
Take Canny Controlnet as an example:
python
import torchfrom diffusers.utils import load_imageimport numpy as npimport cv2from PIL import Imagefrom diffusers import ControlNetModel, StableDiffusionXLControlNetPipeline, AutoencoderKL, TCDSchedulerfrom huggingface_hub import hf_hub_download# Load original imageimage = load_image("https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/sd_controlnet/hf-logo.png")image = np.array(image)# Prepare Canny Control Imagelow_threshold = 100high_threshold = 200image = cv2.Canny(image, low_threshold, high_threshold)image = image[:, :, None]image = np.concatenate([image, image, image], axis=2)control_image = Image.fromarray(image)control_image.save("control.png")control_weight = 0.5 # recommended for good generalization# Initialize pipelinecontrolnet = ControlNetModel.from_pretrained("diffusers/controlnet-canny-sdxl-1.0",torch_dtype=torch.float16)vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", torch_dtype=torch.float16)pipe = StableDiffusionXLControlNetPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0",controlnet=controlnet, vae=vae, torch_dtype=torch.float16).to("cuda")# Load Hyper-SD15-1step lorapipe.load_lora_weights(hf_hub_download("ByteDance/Hyper-SD", "Hyper-SDXL-1step-lora.safetensors"))pipe.fuse_lora()# Use TCD scheduler to achieve better image qualitypipe.scheduler = TCDScheduler.from_config(pipe.scheduler.config)# Lower eta results in more detail for multi-steps inferenceeta=1.0image = pipe("A chocolate cookie", num_inference_steps=4, image=control_image, guidance_scale=0, controlnet_conditioning_scale=control_weight, eta=eta).images[0]image.save('image_out.png')
Take Canny Controlnet and 2-steps inference as an example:
python
import torchfrom diffusers.utils import load_imageimport numpy as npimport cv2from PIL import Imagefrom diffusers import ControlNetModel, StableDiffusionControlNetPipeline, DDIMSchedulerfrom huggingface_hub import hf_hub_downloadcontrolnet_checkpoint = "lllyasviel/control_v11p_sd15_canny"# Load original imageimage = load_image("https://huggingface.co/lllyasviel/control_v11p_sd15_canny/resolve/main/images/input.png")image = np.array(image)# Prepare Canny Control Imagelow_threshold = 100high_threshold = 200image = cv2.Canny(image, low_threshold, high_threshold)image = image[:, :, None]image = np.concatenate([image, image, image], axis=2)control_image = Image.fromarray(image)control_image.save("control.png")# Initialize pipelinecontrolnet = ControlNetModel.from_pretrained(controlnet_checkpoint, torch_dtype=torch.float16)pipe = StableDiffusionControlNetPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", controlnet=controlnet, torch_dtype=torch.float16).to("cuda")pipe.load_lora_weights(hf_hub_download("ByteDance/Hyper-SD", "Hyper-SD15-2steps-lora.safetensors"))pipe.fuse_lora()# Ensure ddim scheduler timestep spacing set as trailing !!!pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config, timestep_spacing="trailing")image = pipe("a blue paradise bird in the jungle", num_inference_steps=2, image=control_image, guidance_scale=0).images[0]image.save('image_out.png')
Take Canny Controlnet as an example:
python
import torchfrom diffusers.utils import load_imageimport numpy as npimport cv2from PIL import Imagefrom diffusers import ControlNetModel, StableDiffusionControlNetPipeline, TCDSchedulerfrom huggingface_hub import hf_hub_downloadcontrolnet_checkpoint = "lllyasviel/control_v11p_sd15_canny"# Load original imageimage = load_image("https://huggingface.co/lllyasviel/control_v11p_sd15_canny/resolve/main/images/input.png")image = np.array(image)# Prepare Canny Control Imagelow_threshold = 100high_threshold = 200image = cv2.Canny(image, low_threshold, high_threshold)image = image[:, :, None]image = np.concatenate([image, image, image], axis=2)control_image = Image.fromarray(image)control_image.save("control.png")# Initialize pipelinecontrolnet = ControlNetModel.from_pretrained(controlnet_checkpoint, torch_dtype=torch.float16)pipe = StableDiffusionControlNetPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", controlnet=controlnet, torch_dtype=torch.float16).to("cuda")# Load Hyper-SD15-1step lorapipe.load_lora_weights(hf_hub_download("ByteDance/Hyper-SD", "Hyper-SD15-1step-lora.safetensors"))pipe.fuse_lora()# Use TCD scheduler to achieve better image qualitypipe.scheduler = TCDScheduler.from_config(pipe.scheduler.config)# Lower eta results in more detail for multi-steps inferenceeta=1.0image = pipe("a blue paradise bird in the jungle", num_inference_steps=1, image=control_image, guidance_scale=0, eta=eta).images[0]image.save('image_out.png')
Hyper-SDXL-Nsteps-lora.safetensors: text-to-image workflowHyper-SD15-Nsteps-lora.safetensors: text-to-image workflowHyper-SDXL-1step-Unet-Comfyui.fp16.safetensors: text-to-image workflow
ComfyUI/custom_nodes to enable sampling from 800 timestep instead of 999.bibtex
@misc{ren2024hypersd,title={Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesis},author={Yuxi Ren and Xin Xia and Yanzuo Lu and Jiacheng Zhang and Jie Wu and Pan Xie and Xing Wang and Xuefeng Xiao},year={2024},eprint={2404.13686},archivePrefix={arXiv},primaryClass={cs.CV}}
ComfyUI/custom_nodes/ComfyUI-HyperSDXL1StepUnetSchedulerHyper-SD15-1step-lora.safetensors: text-to-image workflowHyper-SDXL-1step-lora.safetensors: text-to-image workflow
ComfyUI/custom_nodes to enable TCDScheduler with support of different inference steps (1~8) using single checkpoint.ComfyUI/custom_nodes/ComfyUI-TCD folder exist.