GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
DreamFast
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
DreamFast
Model Tree
Qwen/Qwen3-4B
Input Modalities
Output Modalities
Supported Functionality
markdown
model-00001-of-00002.safetensorsmodel-00002-of-00002.safetensorsconfig.jsontokenizer.jsontokenizer_config.json
markdown
comfyui/qwen3-4b-heretic.safetensors # bf16, 7.5GBcomfyui/qwen3-4b-heretic_fp8_e4m3fn.safetensors # fp8 row-wise, 4.2GBcomfyui/qwen3-4b-heretic_int8.safetensors # int8 ConvRot row-wise, 4.2GBcomfyui/qwen3-4b-heretic_int4.safetensors # int4 W4A4 ConvRot, 2.5GBcomfyui/qwen3-4b-heretic_nvfp4.safetensors # nvfp4, 2.7GBcomfyui/qwen3-4b-heretic_mxfp8.safetensors # mxfp8, 4.3GB
Quality: All quantized variants use SVD-guided learned rounding (AdaRound via convert_to_quant), which optimizes each weight's rounding direction to minimize output reconstruction error — noticeably higher fidelity than naive round-to-nearest quantization.
| Quant | Size | Notes |
|---|---|---|
| F16 | ~7.5GB | Lossless reference |
| Q8_0 | ~4GB | Excellent quality |
| Q6_K | ~3GB | Very good quality |
| Q5_K_M | ~2.7GB | Good quality |
| Q4_K_M | ~2.3GB | Recommended balance |
| Q3_K_M | ~1.9GB |
All variants load natively in ComfyUI 0.30.0+ (no plugins) via the comfy_quant metadata embedded in each file.
| Format | Size | Bits | Notes |
|---|---|---|---|
| FP8 (E4M3, row-wise) | 4.2GB | 8 | Best speed/quality balance; works on Ada/Hopper+ |
| INT8 (ConvRot row-wise) | 4.2GB | 8 | Hadamard-rotated; broad GPU support |
| MXFP8 | 4.3GB | 8 | Microscaling FP8 (E8M0 block scales); Blackwell-accelerated |
| INT4 (W4A4 ConvRot) | 2.5GB | 4 |
NVFP4/MXFP8 inference is fastest on Blackwell (RTX 5090/5080, SM100+), but ComfyUI also supports software dequantization on older GPUs (tested working on RTX 4090). INT8 and INT4 both use Hadamard rotation (ConvRot); INT4 W4A4 uses ComfyUI's convrot_w4a4 path.
Download a ComfyUI format file:
comfyui/qwen3-4b-heretic_fp8_e4m3fn.safetensors (4.2GB)comfyui/qwen3-4b-heretic_int4.safetensors (2.5GB)comfyui/qwen3-4b-heretic_nvfp4.safetensors (2.7GB)comfyui/qwen3-4b-heretic_int8.safetensors (4.2GB)comfyui/qwen3-4b-heretic_mxfp8.safetensors (4.3GB)comfyui/qwen3-4b-heretic.safetensors (7.5GB)Place in
python
from transformers import AutoModelForCausalLM, AutoTokenizerimport torchmodel = AutoModelForCausalLM.from_pretrained("DreamFast/qwen3-4b-heretic",device_map="auto",torch_dtype=torch.bfloat16)tokenizer = AutoTokenizer.from_pretrained("DreamFast/qwen3-4b-heretic")prompt = "Describe a dramatic sunset over a cyberpunk city"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)outputs = model.generate(**inputs, max_new_tokens=200)print(tokenizer.decode(outputs[0], skip_special_tokens=True))
bash
llama-server -m qwen3-4b-heretic-Q4_K_M.gguf
Created using Heretic v1.2.0 with 200 optimization trials:
markdown
? Which trial do you want to use?> [Trial 96] Refusals: 3/100, KL divergence: 0.0000 <-- selected[Trial 90] Refusals: 5/100, KL divergence: 0.0000[Trial 95] Refusals: 9/100, KL divergence: 0.0000[Trial 122] Refusals: 90/100, KL divergence: 0.0000...
Trial 96 was selected for having the fewest refusals (3/100) with zero measurable KL divergence, indicating the abliteration surgically removed the refusal mechanism with no damage to model capabilities.
This model is released under the Apache 2.0 License, following the base Qwen 3 4B model license.
| For low VRAM only |
| Smallest; Hadamard-rotated signed INT4 |
| NVFP4 (E2M1) | 2.7GB | 4 | NVIDIA FP4; Blackwell FP4 tensor cores for best perf |
ComfyUI/models/text_encoders/In your Z-Image workflow, use the ClipLoader node and select the heretic file