GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
r3lax
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
r3lax
Model Tree
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Input Modalities
Output Modalities
Supported Functionality
bash
hf download lyf/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \ --local-dir ./qwen36-35b-a3b-hauhaucs-nvfp4bash
VLLM_NVFP4_GEMM_BACKEND=marlin \vllm serve lyf/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \ --served-model-name qwen36-35b-a3b-hauhaucs-nvfp4 \ --quantization compressed-tensors \ --kv-cache-dtype fp8 \ --max-model-len 131072 \ --max-num-seqs 1 \ --max-num-batched-tokens 4096 \ --gpu-memory-utilization 0.90 \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3 \ --trust-remote-codeLocal path quickstart:
bash
hf download lyf/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \ --local-dir ./qwen36-35b-a3b-hauhaucs-nvfp4 VLLM_NVFP4_GEMM_BACKEND=marlin \vllm serve ./qwen36-35b-a3b-hauhaucs-nvfp4 \ --served-model-name qwen36-35b-a3b-hauhaucs-nvfp4 \ --quantization compressed-tensors \ --kv-cache-dtype fp8 \ --max-model-len 131072 \ --max-num-seqs 1 \ --max-num-batched-tokens 4096 \ --gpu-memory-utilization 0.90 \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3 \ --trust-remote-codepython
recipe = QuantizationModifier( targets="Linear", scheme="NVFP4", ignore=["lm_head", "re:.*visual.*", "re:.*mlp.gate$", "re:.*mlp.shared_expert_gate$", "re:.*linear_attn.*", "re:^mtp.*"],)oneshot( model=model, dataset=ds, recipe=recipe, max_seq_length=1024, num_calibration_samples=128, moe_calibrate_all_experts=True, pipeline="basic",)HuggingFaceH4/ultrachat_200k, 128 samples x 1024 tokensPipeline:
text
Q8_K_P GGUF -> step1_convert_qwen36_moe.py -> HF bf16 -> step2_quantize_qwen36_moe.py -> NVFP4hf download lyf/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \ --local-dir ./qwen36-35b-a3b-hauhaucs-nvfp4VLLM_NVFP4_GEMM_BACKEND=marlin \vllm serve lyf/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \ --served-model-name qwen36-35b-a3b-hauhaucs-nvfp4 \ --quantization compressed-tensors \ --kv-cache-dtype fp8 \ --max-model-len 131072 \ --max-num-seqs 1 \ --max-num-batched-tokens 4096 \ --gpu-memory-utilization 0.90 \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3 \ --trust-remote-codehf download lyf/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \ --local-dir ./qwen36-35b-a3b-hauhaucs-nvfp4 VLLM_NVFP4_GEMM_BACKEND=marlin \vllm serve ./qwen36-35b-a3b-hauhaucs-nvfp4 \ --served-model-name qwen36-35b-a3b-hauhaucs-nvfp4 \ --quantization compressed-tensors \ --kv-cache-dtype fp8 \ --max-model-len 131072 \ --max-num-seqs 1 \ --max-num-batched-tokens 4096 \ --gpu-memory-utilization 0.90 \ --enable-prefix-caching \ --enable-auto-tool-choice \ --tool-call-parser qwen3_coder \ --reasoning-parser qwen3 \ --trust-remote-coderecipe = QuantizationModifier( targets="Linear", scheme="NVFP4", ignore=["lm_head", "re:.*visual.*", "re:.*mlp.gate$", "re:.*mlp.shared_expert_gate$", "re:.*linear_attn.*", "re:^mtp.*"],)oneshot( model=model, dataset=ds, recipe=recipe, max_seq_length=1024, num_calibration_samples=128, moe_calibrate_all_experts=True, pipeline="basic",)