GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
r3lax
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
r3lax
Model Tree
Base
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Quantized
this model
Input Modalities
Text
Image
Video
Output Modalities
Text
Supported Functionality
Dedicated Endpoints
bash
hf download lyf/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \--local-dir ./qwen36-35b-a3b-hauhaucs-nvfp4
bash
VLLM_NVFP4_GEMM_BACKEND=marlin \vllm serve lyf/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \--served-model-name qwen36-35b-a3b-hauhaucs-nvfp4 \--quantization compressed-tensors \--kv-cache-dtype fp8 \--max-model-len 131072 \--max-num-seqs 1 \--max-num-batched-tokens 4096 \--gpu-memory-utilization 0.90 \--enable-prefix-caching \--enable-auto-tool-choice \--tool-call-parser qwen3_coder \--reasoning-parser qwen3 \--trust-remote-code
Local path quickstart:
bash
hf download lyf/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-NVFP4 \--local-dir ./qwen36-35b-a3b-hauhaucs-nvfp4VLLM_NVFP4_GEMM_BACKEND=marlin \vllm serve ./qwen36-35b-a3b-hauhaucs-nvfp4 \--served-model-name qwen36-35b-a3b-hauhaucs-nvfp4 \--quantization compressed-tensors \--kv-cache-dtype fp8 \--max-model-len 131072 \--max-num-seqs 1 \--max-num-batched-tokens 4096 \--gpu-memory-utilization 0.90 \--enable-prefix-caching \--enable-auto-tool-choice \--tool-call-parser qwen3_coder \--reasoning-parser qwen3 \--trust-remote-code
python
recipe = QuantizationModifier(targets="Linear", scheme="NVFP4",ignore=["lm_head", "re:.*visual.*", "re:.*mlp.gate$","re:.*mlp.shared_expert_gate$", "re:.*linear_attn.*", "re:^mtp.*"],)oneshot(model=model,dataset=ds,recipe=recipe,max_seq_length=1024,num_calibration_samples=128,moe_calibrate_all_experts=True,pipeline="basic",)
HuggingFaceH4/ultrachat_200k, 128 samples x 1024 tokensPipeline:
text
Q8_K_P GGUF -> step1_convert_qwen36_moe.py -> HF bf16 -> step2_quantize_qwen36_moe.py -> NVFP4