GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
cloud19
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
cloud19
Model Tree
joyfox/JoyFox-Qwen3.6-35B-A3B-RP-Aggressive
Input Modalities
Output Modalities
Supported Functionality
The following modules were excluded from quantization: lm_head, embeddings,
the vision tower (model.visual.*), the Gated DeltaNet linear-attention layers
(linear_attn.*), MoE routers (mlp.gate), the shared-expert gate
(shared_expert_gate) and the MTP head (mtp.*). The ignore list matches the
reference RedHatAI/Qwen3.6-35B-A3B-FP8-dynamic quantization.
python
from transformers import AutoProcessor, Qwen3_5MoeForConditionalGenerationfrom llmcompressor import oneshotfrom llmcompressor.modifiers.quantization import QuantizationModifierMODEL_ID = "joyfox/JoyFox-Qwen3.6-35B-A3B-RP-Aggressive"model = Qwen3_5MoeForConditionalGeneration.from_pretrained(MODEL_ID, dtype="auto")processor = AutoProcessor.from_pretrained(MODEL_ID)recipe = QuantizationModifier(targets="Linear",scheme="FP8_DYNAMIC",ignore=["re:.*lm_head","re:visual.*","re:model.visual.*","re:.*mlp.gate$","re:.*embed_tokens$","re:.*shared_expert_gate$","re:.*linear_attn.*","re:^mtp.*",],)oneshot(model=model, recipe=recipe)model.save_pretrained("out")processor.save_pretrained("out")
bash
vllm serve cloud19/JoyFox-Qwen3.6-35B-A3B-RP-Aggressive-FP8-Dynamic \--served-model-name joyfox \--enable-chunked-prefill \--max-model-len 16384 --kv-cache-dtype fp8 \--limit-mm-per-prompt '{"image": 0, "audio": 0, "video": 0}' \--default-chat-template-kwargs '{"enable_thinking": false}'
Notes:
align; passing
--no-enable-chunked-prefill makes the engine refuse to start.model_mtp.safetensors, but speculative decoding
is intentionally not used here.<think>) is disabled at the chat-template level via
enable_thinking: false.