GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
amd-satre
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
amd-satre
Model Tree
Qwen/Qwen3.6-35B-A3B
Input Modalities
Output Modalities
Supported Functionality
Quark's built-in qwen3_5_moe recipe quantizes the routed MoE experts only — 30,720
linears (40 layers x 256 experts x 3 projections). 462 modules are excluded and remain
BF16:
lm_head, model.visual.* (vision tower), mtp.*, *mlp.gate, *shared_expert_gate*, *.linear_attn.*, *.self_attn.*, *.shared_expert.*
Because attention, the shared experts, the router and the vision tower all stay BF16, the size reduction is smaller than a whole-model quantization would give.
bash
python examples/torch/language_modeling/llm_ptq/quantize_quark.py \--model_dir Qwen/Qwen3.6-35B-A3B \--device cuda --data_type bfloat16 --model_attn_implementation eager \--dataset pileval --num_calib_data 128 --seq_len 512 --batch_size 1 \--quant_scheme int4_wo_128 \--quant_algo awq --quant_algo_config_file awq <config.json> \--model_export hf_format --export_weight_format real_quantized \--output_dir <out> --skip_evaluation
0.13+50dac6e159a.pileval (mit-han-lab/pile-val-backup), 128 samples, seq_len=512.MODEL_ARCH.LLAMA and supports asymmetric INT4 per-group-32 weight-only on Llama2/Llama3, while this model is Qwen3_5MoeForConditionalGeneration. Safetensors is published instead.