GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
axteam
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
axteam
Model Tree
Qwen/Qwen3.6-35B-A3B
Input Modalities
Output Modalities
Supported Functionality
nvfp4-pack-quantized.model.visual.*), the multi-token-prediction head (mtp.*), all RMSNorms, token embeddings, the LM head, the MoE router (mlp.gate) and shared-expert gate, the entire linear-attention (GatedDeltaNet/SSM) path — FP4 is too coarse for the SSM weights, so the full linear_attn.* stays BF16. Left unquantized to preserve quality (a small fraction of parameters).chat_template / generation_config are identical to the original. config.json is the original plus a quantization_config block.This is post-training quantization (PTQ) — no retraining, so behaviour stays faithful to the original. AWQ uses a small calibration set (no gradient) to protect salient weight channels before rounding.
Fidelity to the BF16 original on an internal mixed English/Korean set (100 prompts, BF16 greedy answers scored under each build — same harness for every row).
| Metric | This model | BF16 original |
|---|---|---|
| Perplexity (lower = closer) | 1.6708 | 1.2934 (+29.2%) |
| Top-1 next-token agreement | 83.9% | 98.0% |
These measure fidelity to the base model, not absolute task scores.
⚠️ Note: this sparse-MoE model is sensitive to 4-bit quantization (+29.2% perplexity here). For production we recommend the 8-bit builds (FP8 or AWQ INT8) of this model.
bash
vllm serve axteam/Qwen3.6-35B-A3B-NVFP4A16Recommended sampling (Qwen):
temperature=0.7, top_p=0.8, top_k=20.
Quantized derivative of Qwen/Qwen3.6-35B-A3B (Apache-2.0). All original weights, tokenizer and chat template belong to the Qwen team.
Qwen/Qwen3.6-35B-A3B 를 NVFP4A16 (W4A16, FP4 weights) 로 양자화한 서빙용(4-bit) 모델입니다. 작성: axteam.
nvfp4-pack-quantized.vllm serve axteam/Qwen3.6-35B-A3B-NVFP4A16, 권장 샘플링 temperature=0.7, top_p=0.8, top_k=20.