GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
axteam
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
axteam
Model Tree
Qwen/Qwen3.5-122B-A10B
Input Modalities
Output Modalities
Supported Functionality
FP8_DYNAMIC.model.visual.*), the multi-token-prediction head (mtp.*), all RMSNorms, token embeddings, the LM head, the MoE router (mlp.gate) and shared-expert gate, the SSM non-linear parts (linear_attn.conv1d/A_log/dt_bias). Left unquantized to preserve quality (a small fraction of parameters).chat_template / generation_config are identical to the original. config.json is the original plus a quantization_config block.This is post-training quantization (PTQ) — no retraining, so behaviour stays faithful to the original. FP8 dynamic is calibration-free.
Fidelity was checked against the BF16 original (perplexity + top-1 next-token agreement, internal EN/KO set). 8-bit builds (AWQ INT8 / FP8) stay closest to the original; 4-bit builds trade some quality for footprint — and for the sparse-MoE models the 4-bit gap is larger.
bash
vllm serve axteam/Qwen3.5-122B-A10B-FP8-dynamicRecommended sampling (Qwen):
temperature=0.7, top_p=0.8, top_k=20.
Quantized derivative of Qwen/Qwen3.5-122B-A10B (Apache-2.0). All original weights, tokenizer and chat template belong to the Qwen team.
Qwen/Qwen3.5-122B-A10B 를 FP8 (dynamic) 로 양자화한 서빙용(8-bit) 모델입니다. 작성: axteam.
FP8_DYNAMIC.vllm serve axteam/Qwen3.5-122B-A10B-FP8-dynamic, 권장 샘플링 temperature=0.7, top_p=0.8, top_k=20.