GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
amd
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
amd
Model Tree
Qwen/Qwen3.5-397B-A17B-FP8
Input Modalities
Output Modalities
Supported Functionality
The model was quantized from Qwen/Qwen3.5-397B-A17B-FP8 using AMD-Quark. The weights are quantized to MXFP4 and activations are quantized to MXFP4.
Quantization scripts:
markdown
import osfrom quark.torch import LLMTemplate, ModelQuantizer# Configurationckpt_path = "Qwen/Qwen3.5-397B-A17B-FP8"output_dir = "amd/Qwen3.5-397B-A17B-MXFP4"quant_scheme = "mxfp4"exclude_layers = ["lm_head", "model.visual.*", "mtp.*", "*mlp.gate", "*shared_expert_gate*", "*.linear_attn.*", "*.self_attn.*", "*.shared_expert.*"]# Get quant config from templatetemplate = LLMTemplate.get("qwen3_5_moe")quant_config = template.get_config(scheme=quant_scheme, exclude_layers=exclude_layers)# Quantize with File-to-file modequantizer = ModelQuantizer(quant_config)quantizer.direct_quantize_checkpoint(pretrained_model_path=ckpt_path,save_path=output_dir,)
For further details or issues, please refer to the AMD-Quark documentation or contact the respective developers.
The model was evaluated on gsm8k benchmarks using the vllm framework.
The GSM8K results were obtained using the vLLM framework, based on the Docker image rocm/vllm-dev:nightly_main_20260211, and vLLM is installed inside the container.
markdown
lm_eval \--model vllm \--model_args pretrained=amd/Qwen3.5-397B-A17B-MXFP4,tensor_parallel_size=4,max_model_len=262144,gpu_memory_utilization=0.90,max_gen_toks=2048,trust_remote_code=True,reasoning_parser=qwen3 \--tasks gsm8k --num_fewshot 5 \--batch_size auto
Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.