GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
jamiefutch
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
jamiefutch
Model Tree
Qwen/Qwen3.5-35B-A3B-FP8
Input Modalities
Output Modalities
Supported Functionality
The model was quantized from Qwen/Qwen3.5-35B-A3B-FP8 using AMD-Quark. The weights are quantized to MXFP4 and activations are quantized to MXFP4.
Quantization scripts:
text
cd Quark/examples/torch/language_modeling/llm_ptq/export exclude_layers="lm_head model.visual.* mtp.* *mlp.gate *shared_expert_gate* *.linear_attn.* *.self_attn.* *.shared_expert.*"python3 quantize_quark.py --model_dir Qwen/Qwen3.5-35B-A3B-FP8 \ --quant_scheme mxfp4 \ --file2file_quantization \ --exclude_layers $exclude_layers \ --output_dir amd/Qwen3.5-35B-A3B-MXFP4For further details or issues, please refer to the AMD-Quark documentation or contact the respective developers.
The model was evaluated on gsm8k benchmarks using the vllm framework.
The GSM8K results were obtained using the vLLM framework, based on the Docker image [rocm/vllm-dev:nightly_main_20260211], and vLLM is installed inside the container.
docker pull rocm/vllm-dev:nightly_main_20260211
text
lm_eval \ --model vllm \ --model_args pretrained=amd/Qwen3.5-35B-A3B-MXFP4,tensor_parallel_size=4,max_model_len=262144,gpu_memory_utilization=0.90,max_gen_toks=2048,trust_remote_code=True,reasoning_parser=qwen3 \ --tasks gsm8k --num_fewshot 5 \ --batch_size autoModifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.
cd Quark/examples/torch/language_modeling/llm_ptq/export exclude_layers="lm_head model.visual.* mtp.* *mlp.gate *shared_expert_gate* *.linear_attn.* *.self_attn.* *.shared_expert.*"python3 quantize_quark.py --model_dir Qwen/Qwen3.5-35B-A3B-FP8 \ --quant_scheme mxfp4 \ --file2file_quantization \ --exclude_layers $exclude_layers \ --output_dir amd/Qwen3.5-35B-A3B-MXFP4lm_eval \ --model vllm \ --model_args pretrained=amd/Qwen3.5-35B-A3B-MXFP4,tensor_parallel_size=4,max_model_len=262144,gpu_memory_utilization=0.90,max_gen_toks=2048,trust_remote_code=True,reasoning_parser=qwen3 \ --tasks gsm8k --num_fewshot 5 \ --batch_size auto