GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
RedHatAI
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
RedHatAI
Model Tree
Qwen/Qwen3.6-35B-A3B
Input Modalities
Output Modalities
Supported Functionality
This model was evaluated on GSM8K-Platinum, MMLU-Pro, IFEval, Math 500, GPQA Diamond, AIME 25, and LiveCodeBench v6 using lm-evaluation-harness and lighteval, served with vLLM using --language-model-only.
| Benchmark | Qwen/Qwen3.6-35B-A3B | RedHatAI/Qwen3.6-35B-A3B-NVFP4 | Recovery (%) |
|---|---|---|---|
| GSM8k Platinum (0-shot) | 95.73 | 96.08 | 100.37 |
| IfEval (0-shot) | 93.09 | 92.45 | 99.31 |
| AIME 2025 | 92.92 | 91.25 | 98.21 |
| GPQA diamond | 84.51 | 84.68 | 100.20 |
| Math 500 | 84.80 | 85.00 | 100.24 |
| Lcb Codegeneration V6 | 77.33 | 74.67 | 96.55 |
| MMLU Pro Chat | 85.32 | 84.70 | 99.28 |
| BFCLv4 Overall | 57.83 | 56.10 | 97.01 |
| BFCLv4 Single Turn | 53.81 | 53.45 | 99.34 |
| BFCLv4 Multi-Turn | 62.25 | 58.13 | 93.38 |
| BFCLv4 Agentic | 49.91 | 49.31 | 98.80 |
| SWEBench Verified | 54.8 | 50.2 | 91.61 |
The results were obtained using the following commands:
The model was served with vLLM using the following command:
text
vllm serve RedHatAI/Qwen3.6-35B-A3B-NVFP4 --reasoning-parser qwen3 --language-model-only --max-model-len 96000Each benchmark was run 3 times with different seeds (42, 1234, 4158), except AIME 25 which used 8 seeds (42, 1234, 4158, 5322, 1356, 9843, 3344, 5678). Scores are averaged across all seeds.
text
lm_eval --model local-chat-completions \ --tasks gsm8k_platinum_cot_llama \ --model_args "model=RedHatAI/Qwen3.6-35B-A3B-NVFP4,max_length=96000,base_url=http://0.0.0.0:8000/v1/chat/completions,num_concurrent=128,max_retries=3,tokenized_requests=False,tokenizer_backend=None,timeout=2400" \ --num_fewshot 0 \ --apply_chat_template \ --output_path results.json \ --seed 42 \ --gen_kwargs "do_sample=true,temperature=1.0,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000,presence_penalty=1.5,repetition_penalty=1.0,seed=42"text
lm_eval --model local-chat-completions \ --tasks ifeval \ --model_args "model=RedHatAI/Qwen3.6-35B-A3B-NVFP4,max_length=96000,base_url=http://0.0.0.0:8000/v1/chat/completions,num_concurrent=128,max_retries=3,tokenized_requests=False,tokenizer_backend=None,timeout=2400" \ --apply_chat_template \ --output_path results.json \ --seed 42 \ --gen_kwargs "do_sample=true,temperature=1.0,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000,presence_penalty=1.5,repetition_penalty=1.0,seed=42"text
lm_eval --model local-chat-completions \ --tasks mmlu_pro_chat \ --model_args "model=RedHatAI/Qwen3.6-35B-A3B-NVFP4,max_length=96000,base_url=http://0.0.0.0:8000/v1/chat/completions,num_concurrent=128,max_retries=3,tokenized_requests=False,tokenizer_backend=None,timeout=3600" \ --num_fewshot 0 \ --apply_chat_template \ --output_path results.json \ --seed 42 \ --gen_kwargs "do_sample=true,temperature=1.0,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000,presence_penalty=1.5,repetition_penalty=1.0,seed=42"litellm_config.yaml:
yaml
model_parameters: provider: "hosted_vllm" model_name: "hosted_vllm/RedHatAI/Qwen3.6-35B-A3B-NVFP4" base_url: "http://0.0.0.0:8000/v1" api_key: "" timeout: 2400 concurrent_requests: 64 generation_parameters: temperature: 1.0 max_new_tokens: 64000 top_p: 0.95 top_k: 20 min_p: 0.0 presence_penalty: 1.5 repetition_penalty: 1.0 seed: 0text
lighteval endpoint litellm litellm_config.yaml \ "math_500|0,gpqa:diamond|0,lcb:codegeneration_v6|0" \ --output-dir results \ --save-detailstext
lighteval endpoint litellm litellm_config.yaml \ "aime25|0" \ --output-dir results \ --save-detailsBFCL requires the model to be registered in the leaderboard codebase before running evaluation.
Step 1 — Register the model in bfcl_eval/constants/model_config.py
Add the following entry to api_inference_model_map:
python
"Qwen3.6-35B-A3B-NVFP4": ModelConfig( model_name="Qwen3.6-35B-A3B-NVFP4", display_name="Qwen3.6-35B-A3B-NVFP4 (FC)", url="https://huggingface.co/RedHatAI/Qwen3.6-35B-A3B-NVFP4", org="Google", license="Apache 2.0", model_handler=OpenAICompletionsHandler, input_price=None, output_price=None, is_fc_model=True, underscore_to_dot=True,),Step 2 — Add the key to bfcl_eval/constants/supported_models.py
Add "Qwen3.6-35B-A3B-NVFP4" to the SUPPORTED_MODELS list.
Step 3 — Start the vLLM server (use the command at the top of this section; the --served-model-name flag ensures BFCL can find the model by its registered slug).
Step 4 — Generate responses and evaluate
text
bfcl generate --model Qwen3.6-35B-A3B-NVFP4 --test-category allbfcl evaluate --model Qwen3.6-35B-A3B-NVFP4 --test-category alllm_eval --model local-chat-completions \ --tasks gsm8k_platinum_cot_llama \ --model_args "model=RedHatAI/Qwen3.6-35B-A3B-NVFP4,max_length=96000,base_url=http://0.0.0.0:8000/v1/chat/completions,num_concurrent=128,max_retries=3,tokenized_requests=False,tokenizer_backend=None,timeout=2400" \ --num_fewshot 0 \ --apply_chat_template \ --output_path results.json \ --seed 42 \ --gen_kwargs "do_sample=true,temperature=1.0,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000,presence_penalty=1.5,repetition_penalty=1.0,seed=42"lm_eval --model local-chat-completions \ --tasks ifeval \ --model_args "model=RedHatAI/Qwen3.6-35B-A3B-NVFP4,max_length=96000,base_url=http://0.0.0.0:8000/v1/chat/completions,num_concurrent=128,max_retries=3,tokenized_requests=False,tokenizer_backend=None,timeout=2400" \ --apply_chat_template \ --output_path results.json \ --seed 42 \ --gen_kwargs "do_sample=true,temperature=1.0,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000,presence_penalty=1.5,repetition_penalty=1.0,seed=42"lm_eval --model local-chat-completions \ --tasks mmlu_pro_chat \ --model_args "model=RedHatAI/Qwen3.6-35B-A3B-NVFP4,max_length=96000,base_url=http://0.0.0.0:8000/v1/chat/completions,num_concurrent=128,max_retries=3,tokenized_requests=False,tokenizer_backend=None,timeout=3600" \ --num_fewshot 0 \ --apply_chat_template \ --output_path results.json \ --seed 42 \ --gen_kwargs "do_sample=true,temperature=1.0,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000,presence_penalty=1.5,repetition_penalty=1.0,seed=42"lighteval endpoint litellm litellm_config.yaml \ "math_500|0,gpqa:diamond|0,lcb:codegeneration_v6|0" \ --output-dir results \ --save-detailslighteval endpoint litellm litellm_config.yaml \ "aime25|0" \ --output-dir results \ --save-details"Qwen3.6-35B-A3B-NVFP4": ModelConfig( model_name="Qwen3.6-35B-A3B-NVFP4", display_name="Qwen3.6-35B-A3B-NVFP4 (FC)", url="https://huggingface.co/RedHatAI/Qwen3.6-35B-A3B-NVFP4", org="Google", license="Apache 2.0", model_handler=OpenAICompletionsHandler, input_price=None, output_price=None, is_fc_model=True, underscore_to_dot=True,),model_parameters: provider: "hosted_vllm" model_name: "hosted_vllm/RedHatAI/Qwen3.6-35B-A3B-NVFP4" base_url: "http://0.0.0.0:8000/v1" api_key: "" timeout: 2400 concurrent_requests: 64 generation_parameters: temperature: 1.0 max_new_tokens: 64000 top_p: 0.95 top_k: 20 min_p: 0.0 presence_penalty: 1.5 repetition_penalty: 1.0 seed: 0