GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
amd
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
amd
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
| Benchmark | Setting | This model (AWQ) | BF16 base | Recovery % |
|---|---|---|---|---|
| GSM8K, 5-shot (flexible-extract / strict-match) | Thinking: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0, max_gen_toks=3072 | 94.996% / 95.30% | 93.33% / 93.33% | 101.8% |
| GSM8K, 5-shot (flexible-extract / strict-match) | Non-thinking: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0, max_gen_toks=1024 | 89.92% / 89.76% | 90.67% / 89.76% | 99.2% |
Recovery % is computed on the flexible-extract metric (this model / bf16 base).
Thinking mode:
bash
lm-eval run --model local-completions \--model_args model=awq,base_url=http://localhost:8002/v1/completions,tokenizer=amd/Qwen3.8-27B-Quark-AWQ-MXFP4,num_concurrent=16,max_retries=5,timeout=900,tokenized_requests=False,max_length=16384 \--gen_kwargs max_gen_toks=3072 temperature=1.0 top_p=0.95 top_k=20 min_p=0.0 presence_penalty=0.0 repetition_penalty=1.0 do_sample=True \--tasks gsm8k \--output_path ./results/awq_gsm8k \--log_samples
Non-thinking mode: the raw-completions endpoint used above doesn't support passing enable_thinking=False through a chat template, so non-thinking mode is approximated with a custom task that pre-closes an empty <think></think> block in the prompt:
gsm8k_nothink.yaml:
yaml
tag:- math_word_problemstask: gsm8k_nothinkdataset_path: openai/gsm8kdataset_name: mainoutput_type: generate_untiltraining_split: trainfewshot_split: traintest_split: testdoc_to_text: "Question: {{question}}\nAnswer:<think>\n\n</think>\n\n"doc_to_target: "{{answer}}"metric_list:- metric: exact_matchaggregation: meanhigher_is_better: trueignore_case: trueignore_punctuation: falseregexes_to_ignore:- ","- "\\$"- "(?s).*#### "- "\\.$"generation_kwargs:until:- "Question:"- "</s>"- "<|im_end|>"do_sample: falsetemperature: 0.0repeats: 1num_fewshot: 5filter_list:- name: "strict-match"filter:- function: "regex"regex_pattern: "#### (\\-?[0-9\\.\\,]+)"- function: "take_first"- name: "flexible-extract"filter:- function: "regex"group_select: -1regex_pattern: "(-?[$0-9.,]{2,})|(-?[0-9]+)"- function: "take_first"metadata:version: 3.0
bash
lm-eval run --model local-completions \--model_args model=awq,base_url=http://localhost:8002/v1/completions,tokenizer=amd/Qwen3.8-27B-Quark-AWQ-MXFP4,num_concurrent=16,max_retries=5,timeout=900,tokenized_requests=False,max_length=16384 \--include_path ./custom_tasks \--gen_kwargs max_gen_toks=1024 temperature=0.7 top_p=0.80 top_k=20 min_p=0.0 presence_penalty=1.5 repetition_penalty=1.0 do_sample=True \--tasks gsm8k_nothink \--output_path ./results/awq_gsm8k_nothink \--log_samples
bash
HIP_VISIBLE_DEVICES=<gpu> python3 quantize_quark.py \--model_dir Qwen/Qwen3.8-27B \--output_dir ./Qwen3.8-27B-w_mxfp4_a_mxfp4-awq \--quant_scheme mxfp4 \--num_calib_data 128 \--seq_len 512 \--quant_algo awq \--model_export hf_format \--data_type auto \--device cuda \--trust_remote_code
bash
vllm serve amd/Qwen3.8-27B-Quark-AWQ-MXFP4 \--served-model-name awq \--trust-remote-code \--tensor-parallel-size 1 \--max-model-len 16384
Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.