Evaluation
Evaluation on IFeval/GSM8K of both the original and REAM model is done as:
python -m vllm.entrypoints.openai.api_server --model $model \
--tensor-parallel-size 4 \
--dtype auto \
--gpu-memory-utilization 0.9 \
--reasoning-parser qwen3 \
--port $PORT \
--max-model-len 65536
python -m lm_eval --model local-chat-completions \
--model_args base_url=http://localhost:$PORT/v1/chat/completions,model=${model},tokenized_requests=False \
--tasks ifeval \
--batch_size 1 \
--apply_chat_template \
--gen_kwargs '{"chat_template_kwargs": {"enable_thinking": false}, "max_tokens": 8192}'
For HumanEval, it is evaluated as in Qwen3-30B-A3B-Instruct-2507-REAM.
Results
Table with columns: Model, IFeval, GSM8K, HumanEval, AVG| Model | IFeval | GSM8K | HumanEval | AVG |
|---|
| Qwen3.5-122B-A10B | 94.7 | 97.0 | 95.1 | 95.6 |
| Qwen3.5-122B-A10B-REAM | 93.4 | 96.3 | 93.3 | 94.3 |
License
Please refer to the license of the original model Qwen/Qwen3.5-122B-A10B.